D3 - Transitions Failing Silently

I was recently working on integrating a large JavaScript module into an already existing web application and ran into a few problems. The most frustrating of all of them was related to D3, a popular JavaScript charting library. The module that my co-worker and I created was essentially a hand-rolled MVC to gather, prepare, and chart data. After dealing with the inevitable namespace conflicts, I finally got the graph to chart, or did I....

D3 would load, the AJAX call to fetch data would complete successfully, even the axes would render, but the data would not plot!. There weren't any exceptions thrown and there weren't any console log statements, so I didn't have any clues to lead me in the right direction. After inspecting the generated HTML elements I realized that D3 was, in fact, generating the coordinates for the SVG, all of the lines, shapes, etc. However, the opacity was set to 0.000001 for all of those elements. So, within the developer tools I slowly set the opacity level to 1 on all of the relevant elements and I could see the chart as it was intended. I was clueless as to why it would set the opacity incorrectly, but I figured I would just counteract that by explicitly setting it to 1 for all of the relevant elements once the chart had rendered. I did so, but then anytime I interacted with the chart and a transition should occur all of the opacity levels were overwritten and the chart looked like crap again.

I was dumbfounded. The chart worked perfectly when we were developing our module outside of this web application, but I could not get it to render correctly within it. After extensive Googling, I eventually found an article/GitHub issue (https://github.com/mbostock/d3/issues/1302) that described my exact problem and the exact solution. D3 relies on the standard Date object within JavaScript (it's now() method specifically), but the web application that I was integrating my charting module into was using Date.js; which overrides the method and breaks the standard. This was causing all of the transitions to fail...silently (no exceptions, no log statements).

Although I knew that there was a conflict of some kind somewhere, I was pretty close to looking into other charting/graphing libraries. It was that or step through D3; which I just didn't have the patience for at the time. Kudos to baversjo for doing that work and identifying the problem. The point of this post is just to direct more attention to the cause of the error/unexpected behavior, shed some light on the situation, and really just prevent me from dealing with this madness again in the future. :)

And for the record, I REALLY wish that D3 just logged the fact that the object returned by Date.now() was not of the expected type. It would have saved me a lot of time.

Play! Framework: Database First!

I've known about Play!, the Java framework, for a couple of years now. I've played with it a few times going through their tutorial and documentation, but hadn't been serious about using it for a personal project until now.

The majority of their documentation and their tutorials describe how to develop an application with the code-first approach, but I've always been fond of creating applications with the database-first/model-first approach. I feel that it gives you better control of your data and ensures that your data set is completely normalized and devoid of redundant data. That being said, I found myself repeatedly hitting road blocks with this approach which almost led me to reconsider my choice of framework. But I endured because I see a lot of positive things in Play! (mainly the lack of a compile/build cycle). I ended up using Hibernate Tools to reverse engineer the model classes, but it took some tweaking.

I'm writing this post as a reminder to myself of all of the necessary steps I took, so that I can avoid hitting any road blocks in the future. Enjoy.

Gotchas to Remember

1. All members in Play model classes must be public
2. Getters and setters are implied (generated at runtime, but can be explicitly defined if need be)
3. Hibernate annotations must be at the member-level and not at the getter-level
4. All Play model classes that have explicit ID fields must extend GenericModel

General Steps

1. Modify Hibernate Tools freemarker templates to move annotations to the members and not the getters
2. Reverse engineer the mapper files (hbm.xml) from the database using Hibernate Tools with the newly modified templates
3. Add meta tags to the mapper files to extend GenericModel, add imports, etc.
4. Generate POJOs from the newly modified mapper files

Modify Hibernate Tools FreeMarker Templates to Move Annotations

1. Extract "hibernate-tools.jar" to a temp directory (e.g. C:\tmp\hibernateForPlay).
2. Within C:\tmp\hibernateForPlay\template\pojo, create a file named "Ejb3FieldGetAnnotation.ftl"

   This is actually a copy of "Ejb3PropertyGetAnnotation.ftl", but all instances of the word "property" are replaced by "field". This template will be placed in a loop that iterates through all fields instead of properties.

   The resulting file looks something like:

   <#if ejb3>
   <#if pojo.hasIdentifierProperty()>
   <#if field.equals(clazz.identifierProperty)>
    ${pojo.generateAnnIdGenerator()}
   <#-- if this is the id field (getter)-->
   <#-- explicitly set the column name for this field-->
   
   
   
   <#if c2h.isOneToOne(field)>
   ${pojo.generateOneToOneAnnotation(field, cfg)}
   <#elseif c2h.isManyToOne(field)>
   ${pojo.generateManyToOneAnnotation(field)}
   <#--TODO support optional and targetEntity-->    
   ${pojo.generateJoinColumnsAnnotation(field, cfg)}
   <#elseif c2h.isCollection(field)>
   ${pojo.generateCollectionAnnotation(field, cfg)}
   <#else>
   ${pojo.generateBasicAnnotation(field)}
   ${pojo.generateAnnColumnAnnotation(field)}
   
   
   

3. Remove property-level annotations by commenting out the following line from "PojoPropertyAccessors.ftl"

   <#include "GetPropertyAnnotation.ftl"/>

4. Add field-level annotations by adding the following include statement directly before the specified, existing line about getting field modifiers in "PojoFields.ftl"

   <#include "Ejb3FieldGetAnnotation.ftl"/>

   ${pojo.getFieldModifiers(field)}

   The resulting file looks something like:

   <#-- // Fields -->
   
   <#foreach field in pojo.getAllPropertiesIterator()><#if pojo.getMetaAttribAsBool(field, "gen-property", true)> <#if pojo.hasMetaAttribute(field, "field-description")>    /**
        ${pojo.getFieldJavaDoc(field, 0)}
        */
    
        <#include "Ejb3FieldGetAnnotation.ftl"/>
        ${pojo.getFieldModifiers(field)} ${pojo.getJavaTypeName(field, jdk5)} ${field.name}<#if pojo.hasFieldInitializor(field, jdk5)> = ${pojo.getFieldInitialization(field, jdk5)};
   
   
   

5. Remove getters and setters by commenting out the following line from "Pojo.ftl"

   <#include "PojoPropertyAccessors.ftl"/>
   

   Note: This is doing things the Play! way since the getters and setters are implied. If you'd like to keep them explicitly defined, skip this step.

Reverse-engineer the Mapper Files from the Database

1. Connect to your database and create a Hibernate Configuration as described in the attached reference links.
2. Within the Hibernate Code Generation Configuration screen make sure to check "Use custom templates (for custom file generation)" and specify the directory where you saved the custom free marker templates we created in the section above (e.g. C:\tmp\hibernateForPlay).
3. Run the configuration with only the Hibernate XML Mappings (.hbm.xml) exporter selected.

Add Necessary Meta Tags to the Mapper Files

As described in the Hibernate Tools guide you can specify meta tags within mapper files to further control the POJO code generation. To address some of the Play! specific gotchas listed above we are going to add the following (somewhat self-documenting) meta tags to all of our mapper files:

    <meta attribute="extends">GenericModel</meta>
    <meta attribute="extra-import">play.db.jpa.GenericModel</meta>
    <meta attribute="extra-import">play.data.validation.Required</meta>
    <meta attribute="scope-field">public</meta>

These should all be added directly within the class tag of the hiberate-mapping tag.

Generate POJOs from the Newly Modified Mapper Files

1. Within the Hibernate Code Generation screen uncheck "Reverse engineer from JDBC Connection" because we want to use our modified mapper files.
2. Within the Hibernate perspective, right-click on your configuration and click Edit Configuration.
3. Under the Mappings tab at the Edit Configuration screen add our modified mapper files, so that it uses those to generate our POJOs.
4. Run the configuration with only the Domain code (.java) exporter selected.

References:

• http://stackoverflow.com/questions/1861817/hibernate-moving-annotations-from-property-method-level-to-field-level
• http://docs.jboss.org/tools/latest/en/hibernatetools/html_single/index.html
• http://www.wikihow.com/Generate-Hibernate-Pojo-Classes-from-DB-Tables

And Here...We...Go

On this blog I will primarily post about various tech gotchas and tips & tricks, but I hope to eventually write some op-ed style articles about life in the tech world as well.