ZEPPELIN-268 : Adding Data Validation Feature and Test Specs

[Madhuka]

Data Validation

• Data validator components
  • Generalized (Data validator factory which can be extend-able)
  • Covered the all chart types in Zeppelin
• Test Specs are added
• Documentation (Added PR ZEPPELIN-217 : Adding documentation on adding new chart types and Data validation serivce #203 )[2]

More details can be found in Docs or here[1]

[1] http://madhukaudantha.blogspot.com/2015/08/zeppelin-data-validation-service.html
[2] #203

[Madhuka]

Since PR #152 have CI issue this PR is coming with data validation feature which was PR #152

@bzz, @corneadoug : PR contains data validation feature with covering all the zeppelin charts data models.
@Leemoonsoo : Ready for Merge

[corneadoug]

@Madhuka I'm currently testing this branch:

• - Running a few cases
• - Checking the code
• - Verifying the tests
• - Double Checking the docs

During that time, could you bring #203 into this branch/PR? The docs moved from gh-pages to the zeppelin repository (/docs folder), so it allows us to group the docs in the same PR as the feature.

[Madhuka]

@corneadoug , I will add the docs here.

[corneadoug]

My main concern with this PR is performances:

• We are parsing the full data to make sure it fit the graph format
• The validate function is launched a lot of times, even when not needed (COMMIT_PARAGRAPH after updated the width of the paragraph for example)

We are already limited in term of memory in the web browser. Inspection with ng-stats or batarang didn't really help me getting a better idea on that.

[galleon]

@corneadoug have we measured the overhead in terms of memory/time? If that is a real issue and you believe this will damage the user experience then either we can make it optional or simply disregard it.

[corneadoug]

Haven't really be able to measure the overhead.
But parsing the full query response, can be expensive if the data returned is big.

One thing that could be an alternative however, would be to have some kind of informations tooltip (to show what type is accepted), and maybe improve the error message on the graph when it fails.

[corneadoug]

@Madhuka Should we close this PR?

[Madhuka]

@corneadoug This PR can be useful when we have some different charts models (presenting different data models) This PR need some changes as @galleon says...
This need to optional to execute as user preferred. This will be useful to detected error, missing, etc in data model in Big data analysis.