PARQUET-204: add parquet-schema directory support

[nevillelyh]

No description provided.

[tsdeng]

This is good. +1
Thanks @nevillelyh
Will merge after travis passes

[julienledem]

As this is stateless there could be one static instance of it.
HIDDEN_FILE_FILTER = new HiddenFileFilter();

[rdblue]

Agree

[julienledem]

Thanks for cleaning up all this duplicated code.
+1

[nevillelyh]

I changed it to a static INSTANCE member.

[liancheng]

Should we take schema evolution into account here? I.e., show the merged schemas of all part-files.

[rdblue]

I don't think so. There's no guarantee that there is a single schema for the data and merging all of the schemas into one would be misleading: a union strategy can produce a schema that can't be satisfied (as a column projection) by any of the files. I think it's best to return one or all of the unique schemas, but this is already going slightly beyond what Parquet itself should be doing as a file format. Parquet reads and writes files, while Hive or Kite manages the data as a collection.

[rdblue]

Thanks @nevillelyh!