Conversation
|
Hi, @rdblue , @isnotinvain , I am working on HIVE-10253 and find we may need this change, otherwise filter predicate with Date type cannot pass the validation check. What do you think? Thanks! |
|
We should be adding the mappings for the other types as well (TIME_MILLIS and TIMESTAMP_MILLIS). I am having the same challenge doing PPD for Apache Drill with those types. I can certainly create another JIRA item for those, but it seems trivial to add it to this one. |
|
Sounds good! Thanks. |
|
Verified the updates in my local env again. Build and test successfully. |
There was a problem hiding this comment.
so, for something like this, does it make more sense to use a different java class other than Binary? we could use String for UTF8, and BigNum or something here?
Maybe that could be a higher level API on top of this one though.
There was a problem hiding this comment.
Add this here is just to make filter checking pass if user needs INT96 for a filter.
It seems INT96 is designed as Binary in PrimitiveType. Maybe higher level API could bind it with other java class. Not sure which to use here other than Binary?
|
per: https://issues.apache.org/jira/browse/PARQUET-201 do you mind adding the rest of the primitives too? |
|
Sure, my pleasure! |
|
Fixed by removing OriginalType check in #219 |
When Hive use Parquet filter predicate, the Date type is converted to Integer. In ValidTypeMap, it map the class and Parquet type. It throw exception when checking the data type Date.
We should add the map to support Date.