Describe the enhancement requested
Parquet itself doesn't has a LIST_VIEW type, however, we can read/write a LIST_VIEW inside parquet.
Some notes:
- During loading, I think making a
arrow::list is ok, and converting a List to ListView is so convenient. So maybe we can first support write - For
parquet::arrow::SchemaManifest, maybe it's ok to just store a list as metadata?
TODO:
- For writer, we need support
PathInfo. The logic is so similiar to origin list node, however, we need support a new kind of RangeSelector. We don't need to support a new list because the writer here doesn't make full use of the contigous array - For schema, we need to handle the
parquet/arrow/schema.h well.
Component(s)
C++, Parquet
Describe the enhancement requested
Parquet itself doesn't has a
LIST_VIEWtype, however, we can read/write aLIST_VIEWinside parquet.Some notes:
arrow::listis ok, and converting a List to ListView is so convenient. So maybe we can first support writeparquet::arrow::SchemaManifest, maybe it's ok to just store alistas metadata?TODO:
PathInfo. The logic is so similiar to origin list node, however, we need support a new kind ofRangeSelector. We don't need to support a new list because the writer here doesn't make full use of the contigous arrayparquet/arrow/schema.hwell.Component(s)
C++, Parquet