While the min/max record counts for page size check are configurable via ParquetOutputFormat.MIN_ROW_COUNT_FOR_PAGE_SIZE_CHECK and ParquetOutputFormat.MAX_ROW_COUNT_FOR_PAGE_SIZE_CHECK configs and via ParquetProperties directly, the min/max record counts for block size check are hard coded inside InternalParquetRecordWriter.
These two settings should also be configurable.
Reporter: Pradeep Gollakota
Related issues:
PRs and other links:
Note: This issue was originally created as PARQUET-869. Please see the migration documentation for further details.
While the min/max record counts for page size check are configurable via ParquetOutputFormat.MIN_ROW_COUNT_FOR_PAGE_SIZE_CHECK and ParquetOutputFormat.MAX_ROW_COUNT_FOR_PAGE_SIZE_CHECK configs and via ParquetProperties directly, the min/max record counts for block size check are hard coded inside InternalParquetRecordWriter.
These two settings should also be configurable.
Reporter: Pradeep Gollakota
Related issues:
PRs and other links:
Note: This issue was originally created as PARQUET-869. Please see the migration documentation for further details.