Skip to content

Add table property to disable/enable parquet column statistics #12770

Description

@huaxiangsun

Feature Request / Improvement

We are creating tables with large size columns (up to hundreds Mbs), these large size columns normally are some bytes where stats (min/max) does not make sense. Today, for Iceberg tables' data files in parquet, columns stats is enabled by default. The stats for these large size columns consumes lots of memory. More background information is at parquet PR.

apache/parquet-java#2521

Similar to metrics, we need table properties to enable/disable parquet column statistics.

Query engine

None

Willingness to contribute

  • I can contribute this improvement/feature independently
  • I would be willing to contribute this improvement/feature with guidance from the Iceberg community
  • I cannot contribute this improvement/feature at this time

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementPR that improves existing functionalitystale

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions