Skip to content

Implement Parquet filter pushdown via new filter pushdown APIs - #15769

Merged
berkaysynnada merged 2 commits into
apache:mainfrom
pydantic:parquet-filter-pushdown
May 6, 2025
Merged

Implement Parquet filter pushdown via new filter pushdown APIs#15769
berkaysynnada merged 2 commits into
apache:mainfrom
pydantic:parquet-filter-pushdown

Conversation

@adriangb

@adriangbadriangb commented Apr 18, 2025

Copy link
Copy Markdown
Contributor

Moves predicate pushdown into parquet being something specialized that ListingTable and Parquet to working for any TableProvider and any file format the implements the APIs. The checks for compatibility also happen all within the parquet data source machinery, instead of leaking implementations via supports_filters_pushdown.

Loading
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coreCore DataFusion cratedatasourceChanges to the datasource cratedocumentationImprovements or additions to documentationoptimizerOptimizer rulesprotoRelated to proto cratesqllogictestSQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@adriangb@alamb@berkaysynnada@xudong963@jayzhan211