Skip to content

Miscellaneous cleanups - #17189

Merged
alamb merged 6 commits into
apache:mainfrom
findepi:findepi/drop-redundant-param-from-windowfunctiondefinition-return-field-bdd5f7
Aug 15, 2025
Merged

Miscellaneous cleanups#17189
alamb merged 6 commits into
apache:mainfrom
findepi:findepi/drop-redundant-param-from-windowfunctiondefinition-return-field-bdd5f7

Conversation

@findepi

@findepifindepi commented Aug 14, 2025

Copy link
Copy Markdown
Member

See individual commits

@github-actionsgithub-actionsBot added logical-expr Logical plan and expressions physical-plan Changes to the physical-plan crate labels Aug 14, 2025
It became redundant when `input_expr_types: &[DataType]` was replaced
with `input_expr_fields: &[Field]`, i.e. in
577c424 commit. The benefit of passing
in a "field" is that it combines data type and nullable attribute.
@findepi
findepiforce-pushed the findepi/drop-redundant-param-from-windowfunctiondefinition-return-field-bdd5f7 branch from 7c383fc to dbf832aCompareAugust 14, 2025 09:49
@findepi
findepi requested review from alamb and timsaucerAugust 14, 2025 10:40
@github-actionsgithub-actionsBot added sql SQL Planner optimizer Optimizer rules core Core DataFusion crate catalog Related to the catalog crate datasource Changes to the datasource crate labels Aug 14, 2025
In `UserDefinedLogicalNode::check_invariants`, the actual plan to check
for invariants is `self`. The `plan` is always `LogicalPlan::Extension`
and provides no further information. It's confusing.
@findepifindepi added the api change Changes the API exposed to users of the crate label Aug 14, 2025
@findepifindepi changed the title Drop redundant param from WindowFunctionDefinition::return_fieldMiscellaneous cleanupsAug 14, 2025
@github-actionsgithub-actionsBot added substrait Changes to the substrait crate common Related to common crate labels Aug 14, 2025

@timsaucertimsaucer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These all look like very nice cleanups. Thank you for the PR!

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @findepi -- the only thing I am concerned about with this PR is the change to check_invariants. Otherwise this entire PR looks like a very nice improvement

Comment threaddatafusion-examples/examples/pruning.rs
field_qualifiers: vec![None; field_count],
functional_dependencies: FunctionalDependencies::empty(),
};
dfschema.check_names()?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context is that checking names here makes this consistent with the other (fallible) ways to create a DFSchema from a Schema

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW it seems this has caused a regression for some users in 50.0.0 (there are some code paths that violate an invariant). See #17706

Comment threaddatafusion/expr/src/logical_plan/extension.rs Outdated
Comment threaddatafusion/pruning/src/pruning_predicate.rs
@github-actionsgithub-actionsBot removed the substrait Changes to the substrait crate label Aug 14, 2025

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @findepi and @timsaucer

@alamb
alamb merged commit 0a024a2 into apache:mainAug 15, 2025
27 checks passed
@alambalamb removed the api change Changes the API exposed to users of the crate label Aug 15, 2025
@findepi
findepi deleted the findepi/drop-redundant-param-from-windowfunctiondefinition-return-field-bdd5f7 branch August 17, 2025 06:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

catalogRelated to the catalog cratecommonRelated to common cratecoreCore DataFusion cratedatasourceChanges to the datasource cratelogical-exprLogical plan and expressionsoptimizerOptimizer rulesphysical-planChanges to the physical-plan cratesqlSQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@findepi@alamb@timsaucer