Skip to content

Fix incorrect SortExec removal before AggregateExec - #20245

Closed
alamb wants to merge 1 commit into
apache:mainfrom
alamb:alamb/fix_monotonic_sort_bug
Closed

Fix incorrect SortExec removal before AggregateExec#20245
alamb wants to merge 1 commit into
apache:mainfrom
alamb:alamb/fix_monotonic_sort_bug

Conversation

@alamb

@alambalamb commented Feb 9, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Wrong answers bug was introduced in 52

What changes are included in this PR?

Fix the bug with a big hammer: simply disable the whole optimizatio

Here is an alternative PR to just disable it in a more targeted setting

Are these changes tested?

Yes, a new test is added

Are there any user-facing changes?

A bug is fixed

@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation development-process Related to development process of DataFusion physical-expr Changes to the physical-expr crates optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) catalog Related to the catalog crate execution Related to the execution crate datasource Changes to the datasource crate physical-plan Changes to the physical-plan crate labels Feb 9, 2026
@alamb
alambforce-pushed the alamb/fix_monotonic_sort_bug branch from 70f8863 to 91df46fCompareFebruary 9, 2026 19:57
@alambalamb changed the title Alamb/fix monotonic sort bugFix group by sorting bugFeb 9, 2026
}
} else if maintains_input_order.is_empty()
|| !maintains_input_order.iter().any(|o| *o)
// Aggregate output columns can be computed expressions that are not

@alambalambFeb 9, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pretty large hammer (just disables pushing grouping requirements through all AggregateExec`s convinced this is the best solution / correct yet

@alambalamb changed the title Fix group by sorting bugFix incorrect SortExec removal before AggregateExecFeb 9, 2026
@alambalamb added optimizer Optimizer rules and removed documentation Improvements or additions to documentation development-process Related to development process of DataFusion physical-expr Changes to the physical-expr crates optimizer Optimizer rules core Core DataFusion crate catalog Related to the catalog crate execution Related to the execution crate datasource Changes to the datasource crate physical-plan Changes to the physical-plan crate labels Feb 10, 2026
@alamb

Copy link
Copy Markdown
ContributorAuthor

I am convinced we should go with option 2

@alambalamb closed this Feb 10, 2026
github-merge-queueBot pushed a commit that referenced this pull request Feb 16, 2026
…20247)
## Which issue does this PR close?
- Fixes#20244
This is an alternatative to - #20245
## Rationale for this change
Wrong answers bug was exposed by
#19287 in 52. See
#20244 and backstory here
- #19287 (comment)
## What changes are included in this PR?
Fix the bug by properly implemnting ## Are these changes tested?
Yes, a new test is added
## Are there any user-facing changes?
A bug is fixed
---------
Co-authored-by: Mustafa Akur <akurmustafa@gmail.com>
alamb added a commit to alamb/datafusion that referenced this pull request Feb 23, 2026
…pache#20247)
- Fixesapache#20244
This is an alternatative to
- apache#20245
Wrong answers bug was exposed by
apache#19287 in 52. See
apache#20244 and backstory here
- apache#19287 (comment)
Fix the bug by properly implemnting
Yes, a new test is added
A bug is fixed
---------
Co-authored-by: Mustafa Akur <akurmustafa@gmail.com>
alamb added a commit to alamb/datafusion that referenced this pull request Feb 23, 2026
…pache#20247)
- Fixesapache#20244
This is an alternatative to
- apache#20245
Wrong answers bug was exposed by
apache#19287 in 52. See
apache#20244 and backstory here
- apache#19287 (comment)
Fix the bug by properly implemnting
Yes, a new test is added
A bug is fixed
---------
Co-authored-by: Mustafa Akur <akurmustafa@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

optimizerOptimizer rulessqllogictestSQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect answers due to an incorrectly removed Sort before AggregateExec

1 participant

@alamb