Skip to content

fix: update group by columns for merge phase after spill - #15531

Merged
alamb merged 5 commits into
apache:mainfrom
rluvaton:reproduce-single-error
Apr 3, 2025
Merged

fix: update group by columns for merge phase after spill#15531
alamb merged 5 commits into
apache:mainfrom
rluvaton:reproduce-single-error

Conversation

@rluvaton

@rluvatonrluvaton commented Apr 1, 2025

Copy link
Copy Markdown
Member

Which issue does this PR close?

Rationale for this change

I think the PR below forgot to update the group by expressions:

What changes are included in this PR?

Update group by columns to be based on their index for merging phase

Are these changes tested?

Yes

Are there any user-facing changes?

No

@rluvaton
rluvatonforce-pushed the reproduce-single-error branch from 3fabcc4 to ae94ef4CompareApril 1, 2025 15:34
@rluvaton

Copy link
Copy Markdown
MemberAuthor

The CI failures are infra related...

Comment threaddatafusion/physical-plan/src/aggregates/row_hash.rs Outdated

@compheadcomphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @rluvaton

Co-authored-by: Oleks V <comphead@users.noreply.github.com>
@rluvatonrluvaton mentioned this pull request Apr 2, 2025
39 tasks

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rluvaton and @comphead

FYI @2010YOUY01 who is working on spilling and @korowa who worked on the code in question

Comment threaddatafusion/physical-plan/src/aggregates/mod.rs Outdated
@github-actionsgithub-actionsBot added the core Core DataFusion crate label Apr 3, 2025
@alamb
alamb merged commit 0b061be into apache:mainApr 3, 2025
@alamb

alamb commented Apr 3, 2025

Copy link
Copy Markdown
Contributor

Thank you @rluvaton

@rluvaton
rluvaton deleted the reproduce-single-error branch April 3, 2025 19:10
nirnayroy pushed a commit to nirnayroy/datafusion that referenced this pull request May 2, 2025
* fix: update group by columns for merge phase after spill
fixesapache#15530
* Update datafusion/physical-plan/src/aggregates/row_hash.rs
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
* move test to aggregate_fuzz
---------
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coreCore DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Internal error: PhysicalExpr Column references bound error, Failure in spilling for AggregateMode::Single

3 participants

@rluvaton@alamb@comphead