Skip to content

[SPARK-43087][SQL] Support coalesce buckets in join in AQE - #40731

Closed
wangyum wants to merge 2 commits into
apache:masterfrom
wangyum:SPARK-43087
Closed

[SPARK-43087][SQL] Support coalesce buckets in join in AQE#40731
wangyum wants to merge 2 commits into
apache:masterfrom
wangyum:SPARK-43087

Conversation

@wangyum

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR adds CoalesceBucketsInJoin to AdaptiveSparkPlanExec.queryStagePreparationRules.

Why are the changes needed?

Coalesce buckets in join will not work if AQE is enabled.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Unit test.

@wangyum

Copy link
Copy Markdown
MemberAuthor

cc @cloud-fan

@cloud-fan

Copy link
Copy Markdown
Contributor

there is already a PR doing the same thing: #40688

@wangyum

Copy link
Copy Markdown
MemberAuthor

Close this one.

@wangyumwangyum closed this Apr 11, 2023
@wangyum
wangyum deleted the SPARK-43087 branch April 11, 2023 14:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@wangyum@cloud-fan