Skip to content

[SPARK-35829][SQL][FOLLOWUP] Use subExprCode to avoid duplicate call of addNewFunction - #33305

Closed
viirya wants to merge 1 commit into
apache:masterfrom
viirya:fix-minor
Closed

[SPARK-35829][SQL][FOLLOWUP] Use subExprCode to avoid duplicate call of addNewFunction#33305
viirya wants to merge 1 commit into
apache:masterfrom
viirya:fix-minor

Conversation

@viirya

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

A followup of #32980. We should use subExprCode to avoid duplicate call of addNewFunction.

Why are the changes needed?

Avoid duplicate all of addNewFunction.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing test.

@viirya

Copy link
Copy Markdown
MemberAuthor

cc @cloud-fan

@SparkQA

Copy link
Copy Markdown

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45444/

@SparkQA

Copy link
Copy Markdown

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45444/

@SparkQA

Copy link
Copy Markdown

Test build #140932 has finished for PR 33305 at commit 57d09e5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@viirya

Copy link
Copy Markdown
MemberAuthor

cc @dongjoon-hyun@maropu

// at least two nodes) as the cost of doing it is expected to be low.

val subExprCode = s"${addNewFunction(fnName, fn)}($INPUT_ROW);"
subexprFunctions += s"${addNewFunction(fnName, fn)}($INPUT_ROW);"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I see...

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addNewFunction will overwrite in the method map. So this won't add new function. But it is better to avoid duplicatie call.

@viirya

Copy link
Copy Markdown
MemberAuthor

Thanks @maropu. Merging to master.

@viirya
viirya deleted the fix-minor branch December 27, 2023 18:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@viirya@SparkQA@maropu