Skip to content

[SPARK-39259][SQL][3.3] Evaluate timestamps consistently in subqueries - #36752

Closed
olaky wants to merge 1 commit into
apache:branch-3.3from
olaky:SPARK-39259-spark_3_3
Closed

[SPARK-39259][SQL][3.3] Evaluate timestamps consistently in subqueries#36752
olaky wants to merge 1 commit into
apache:branch-3.3from
olaky:SPARK-39259-spark_3_3

Conversation

@olaky

@olakyolaky commented Jun 2, 2022

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Apply the optimizer rule ComputeCurrentTime consistently across subqueries.

This is a backport of #36654.

Why are the changes needed?

At the moment timestamp functions like now() can return different values within a query if subqueries are involved

Does this PR introduce any user-facing change?

No

How was this patch tested?

A new unit test was added

@AmplabJenkins

Copy link
Copy Markdown

Can one of the admins verify this patch?

@MaxGekkMaxGekk changed the title [SPARK-39259] Evaluate timestamps consistently in subqueries[SPARK-39259][SQL][3.3] Evaluate timestamps consistently in subqueriesJun 3, 2022

@MaxGekkMaxGekk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@olaky Could you update PR description, please.

@MaxGekk

Copy link
Copy Markdown
Member

+1, LGTM. Merging to 3.3.
Thank you, @olaky.

MaxGekk pushed a commit that referenced this pull request Jun 3, 2022
### What changes were proposed in this pull request?
Apply the optimizer rule ComputeCurrentTime consistently across subqueries.
This is a backport of #36654.
### Why are the changes needed?
At the moment timestamp functions like now() can return different values within a query if subqueries are involved
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
A new unit test was added
Closes#36752 from olaky/SPARK-39259-spark_3_3.
Authored-by: Ole Sasse <ole.sasse@databricks.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
@MaxGekkMaxGekk closed this Jun 3, 2022
@olaky

olaky commented Jun 3, 2022

Copy link
Copy Markdown
ContributorAuthor

Thanks for picking this up so quickly, I wanted to first see if the build passes and then update the comment

@dongjoon-hyun

Copy link
Copy Markdown
Member

Hi, @olaky and @MaxGekk . Unfortunately, this broke Scala 2.13 in master/branch-3.3 and RC4. I made a PR to fix it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@olaky@AmplabJenkins@MaxGekk@dongjoon-hyun