Skip to content

[SPARK-28002][SQL][FOLLOWUP] Fix duplicate CTE error message and add more test cases - #24949

Closed
peter-toth wants to merge 4 commits into
apache:masterfrom
peter-toth:SPARK-28002-follow-up
Closed

[SPARK-28002][SQL][FOLLOWUP] Fix duplicate CTE error message and add more test cases#24949
peter-toth wants to merge 4 commits into
apache:masterfrom
peter-toth:SPARK-28002-follow-up

Conversation

@peter-toth

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This PR adds some more WITH test cases as a follow-up to #24842

How was this patch tested?

Add new UTs.

@HyukjinKwon

Copy link
Copy Markdown
Member

ok to test

@HyukjinKwon

Copy link
Copy Markdown
Member

add to whitelist

@peter-toth

Copy link
Copy Markdown
ContributorAuthor

cc. @gatorsmile, @dongjoon-hyun

@SparkQA

Copy link
Copy Markdown

Test build #106829 has finished for PR 24949 at commit efc96bb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyundongjoon-hyunJun 24, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to @gatorsmile 's comment, could you add a t(x, x) test case, too?

Duplicate names within a single CTE definition are not allowed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to capture the duplicate names earlier and issue a better error message

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I added the t(x, x) case, but it is allowed, and I think it is right to be allowed.
I thought @gatorsmile referred to the

WITH
t(x) AS (SELECT 1),
t(x) AS (SELECT 2)
SELECT * FROM t;

case, which is not.

@gatorsmile we capture the duplicate CTE name during parsing, I fixed the error message though.

If I got something wrong please let me know.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gatorsmile, @dongjoon-hyun do you think the PR is ok now?

@SparkQA

Copy link
Copy Markdown

Test build #106845 has finished for PR 24949 at commit 815bb27.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #106871 has finished for PR 24949 at commit 8fbae02.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@peter-toth

Copy link
Copy Markdown
ContributorAuthor

retest this please

@SparkQA

Copy link
Copy Markdown

Test build #106875 has finished for PR 24949 at commit 8fbae02.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@peter-toth

Copy link
Copy Markdown
ContributorAuthor

@gatorsmile, @dongjoon-hyun is there anything I can add to this PR?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we can have a more meaningful message?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this standard message comes from antlr when the query doesn't fit to the .g4 definition. I'm not sure if it can be fixed.

@peter-tothpeter-toth changed the title [SPARK-28002][SQL][FOLLOWUP] Add more WITH test cases[SPARK-28002][SQL][FOLLOWUP] Fix duplicate CTE error message and add more WITH test casesJul 5, 2019
@peter-tothpeter-toth changed the title [SPARK-28002][SQL][FOLLOWUP] Fix duplicate CTE error message and add more WITH test cases[SPARK-28002][SQL][FOLLOWUP] Fix duplicate CTE error message and add more test casesJul 5, 2019
@peter-toth
peter-tothforce-pushed the SPARK-28002-follow-up branch from 8fbae02 to 0c39fb2CompareJuly 5, 2019 10:45
@SparkQA

Copy link
Copy Markdown

Test build #107277 has finished for PR 24949 at commit 0c39fb2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyundongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Merged to master.
Thank you, @peter-toth , @gatorsmile , @HyukjinKwon .

@peter-toth

Copy link
Copy Markdown
ContributorAuthor

Thanks for the review @dongjoon-hyun, @HyukjinKwon, @gatorsmile!

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@peter-toth@HyukjinKwon@SparkQA@dongjoon-hyun@gatorsmile