Skip to content

[SPARK-33688][SQL] Migrate SHOW TABLE EXTENDED to new resolution framework - #30645

Closed
MaxGekk wants to merge 5 commits into
apache:masterfrom
MaxGekk:show-table-extended-statement
Closed

[SPARK-33688][SQL] Migrate SHOW TABLE EXTENDED to new resolution framework#30645
MaxGekk wants to merge 5 commits into
apache:masterfrom
MaxGekk:show-table-extended-statement

Conversation

@MaxGekk

@MaxGekkMaxGekk commented Dec 7, 2020

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

  1. Remove old statement ShowTableStatement
  2. Introduce new command ShowTableExtended for SHOW TABLE EXTENDED.

This PR is the first step of new V2 implementation of SHOW TABLE EXTENDED, see SPARK-33393.

Why are the changes needed?

This is a part of effort to make the relation lookup behavior consistent: SPARK-29900.

Does this PR introduce any user-facing change?

The changes should not affect V1 tables. For V2, Spark outputs the error:

SHOW TABLE EXTENDED is not supported for v2 tables.

How was this patch tested?

By running SHOW TABLE EXTENDED tests:

$ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *ShowTablesSuite"

@MaxGekk

MaxGekk commented Dec 7, 2020

Copy link
Copy Markdown
MemberAuthor

@cloud-fan@imback82@HyukjinKwon Please, review this PR.

@HyukjinKwonHyukjinKwon mentioned this pull request Dec 7, 2020
Closed
@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36964/

@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Test build #132364 has finished for PR 30645 at commit e94f681.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36964/

@MaxGekk

MaxGekk commented Dec 7, 2020

Copy link
Copy Markdown
MemberAuthor

@cloud-fan Don't think the failed test are related to the changes:

org.apache.spark.sql.hive.thriftserver.HiveThriftHttpServerSuite:
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:13884/default?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice;a=avalue;b=bvalue#c=cvalue;d=dvalue: Could not establish connection to jdbc:hive2://localhost:13884/default?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice;a=avalue;b=bvalue#c=cvalue;d=dvalue: HTTP Response code: 500

@cloud-fan

cloud-fan commented Dec 7, 2020

Copy link
Copy Markdown
Contributor

retest this please

imback82
imback82 reviewed Dec 7, 2020
View reviewed changes

@imback82imback82 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment threadsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala Outdated
* Create a [[ShowTableStatement]] command.
* Create a [[ShowTableExtended]] command.
*/
override def visitShowTable(ctx: ShowTableContext): LogicalPlan = withOrigin(ctx) {

@imback82imback82Dec 7, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should we update sqlbase.g4 so that this becomes visitShowTableExtended?

@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36976/

@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36976/

@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36983/

@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36983/

@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Test build #132376 has finished for PR 30645 at commit e94f681.

  • This patch fails SparkR unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Dec 7, 2020

Copy link
Copy Markdown

Test build #132383 has finished for PR 30645 at commit 4d3c06e.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan

cloud-fan commented Dec 8, 2020

Copy link
Copy Markdown
Contributor

retest this please

@SparkQA

SparkQA commented Dec 8, 2020

Copy link
Copy Markdown

Test build #132410 has finished for PR 30645 at commit 4d3c06e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan

cloud-fan commented Dec 8, 2020

Copy link
Copy Markdown
Contributor

thanks, merging to master!

@MaxGekk
MaxGekk deleted the show-table-extended-statement branch February 19, 2021 15:04
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cloud-fancloud-fancloud-fan approved these changes
+1 more reviewer
@imback82imback82imback82 left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@MaxGekk@SparkQA@cloud-fan@imback82