[SPARK-17180] [SQL] Fix View Resolution Order in ALTER VIEW AS SELECT by gatorsmile · Pull Request #14746 · apache/spark

gatorsmile · 2016-08-22T06:54:20Z

What changes were proposed in this pull request?

In the current master branch, when users do not specify the database name in the ALTER VIEW AS SELECT command, we always try to alter the permanent view even if the temporary view exists. This PR is to resolve this issue.

After the fix, ALTER VIEW AS SELECT command alters the temporary view if the temp view exists; otherwise, try to alter the permanent view. This order is consistent with another command DROP VIEW, since users are unable to specify the keyword TEMPORARY.

How was this patch tested?

Added test cases.

SparkQA · 2016-08-22T08:54:24Z

Test build #64182 has finished for PR 14746 at commit db45b4c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

gatorsmile · 2016-08-24T21:20:28Z

cc @cloud-fan @yhuai

cloud-fan · 2016-08-25T03:58:15Z

+ withView("testView", "default.testView") {
+ val catalog = spark.sessionState.catalog
+ val oldViewQuery = "SELECT id FROM jt"
+ val newViewQuery = "SELECT id, id1 FROM jt"


nit: remove extra spaces

SparkQA · 2016-08-25T06:41:30Z

Test build #64396 has finished for PR 14746 at commit c5add2c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2016-08-25T07:09:14Z

+ // 1) CREATE VIEW: create a temp view when users explicitly specify the keyword TEMPORARY;
+ // otherwise, create a permanent view no matter whether the temporary view
+ // with the same name exists or not.
+ // 2) ALTER VIEW: alter the temporary view if the temp view exists; otherwise, try to alter


question: how can you tell whether it's CREATE VIEW or ALTER VIEW?

Yeah! The only way is to pass a flag.

it already has 3 flags: allowExisting, replace, isTemporary. We should rethink about it and decide what flags we really need for this command.

True, let me think about it. Thanks!

CREATE (OR REPLACE)? TEMPORARY? VIEW (IF NOT EXISTS)? tableIdentifier identifierCommentList? (COMMENT STRING)? (PARTITIONED ON identifierList)? (TBLPROPERTIES tablePropertyList)? AS query
Each flag corresponds to a keyword in the CREATE VIEW DDL command.
OR REPLACE -> replace
TEMPORARY -> isTemporary
IF NOT EXISTS -> allowExisting
If we want to use the same command CreateViewCommand to process both CREATE VIEW and ALTER VIEW, it sounds reasonable to add a new flag. So far, I have not found a way to combine them.

like CreateMode, can we use SaveMode for replace and allowExisting?

uh, I got it. : ) Copied what @liancheng wrote one month ago and did the update in the last case.
allowExisting replace SaveMode
true false Ignore
false false ErrorIfExists
false true Overwrite
true true AnalysisException
Let me try it. Thanks!

SparkQA · 2016-08-26T08:38:08Z

Test build #64458 has finished for PR 14746 at commit 51dbf72.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2016-08-27T06:51:46Z

 replace: Boolean,
- isTemporary: Boolean): LogicalPlan = {
+ isTemporary: Boolean,
+ isAlterViewAsSelect: Boolean): LogicalPlan = {


how about a viewType parameter which is an enum?
public enum ViewType { Temporary, Permanent, Any }
We are going to add global temp view soon, so this enum will also be useful at that time.

Sure, will do it. Thanks!

SparkQA · 2016-08-27T08:40:57Z

Test build #64526 has finished for PR 14746 at commit d84db62.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-08-28T08:34:52Z

Test build #64547 has finished for PR 14746 at commit 47fdf1f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-08-28T08:53:27Z

Test build #64549 has finished for PR 14746 at commit 5606344.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2016-08-29T12:16:37Z

+/**
+ * ViewType is used to specify the type of views.
+ */
+public enum ViewType {


This doesn't need to be public to end users, we can put it in view.scala and use sealed trait to implement it.

I thought you want me to use public enum. Let me change it now. Thanks!

SparkQA · 2016-08-29T20:27:13Z

Test build #64588 has finished for PR 14746 at commit baa8a1f.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

gatorsmile · 2016-08-29T21:53:12Z

retest this please

SparkQA · 2016-08-29T23:58:48Z

Test build #64603 has finished for PR 14746 at commit baa8a1f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2016-08-30T09:40:32Z

after thinking about it more, I think it's better to use a different code path for ALTER VIEW, instead of sharing the same implementation of CREATE VIEW. Do you mind review #14874? thanks!

gatorsmile · 2016-08-30T16:52:42Z

Yeah, I am also fine about splitting them into two commands. Let me close it now.

fix.
db45b4c

cloud-fan reviewed Aug 25, 2016
View reviewed changes

address comments.
c5add2c

cloud-fan reviewed Aug 25, 2016
View reviewed changes

address comments
51dbf72

address comments.
d84db62

cloud-fan reviewed Aug 27, 2016
View reviewed changes

gatorsmile added 2 commits August 27, 2016 23:32

add ViewType
47fdf1f

remove isAlterViewAsSelect
5606344

cloud-fan reviewed Aug 29, 2016
View reviewed changes

gatorsmile added 2 commits August 29, 2016 09:51

Merge remote-tracking branch 'upstream/master' into alterViewFix
5c60b4c

address comments
baa8a1f

gatorsmile closed this Aug 30, 2016

allowExisting	replace	SaveMode
true	false	Ignore
false	false	ErrorIfExists
false	true	Overwrite
true	true	AnalysisException

Uh oh!

Conversation

gatorsmile commented Aug 22, 2016

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

SparkQA commented Aug 22, 2016

Uh oh!

gatorsmile commented Aug 24, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 25, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gatorsmileAug 27, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 26, 2016

Uh oh!

cloud-fanAug 27, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 27, 2016

Uh oh!

SparkQA commented Aug 28, 2016

Uh oh!

SparkQA commented Aug 28, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 29, 2016

Uh oh!

gatorsmile commented Aug 29, 2016

Uh oh!

SparkQA commented Aug 29, 2016

Uh oh!

cloud-fan commented Aug 30, 2016

Uh oh!

gatorsmile commented Aug 30, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gatorsmileAug 27, 2016•
edited
Loading

cloud-fanAug 27, 2016•
edited
Loading