[SPARK-17206][SQL] Support ANALYZE TABLE on analyzable temporary view

[viirya]

What changes were proposed in this pull request?

Currently ANALYZE TABLE DDL command can't work on temporary view. However, for the specified type of temporary view which is analyzable, we can support the DDL command for it. So the CBO can work with temporary view too.

How was this patch tested?

Jenkins tests.

[viirya]

@hvanhovell Based on the prior discussion, I opened a JIRA and this PR. Can you review it if it is on the right direction? Thanks.

[SparkQA]

Test build #64330 has finished for PR 14780 at commit cfbfefc.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

@hvanhovell@cloud-fan Can you help review this?

[viirya]

cc @hvanhovell@cloud-fan Can you take a look? Thanks.

[cloud-fan]

what's the main benefit to analyze a temp view? I think table analyzing is an expensive operation, for temp views, we can't put the resulting statistics into metastore and it will go away if session terminates.

[viirya]

@cloud-fan I have a discussion with @hvanhovell at #14729 (comment).

I think the main benefit is the CBO can work on all tables. If we can't analyze on a temp view, the query plan involving temp views will not be applied on CBO.

We might not be able to correctly guess the use case of users. I try to image an use case like: If the user creates a temp view on a data source relation as in @hvanhovell's comment, because the data is temporary and will be changed, so the user doesn't need it to be persisted in metastore. But the user needs the query involving the temp view to execute in cost-efficient way with CBO.

[SparkQA]

Test build #64986 has finished for PR 14780 at commit 29e22fe.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #64991 has finished for PR 14780 at commit 6a3aff2.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

ping @hvanhovell@cloud-fan any more thoughts on this?

[viirya]

@hvanhovell Would you like to comment on this? Thanks.

[hvanhovell]

@viirya this seems like a good idea. However, I want to wait with adding this until we have finished merging all the CBO related statistics stuff.

[viirya]

@hvanhovell ok. Thanks!

[SparkQA]

Test build #67746 has finished for PR 14780 at commit 6a3aff2.

• This patch fails Spark unit tests.
• This patch does not merge cleanly.
• This patch adds no public classes.

[viirya]

I close this for now and maybe reopen it when all the CBO related statistics stuff are merged.