[SPARK-19799] Support WITH clause (CTE) in subqueries

[gbloisi]

Because of SPARK-17590 support of WITH clause (CTE) in subqueries requires only grammar support.

Test for augmented syntax is provided.

[gatorsmile]

ok to test

[SparkQA]

Test build #98453 has finished for PR 22936 at commit 66cd537.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #98568 has finished for PR 22936 at commit 5377087.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #98569 has finished for PR 22936 at commit 3080ed4.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

Could you add more test cases? For example, can a with clause in a subquery shadow a with clause in an enclosing query with the same name? Another example, use with clauses in a subquery expression?

[gbloisi]

I need to find some spare time for that, but I'll surely do sooner or later. I assumed that most of the correctness comes from #15146 but surely there may be interactions that goes beyond that.

[AmplabJenkins]

Can one of the admins verify this patch?

[maropu]

@gbloisi Any update?

[gatorsmile]

@chrysanxia Could you take over this?

[peter-toth]

@gatorsmile@gbloisi@chrysanxia may I take over this?
I played a bit with recursion support recently:
#23531 and during porting https://github.com/postgres/postgres/blob/master/src/test/regress/sql/with.sql I also found that WITH is not supported in subqueries #23531 (comment) (point 6).

[peter-toth]

@gatorsmile here is my PR: #24831