Skip to content

[SPARK-13306][SQL] Addendum to uncorrelated scalar subquery - #11285

Closed
rxin wants to merge 2 commits into
apache:masterfrom
rxin:subquery
Closed

[SPARK-13306][SQL] Addendum to uncorrelated scalar subquery#11285
rxin wants to merge 2 commits into
apache:masterfrom
rxin:subquery

Conversation

@rxin

@rxinrxin commented Feb 21, 2016

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This pull request fixes some minor issues (documentation, test flakiness, test organization) with #11190, which was merged earlier tonight.

How was the this patch tested?

unit tests.

}

// TODO: Specialize
case class MutableLiteral(var value: Any, dataType: DataType, nullable: Boolean = true)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was unused. just deleting it.

@rxin

rxin commented Feb 21, 2016

Copy link
Copy Markdown
ContributorAuthor

cc @davies
can you explain what the following test case is testing?

 assertResult(Array(Row(1))) {
sql("with t2 as (select 1 as b, 2 as c) " +
"select a from (select 1 as a union all select 2 as a) t " +
"where a = (select max(b) from t2) ").collect()
}

@rxin

rxin commented Feb 21, 2016

Copy link
Copy Markdown
ContributorAuthor

Also explain why wholestagecodegen needs to call waitforsubqueries explicitly.

@davies

Copy link
Copy Markdown
Contributor

For those operators that is generated in WholeStageCodegen, the execute() of them are not called, but produce() will be called, we should make sure that all the subqueries all executed before generating the code in doProduce() and doConsume()

@davies

Copy link
Copy Markdown
Contributor

That case is used to test CTE and aggregation inside subquery.

@SparkQA

Copy link
Copy Markdown

Test build #51614 has finished for PR 11285 at commit de512f6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #51611 has finished for PR 11285 at commit c9bd987.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PlanSubqueries(sqlContext: SQLContext) extends Rule[SparkPlan]

@rxin

rxin commented Feb 21, 2016

Copy link
Copy Markdown
ContributorAuthor

Merging this in master.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@rxin@davies@SparkQA