Skip to content

[SPARK-11390] [SQL] Query plan with/without filterPushdown indistinguishable - #9679

Closed
zeocio wants to merge 1 commit into
apache:masterfrom
zeocio:spark-11390
Closed

[SPARK-11390] [SQL] Query plan with/without filterPushdown indistinguishable#9679
zeocio wants to merge 1 commit into
apache:masterfrom
zeocio:spark-11390

Conversation

@zeocio

Copy link
Copy Markdown

…ishable

Propagate pushed filters to PhyicalRDD in DataSourceStrategy.apply

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets call it extraInformation in both places.

@marmbrus

Copy link
Copy Markdown
Contributor

Can you fix the title / description so it doesn't truncate. Maybe add a test too so this doesn't get lost in refactoring.

@zeocio

Copy link
Copy Markdown
Author

@marmbrus i was thinking about test cases, which sql/xxxSuite should this test case be added to?

@marmbrus

Copy link
Copy Markdown
Contributor

maybe sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala ?

@zeociozeocio changed the title [SPARK-11390] [SQL] Query plan with/without filterPushdown indistingu…[SPARK-11390] [SQL] Query plan with/without filterPushdown indistinguishableNov 13, 2015
@zeocio

Copy link
Copy Markdown
Author

Changes to string formatting of Filter expr can break the test case. Did this recently change from "(key#27 = 15)" to "EqualTo(key,15)" ?

@zeocio

Copy link
Copy Markdown
Author

@marmbrus is the PR good to go?

@yhuai

Copy link
Copy Markdown
Contributor

ok to test

@SparkQA

Copy link
Copy Markdown

Test build #45891 has finished for PR 9679 at commit e8a40cc.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #45900 has finished for PR 9679 at commit 96684bc.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #45903 has finished for PR 9679 at commit c697837.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #45960 has finished for PR 9679 at commit 9597b39.

  • This patch passes all tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

…ishable
Propagate pushed filters to PhyicalRDD in DataSourceStrategy.apply
Add test case with parquet data source.
Style fix.
style fix
Revert PhysicalRDD string formatting.
@SparkQA

Copy link
Copy Markdown

Test build #46011 has finished for PR 9679 at commit 7e47f43.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * case class JSONOptions(\n

@zeocio

Copy link
Copy Markdown
Author

Please merge.

@marmbrus

Copy link
Copy Markdown
Contributor

Thanks, merging to master and 1.6.

asfgit pushed a commit that referenced this pull request Nov 16, 2015
…shable
…ishable
Propagate pushed filters to PhyicalRDD in DataSourceStrategy.apply
Author: Zee Chen <zeechen@us.ibm.com>
Closes#9679 from zeocio/spark-11390.
(cherry picked from commit 985b38d)
Signed-off-by: Michael Armbrust <michael@databricks.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@zeocio@marmbrus@yhuai@SparkQA