Generalize pattern for planning hash joins. by marmbrus · Pull Request #418 · apache/spark

marmbrus · 2014-04-15T18:20:31Z

This will be helpful for SPARK-1495 and other cases where we want to have custom hash join implementations but don't want to repeat the logic for finding the join keys.

AmplabJenkins · 2014-04-15T18:23:12Z

Merged build triggered.

AmplabJenkins · 2014-04-15T18:23:22Z

Merged build started.

AmplabJenkins · 2014-04-15T18:24:47Z

Merged build finished.

AmplabJenkins · 2014-04-15T18:24:47Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14152/

adrian-wang · 2014-04-16T03:06:14Z

Thanks Michael! This will be really helpful.

pwendell · 2014-04-16T03:37:35Z

Jenkins, retest this please (need to test our test harness on a SQL patch).

AmplabJenkins · 2014-04-16T03:38:14Z

Merged build triggered.

AmplabJenkins · 2014-04-16T03:38:24Z

Merged build started.

AmplabJenkins · 2014-04-16T03:40:03Z

Merged build finished.

AmplabJenkins · 2014-04-16T03:40:03Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14167/

marmbrus · 2014-04-23T21:00:08Z

test this please

AmplabJenkins · 2014-04-23T21:02:58Z

Merged build triggered.

AmplabJenkins · 2014-04-23T21:21:36Z

Merged build started.

AmplabJenkins · 2014-04-23T21:22:56Z

Merged build finished.

AmplabJenkins · 2014-04-23T21:22:56Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14399/

AmplabJenkins · 2014-04-23T21:57:55Z

Merged build triggered.

AmplabJenkins · 2014-04-23T21:58:02Z

Merged build started.

AmplabJenkins · 2014-04-23T23:14:10Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-04-23T23:14:10Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14402/

marmbrus · 2014-04-23T23:21:12Z

@rxin PTAL

rxin · 2014-04-24T05:49:43Z

Is this used anywhere? (it is private def)

Good catch. Removed as reduceLeft(And) is shorter and probably more clear anyway.

rxin · 2014-04-24T06:13:45Z

Overall looks pretty good. I left some comments mainly on readability.

AmplabJenkins · 2014-04-25T00:37:56Z

Merged build triggered.

AmplabJenkins · 2014-04-25T00:38:04Z

Merged build started.

AmplabJenkins · 2014-04-25T01:51:59Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-04-25T01:51:59Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14464/

rxin · 2014-04-25T04:42:48Z

Thanks. I've merged this.

@rxin

This will be helpful for [SPARK-1495](https://issues.apache.org/jira/browse/SPARK-1495) and other cases where we want to have custom hash join implementations but don't want to repeat the logic for finding the join keys. Author: Michael Armbrust <michael@databricks.com> Closes#418 from marmbrus/hashFilter and squashes the following commits: d5cc79b [Michael Armbrust] Address @rxin 's comments. 366b6d9 [Michael Armbrust] style fixes 14560eb [Michael Armbrust] Generalize pattern for planning hash joins. f4809c1 [Michael Armbrust] Move common functions to PredicateHelper. (cherry picked from commit 86ff8b1) Signed-off-by: Reynold Xin <rxin@apache.org>

@rxin

This will be helpful for [SPARK-1495](https://issues.apache.org/jira/browse/SPARK-1495) and other cases where we want to have custom hash join implementations but don't want to repeat the logic for finding the join keys. Author: Michael Armbrust <michael@databricks.com> Closesapache#418 from marmbrus/hashFilter and squashes the following commits: d5cc79b [Michael Armbrust] Address @rxin 's comments. 366b6d9 [Michael Armbrust] style fixes 14560eb [Michael Armbrust] Generalize pattern for planning hash joins. f4809c1 [Michael Armbrust] Move common functions to PredicateHelper.

Version changes for release 0.9.0.

Switch to official ansible project

marmbrus added 3 commits April 23, 2014 14:54

Move common functions to PredicateHelper.
f4809c1

Generalize pattern for planning hash joins.
14560eb

style fixes
366b6d9

rxin reviewed Apr 24, 2014
View reviewed changes

Address@rxin's comments.
d5cc79b

asfgit closed this in 86ff8b1Apr 25, 2014

adrian-wang mentioned this pull request May 12, 2014
support leftsemijoin for sparkSQL #395
Closed

adrian-wang mentioned this pull request May 20, 2014
[SPARK-1495][SQL]add support for left semi join #837
Closed

marmbrus deleted the hashFilter branch June 6, 2014 05:25

andrewor14 pushed a commit to andrewor14/spark that referenced this pull request Jan 8, 2015
Merge pull requestapache#418from pwendell/0.9-versions
b60840e
Version changes for release 0.9.0.

bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019
Merge pull requestapache#418from theopenlab/ansible-switch
ada4f0e
Switch to official ansible project

imback82 mentioned this pull request Apr 29, 2020
[SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable #28123
Closed

Uh oh!

Conversation

marmbrus commented Apr 15, 2014

Uh oh!

AmplabJenkins commented Apr 15, 2014

Uh oh!

AmplabJenkins commented Apr 15, 2014

Uh oh!

AmplabJenkins commented Apr 15, 2014

Uh oh!

AmplabJenkins commented Apr 15, 2014

Uh oh!

adrian-wang commented Apr 16, 2014

Uh oh!

pwendell commented Apr 16, 2014

Uh oh!

AmplabJenkins commented Apr 16, 2014

Uh oh!

AmplabJenkins commented Apr 16, 2014

Uh oh!

AmplabJenkins commented Apr 16, 2014

Uh oh!

AmplabJenkins commented Apr 16, 2014

Uh oh!

marmbrus commented Apr 23, 2014

Uh oh!

AmplabJenkins commented Apr 23, 2014

Uh oh!

AmplabJenkins commented Apr 23, 2014

Uh oh!

AmplabJenkins commented Apr 23, 2014

Uh oh!

AmplabJenkins commented Apr 23, 2014

Uh oh!

AmplabJenkins commented Apr 23, 2014

Uh oh!

AmplabJenkins commented Apr 23, 2014

Uh oh!

AmplabJenkins commented Apr 23, 2014

Uh oh!

AmplabJenkins commented Apr 23, 2014

Uh oh!

marmbrus commented Apr 23, 2014

Uh oh!

rxinApr 24, 2014

Choose a reason for hiding this comment

Uh oh!

marmbrusApr 24, 2014

Choose a reason for hiding this comment

Uh oh!

rxin commented Apr 24, 2014

Uh oh!

AmplabJenkins commented Apr 25, 2014

Uh oh!

AmplabJenkins commented Apr 25, 2014

Uh oh!

AmplabJenkins commented Apr 25, 2014

Uh oh!

AmplabJenkins commented Apr 25, 2014

Uh oh!

rxin commented Apr 25, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants