[SPARK-2235][SQL]Spark SQL basicOperator add Intersect operator by YanjieGao · Pull Request #1150 · apache/spark

YanjieGao · 2014-06-20T07:20:34Z

Hi all,
I want to submit a basic operator Intersect
For example , in sql case
select * from table1
intersect
select * from table2
So ,i want use this operator support this function in Spark SQL
This operator will return the the intersection of SparkPlan child table RDD .
JIRA:https://issues.apache.org/jira/browse/SPARK-2235

Hi all, I want to submit a basic operator Intersect For example , in sql case select * from table1 intersect select * from table2 So ,i want use this operator support this function in Spark SQL This operator will return the the intersection of SparkPlan child table RDD .

AmplabJenkins · 2014-06-20T07:24:58Z

Can one of the admins verify this patch?

YanjieGao · 2014-06-23T08:18:33Z

hi all ,I have finished the INTERSECT operator ,and has update the 6 files this operator needed .
Now i need your help to review the code .Thanks a lot !

YanjieGao · 2014-07-03T05:53:10Z

Hi all, I have resolved the conflict .

marmbrus · 2014-07-04T09:51:50Z

This is conflicting now that we merged the other PR.

test this please

marmbrus · 2014-07-04T09:57:12Z

Does hive support this? if so we should add a test to HiveQuerySuite too.

Hive doesn't support ,I will remove this

YanjieGao · 2014-07-04T10:45:32Z

I don't know why this line always red

Hi, it says you deleted some whitespace after Nil.

YanjieGao · 2014-07-04T10:46:26Z

Hi Michael,I have resolve the conflict .And modify as your suggestion

YanjieGao · 2014-07-05T04:55:25Z

Thanks ,I add three blank after that line ,Now there aren't red lines.

marmbrus · 2014-07-05T20:16:34Z

Space between ) and }

marmbrus · 2014-07-05T20:17:11Z

Space after ,

marmbrus · 2014-07-05T20:19:03Z

Thanks ,I add three blank after that line ,Now there aren't red lines.

If there is trailing whitespace we should remove it. It is okay if it is not code you added in the PR.

YanjieGao · 2014-07-06T00:31:12Z

Thanks a lot, I have reformat the code style.

marmbrus · 2014-07-08T02:40:57Z

Thanks, I've merged this into master. Did not apply cleanly to 1.0 so we'll backport it later.

YanjieGao · 2014-07-08T02:45:25Z

Hi, Michael I also have a Skew Join pr need to be reviewed and need some suggestions .Could you help me review it ?I have test it can pass the testsuite .Thanks a lot

#1134

Hi all, I want to submit a basic operator Intersect For example , in sql case select * from table1 intersect select * from table2 So ,i want use this operator support this function in Spark SQL This operator will return the the intersection of SparkPlan child table RDD . JIRA:https://issues.apache.org/jira/browse/SPARK-2235 Author: Yanjie Gao <gaoyanjie55@163.com> Author: YanjieGao <396154235@qq.com> Closesapache#1150 from YanjieGao/patch-5 and squashes the following commits: 4629afe [YanjieGao] reformat the code bdc2ac0 [YanjieGao] reformat the code as Michael's suggestion 3b29ad6 [YanjieGao] Merge remote branch 'upstream/master' into patch-5 1cfbfe6 [YanjieGao] refomat some files ea78f33 [YanjieGao] resolve conflict and add annotation on basicOperator and remove HiveQl 0c7cca5 [YanjieGao] modify format problem a802ca8 [YanjieGao] Merge remote branch 'upstream/master' into patch-5 5e374c7 [YanjieGao] resolve conflict in SparkStrategies and basicOperator f7961f6 [Yanjie Gao] update the line less than bdc4a05 [Yanjie Gao] Update basicOperators.scala 0b49837 [Yanjie Gao] delete the annotation f1288b4 [Yanjie Gao] delete annotation e2b64be [Yanjie Gao] Update basicOperators.scala 4dd453e [Yanjie Gao] Update SQLQuerySuite.scala 790765d [Yanjie Gao] Update SparkStrategies.scala ac73e60 [Yanjie Gao] Update basicOperators.scala d4ac5e5 [Yanjie Gao] Update HiveQl.scala 61e88e7 [Yanjie Gao] Update SqlParser.scala 469f099 [Yanjie Gao] Update basicOperators.scala e5bff61 [Yanjie Gao] Spark SQL basicOperator add Intersect operator

…s" message appears when spark session is starting (apache#1150)

YanjieGao changed the title ~~Spark SQL basicOperator add Intersect operator~~[SPARK-2235][SQL]Spark SQL basicOperator add Intersect operatorJun 22, 2014

YanjieGao added 6 commits June 23, 2014 16:04

Update basicOperators.scala
469f099

Update SqlParser.scala
61e88e7

Update HiveQl.scala
d4ac5e5

Update basicOperators.scala
ac73e60

Update SparkStrategies.scala
790765d

Update SQLQuerySuite.scala
4dd453e

YanjieGao added 5 commits June 24, 2014 11:14

Update basicOperators.scala
e2b64be

delete annotation
f1288b4

delete the annotation
0b49837

Update basicOperators.scala
bdc4a05

update the line less than
f7961f6

YanjieGao added 3 commits July 3, 2014 14:07

resolve conflict in SparkStrategies and basicOperator
5e374c7

Merge remote branch 'upstream/master' into patch-5
a802ca8

modify format problem
0c7cca5

YanjieGao mentioned this pull request Jul 4, 2014
[SPARK-2234][SQL]Spark SQL basicOperators add Except operator #1151
Closed

marmbrus reviewed Jul 4, 2014
View reviewed changes

YanjieGao added 2 commits July 4, 2014 18:27

resolve conflict and add annotation on basicOperator and remove HiveQl
ea78f33

refomat some files
1cfbfe6

YanjieGao reviewed Jul 4, 2014
View reviewed changes

marmbrus reviewed Jul 5, 2014
View reviewed changes

Comment threadsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala Outdated
marmbrusJul 5, 2014
Copy link
Copy Markdown
Contributor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Space between ) and }

marmbrus reviewed Jul 5, 2014
View reviewed changes

Comment threadsql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala Outdated
marmbrusJul 5, 2014
Copy link
Copy Markdown
Contributor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Space after ,

Merge remote branch 'upstream/master' into patch-5
3b29ad6

YanjieGao added 2 commits July 6, 2014 08:39

reformat the code as Michael's suggestion
bdc2ac0

reformat the code
4629afe

asfgit closed this in 50561f4Jul 8, 2014

udaynpusa pushed a commit to mapr/spark that referenced this pull request Jan 30, 2024
MapR [SPARK-1228] "Warning: Ignoring non-Spark config property: maprf…
af1f597
…s" message appears when spark session is starting (apache#1150)

mapr-devops pushed a commit to mapr/spark that referenced this pull request May 8, 2025
MapR [SPARK-1228] "Warning: Ignoring non-Spark config property: maprf…
e02d9f7
…s" message appears when spark session is starting (apache#1150)

Uh oh!

Conversation

YanjieGao commented Jun 20, 2014

Uh oh!

AmplabJenkins commented Jun 20, 2014

Uh oh!

YanjieGao commented Jun 23, 2014

Uh oh!

YanjieGao commented Jul 3, 2014

Uh oh!

marmbrus commented Jul 4, 2014

Uh oh!

marmbrusJul 4, 2014

Choose a reason for hiding this comment

Uh oh!

YanjieGaoJul 4, 2014

Choose a reason for hiding this comment

Uh oh!

YanjieGaoJul 4, 2014

Choose a reason for hiding this comment

Uh oh!

ueshinJul 5, 2014

Choose a reason for hiding this comment

Uh oh!

YanjieGao commented Jul 4, 2014

Uh oh!

YanjieGao commented Jul 5, 2014

Uh oh!

marmbrusJul 5, 2014

Choose a reason for hiding this comment

Uh oh!

marmbrusJul 5, 2014

Choose a reason for hiding this comment

Uh oh!

marmbrus commented Jul 5, 2014

Uh oh!

YanjieGao commented Jul 6, 2014

Uh oh!

marmbrus commented Jul 8, 2014

Uh oh!

YanjieGao commented Jul 8, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants