Skip to content

remove org.apache.spark.sql.hive.HiveSharedState class check - #1856

Closed
lichenglin wants to merge 2 commits into
apache:masterfrom
lichenglin:master
Closed

remove org.apache.spark.sql.hive.HiveSharedState class check#1856
lichenglin wants to merge 2 commits into
apache:masterfrom
lichenglin:master

Conversation

@lichenglin

Copy link
Copy Markdown
Contributor

What is this PR for?

remove org.apache.spark.sql.hive.HiveSharedState class check
because it has been removed since spark 2.1.x

What type of PR is it?

[Bug Fix ]

What is the Jira issue?

ZEPPELIN-1909

remove org.apache.spark.sql.hive.HiveSharedState class check .
Because this class has been removed from spark since 2.1.0
https://issues.apache.org/jira/browse/ZEPPELIN-1909
@felixcheung

Copy link
Copy Markdown
Member

looks good, why doesn't CI tests for Spark 2.1 fails for this?

@felixcheung

Copy link
Copy Markdown
Member

would be great to understand why it worked before, but might merge this fix first.
merging if no more comment

@lichenglin

Copy link
Copy Markdown
ContributorAuthor

I don't know either,It fails with spark 1.6.1.
May be the class check is still needed by 1.6,1?I'm not sure.
How can I trigger the build once more on travis-cli?

@felixcheung

Copy link
Copy Markdown
Member

I don't think we need to check for all 3 classes, so I believe this fix is good.
what I don't understand is why it is failing for you but running just fine in travis CI where we have tests for Spark 2.1 - which is why I'm asking.

to trigger test, close this PR and reopen it.

@lichenglin

Copy link
Copy Markdown
ContributorAuthor

e...
It works on my local mac with spark 2.1.0.
And pass the build with 2.1.0.
I seems that the build fails with
{code}
"env": "PYTHON="2" SCALA_VER="2.10" SPARK_VER="1.6.1" HADOOP_VER="2.6" PROFILE="-Pspark-1.6 -Phadoop-2.6 -Ppyspark" BUILD_FLAG="package -pl spark,python -am -DskipTests -DskipRat" TEST_FLAG="verify -DskipRat" TEST_PROJECTS="-pl zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python -Dtest=org.apache.zeppelin.spark.PySparkTest,org.apache.zeppelin.python. -Dpyspark.test.exclude='' -DfailIfNoTests=false"",
{code}

I think we'd better reopen the PR and build once more.
Maybe the class check is still necessary for spark 1.6.1

@felixcheung

Copy link
Copy Markdown
Member

it's only the python2 + spark 1.6 profile. It ran fine with the other spark 1.6 profile so I think we are good.

@asfgitasfgit closed this in 0f65714Jan 7, 2017
@felixcheungfelixcheung mentioned this pull request Jan 7, 2017
1 task
beriaanirudh pushed a commit to beriaanirudh/zeppelin that referenced this pull request Mar 8, 2017
### What is this PR for?
remove org.apache.spark.sql.hive.HiveSharedState class check
because it has been removed since spark 2.1.x
### What type of PR is it?
[Bug Fix ]
### What is the Jira issue?
[ZEPPELIN-1909](https://issues.apache.org/jira/browse/ZEPPELIN-1909)
Author: lichenglin <lichenglin0616@126.com>
Closesapache#1856 from lichenglin/master and squashes the following commits:
60c3c04 [lichenglin] Merge pull request apache#1 from lichenglin/lichenglin-patch-1
3f90485 [lichenglin] make hive enable under spark 2.1.0
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@lichenglin@felixcheung