[SPARK-17710][HOTFIX] Fix ClassCircularityError in ReplSuite tests in Maven build: use 'Class.forName' instead of 'Utils.classForName'

[weiqingy]

What changes were proposed in this pull request?

Fix ClassCircularityError in ReplSuite tests when Spark is built by Maven build.

How was this patch tested?

(1)

build/mvn -DskipTests -Phadoop-2.3 -Pyarn -Phive -Phive-thriftserver -Pkinesis-asl -Pmesos clean package

Then test:

build/mvn -Dtest=none -DwildcardSuites=org.apache.spark.repl.ReplSuite test

ReplSuite tests passed

(2)
Manual Tests against some Spark applications in Yarn client mode and Yarn cluster mode. Need to check if spark caller contexts are written into HDFS hdfs-audit.log and Yarn RM audit log successfully.

[tgravescs]

please change [Follow UP] to [HOTFIX]

[tgravescs]

Also please file a separate jira to investigate more. Link it to https://issues.apache.org/jira/browse/SPARK-15857 and if needed we pull in people more familiar with the Repl classloader stuff.

[weiqingy]

Yes. The title has been changed. Thanks. @tgravescs

[tgravescs]

changes look fine, waiting on Jenkins.

[JoshRosen]

Can you change the JIRA number to SPARK-17710, which is the JIRA that I created for this build break?

https://issues.apache.org/jira/browse/SPARK-17710

[weiqingy]

@JoshRosen Yes. The title has been changed.
@tgravescs Yes. Now I am creating a separate jira to investigate this more.

[SparkQA]

Test build #66066 has finished for PR 15286 at commit 59bfa23.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[tgravescs]

committed this to master

[weiqingy]

Thank you very much. @tgravescs@JoshRosen

[rxin]

I think it's important to document why we are not using Utils.classForName here. Otherwise somebody is going to come in the future and get confused or change it.

[weiqingy]

Sure. I will submit a PR for this.

[weiqingy]

@rxin I have submitted a PR #15776 for this.