[SPARK-11672] [ML] flaky spark.ml read/write tests by mengxr · Pull Request #9677 · apache/spark

mengxr · 2015-11-12T20:59:05Z

We set sqlContext = null in afterAll. However, this doesn't change SQLContext.activeContext and then SQLContext.getOrCreate might use the SparkContext from previous test suite and hence causes the error. This PR calls clearActive in beforeAll and afterAll to avoid using an old context from other test suites.

cc: @yhuai

This reverts commit 1a21be1.

mengxr · 2015-11-12T21:16:44Z

test this please

mengxr · 2015-11-12T22:26:13Z

test this please

SparkQA · 2015-11-12T23:28:43Z

Test build #45770 has finished for PR 9677 at commit e130df6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-11-12T23:33:52Z

Test build #45773 has finished for PR 9677 at commit e130df6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

mengxr · 2015-11-12T23:56:10Z

Okay, I think the problem is SQLContext.getOrCreate may return an old SQLContext and we need to clear the active SQLContext in afterAll.

mengxr · 2015-11-12T23:56:32Z

[info] - read/write *** FAILED *** (2 milliseconds)
[info] java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext.
[info] This stopped SparkContext was created at:
[info] [info] org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
[info] org.apache.spark.mllib.util.MLlibTestSparkContext$class.beforeAll(MLlibTestSparkContext.scala:34)
[info] org.apache.spark.mllib.classification.LogisticRegressionSuite.beforeAll(LogisticRegressionSuite.scala:172)
[info] org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
[info] org.apache.spark.mllib.classification.LogisticRegressionSuite.beforeAll(LogisticRegressionSuite.scala:172)
[info] org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
[info] org.apache.spark.mllib.classification.LogisticRegressionSuite.run(LogisticRegressionSuite.scala:172)
[info] org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
[info] org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
[info] sbt.ForkMain$Run$2.call(ForkMain.java:294)
[info] sbt.ForkMain$Run$2.call(ForkMain.java:284)
[info] java.util.concurrent.FutureTask.run(FutureTask.java:262)
[info] java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[info] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[info] java.lang.Thread.run(Thread.java:745)
[info] [info] The currently active SparkContext was created at:
[info] [info] org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
[info] org.apache.spark.mllib.util.MLlibTestSparkContext$class.beforeAll(MLlibTestSparkContext.scala:34)
[info] org.apache.spark.ml.feature.BinarizerSuite.org$apache$spark$ml$util$TempDirectory$$super$beforeAll(BinarizerSuite.scala:26)
[info] org.apache.spark.ml.util.TempDirectory$class.beforeAll(TempDirectory.scala:37)
[info] org.apache.spark.ml.feature.BinarizerSuite.beforeAll(BinarizerSuite.scala:31)
[info] org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
[info] org.apache.spark.ml.feature.BinarizerSuite.beforeAll(BinarizerSuite.scala:26)
[info] org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
[info] org.apache.spark.ml.feature.BinarizerSuite.run(BinarizerSuite.scala:26)
[info] org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
[info] org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
[info] sbt.ForkMain$Run$2.call(ForkMain.java:294)
[info] sbt.ForkMain$Run$2.call(ForkMain.java:284)
[info] java.util.concurrent.FutureTask.run(FutureTask.java:262)
[info] java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[info] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[info] java.lang.Thread.run(Thread.java:745)
[info] at org.apache.spark.SparkContext.org$apache$spark$SparkContext$$assertNotStopped(SparkContext.scala:106)
[info] at org.apache.spark.SparkContext$$anonfun$parallelize$1.apply(SparkContext.scala:732)
[info] at org.apache.spark.SparkContext$$anonfun$parallelize$1.apply(SparkContext.scala:731)
[info] at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
[info] at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
[info] at org.apache.spark.SparkContext.withScope(SparkContext.scala:717)
[info] at org.apache.spark.SparkContext.parallelize(SparkContext.scala:731)
[info] at org.apache.spark.ml.util.DefaultParamsWriter$.saveMetadata(ReadWrite.scala:206)
[info] at org.apache.spark.ml.util.DefaultParamsWriter.saveImpl(ReadWrite.scala:178)
[info] at org.apache.spark.ml.util.Writer.save(ReadWrite.scala:87)
[info] at org.apache.spark.ml.util.Writable$class.save(ReadWrite.scala:127)
[info] at org.apache.spark.ml.feature.Binarizer.save(Binarizer.scala:35)
[info] at org.apache.spark.ml.util.DefaultReadWriteTest$class.testDefaultReadWrite(DefaultReadWriteTest.scala:40)
[info] at org.apache.spark.ml.feature.BinarizerSuite.testDefaultReadWrite(BinarizerSuite.scala:26)
[info] at org.apache.spark.ml.feature.BinarizerSuite$$anonfun$6.apply$mcV$sp(BinarizerSuite.scala:76)
[info] at org.apache.spark.ml.feature.BinarizerSuite$$anonfun$6.apply(BinarizerSuite.scala:71)
[info] at org.apache.spark.ml.feature.BinarizerSuite$$anonfun$6.apply(BinarizerSuite.scala:71)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] at org.scalatest.Transformer.apply(Transformer.scala:20)
[info] at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
[info] at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:42)
[info] at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
[info] at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
[info] at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
[info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info] at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
[info] at org.scalatest.FunSuite.runTest(FunSuite.scala:1555)
[info] at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
[info] at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
[info] at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
[info] at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
[info] at scala.collection.immutable.List.foreach(List.scala:318)
[info] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
[info] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
[info] at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:208)
[info] at org.scalatest.FunSuite.runTests(FunSuite.scala:1555)
[info] at org.scalatest.Suite$class.run(Suite.scala:1424)
[info] at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1555)
[info] at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
[info] at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
[info] at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
[info] at org.scalatest.FunSuiteLike$class.run(FunSuiteLike.scala:212)
[info] at org.apache.spark.ml.feature.BinarizerSuite.org$scalatest$BeforeAndAfterAll$$super$run(BinarizerSuite.scala:26)
[info] at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:257)
[info] at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:256)
[info] at org.apache.spark.ml.feature.BinarizerSuite.run(BinarizerSuite.scala:26)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:294)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:284)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[info] at java.lang.Thread.run(Thread.java:745)

SparkQA · 2015-11-13T01:43:13Z

Test build #45790 has finished for PR 9677 at commit da857a3.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n * case class StddevSamp(child: Expression,\n * case class StddevPop(\n

SparkQA · 2015-11-13T02:01:00Z

Test build #45788 has finished for PR 9677 at commit 62b09a3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

yhuai · 2015-11-13T04:00:40Z

LGTM

mengxr · 2015-11-13T04:02:07Z

Merged into master and branch-1.6.

We set `sqlContext = null` in `afterAll`. However, this doesn't change `SQLContext.activeContext` and then `SQLContext.getOrCreate` might use the `SparkContext` from previous test suite and hence causes the error. This PR calls `clearActive` in `beforeAll` and `afterAll` to avoid using an old context from other test suites. cc: yhuai Author: Xiangrui Meng <meng@databricks.com> Closes#9677 from mengxr/SPARK-11672.2. (cherry picked from commit e71c075) Signed-off-by: Xiangrui Meng <meng@databricks.com>

We set `sqlContext = null` in `afterAll`. However, this doesn't change `SQLContext.activeContext` and then `SQLContext.getOrCreate` might use the `SparkContext` from previous test suite and hence causes the error. This PR calls `clearActive` in `beforeAll` and `afterAll` to avoid using an old context from other test suites. cc: yhuai Author: Xiangrui Meng <meng@databricks.com> Closesapache#9677 from mengxr/SPARK-11672.2.

shivaram · 2015-11-14T05:07:59Z

@mengxr I just saw a failure that seems related to this PR at https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45914/consoleFull

[info] Test org.apache.spark.ml.util.JavaDefaultReadWriteSuite.testDefaultReadWrite started
[error] Test org.apache.spark.ml.util.JavaDefaultReadWriteSuite.testDefaultReadWrite failed: java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext.
[error] This stopped SparkContext was created at:
[error] [error] org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
[error] org.apache.spark.mllib.util.MLlibTestSparkContext$class.beforeAll(MLlibTestSparkContext.scala:34)
[error] org.apache.spark.mllib.optimization.GradientDescentSuite.beforeAll(GradientDescentSuite.scala:65)
[error] org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
[error] org.apache.spark.mllib.optimization.GradientDescentSuite.beforeAll(GradientDescentSuite.scala:65)
[error] org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
[error] org.apache.spark.mllib.optimization.GradientDescentSuite.run(GradientDescentSuite.scala:65)
[error] org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
[error] org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
[error] sbt.ForkMain$Run$2.call(ForkMain.java:294)
[error] sbt.ForkMain$Run$2.call(ForkMain.java:284)
[error] java.util.concurrent.FutureTask.run(FutureTask.java:262)
[error] java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[error] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[error] java.lang.Thread.run(Thread.java:745)
[error] [error] The currently active SparkContext was created at:
[error] [error] org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:65)
[error] org.apache.spark.ml.util.JavaDefaultReadWriteSuite.setUp(JavaDefaultReadWriteSuite.java:39)
[error] sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:606)
[error] org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
[error] org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
[error] org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
[error] org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
[error] org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
[error] org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
[error] org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
[error] org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
[error] org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
[error] org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
[error] org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
[error] org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
[error] org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
[error] org.junit.runners.ParentRunner.run(ParentRunner.java:309)
[error] , took 0.212 sec
[error] at org.apache.spark.SparkContext.org$apache$spark$SparkContext$$assertNotStopped(SparkContext.scala:106)
[error] at org.apache.spark.SparkContext$$anonfun$parallelize$1.apply(SparkContext.scala:732)
[error] at org.apache.spark.SparkContext$$anonfun$parallelize$1.apply(SparkContext.scala:731)
[error] at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
[error] at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
[error] at org.apache.spark.SparkContext.withScope(SparkContext.scala:717)
[error] at org.apache.spark.SparkContext.parallelize(SparkContext.scala:731)
[error] at org.apache.spark.ml.util.DefaultParamsWriter$.saveMetadata(ReadWrite.scala:209)
[error] at org.apache.spark.ml.util.DefaultParamsWriter.saveImpl(ReadWrite.scala:181)
[error] at org.apache.spark.ml.util.Writer.save(ReadWrite.scala:90)
[error] at org.apache.spark.ml.util.Writable$class.save(ReadWrite.scala:130)
[error] at org.apache.spark.ml.util.MyParams.save(DefaultReadWriteTest.scala:69)
[error] at org.apache.spark.ml.util.JavaDefaultReadWriteSuite.testDefaultReadWrite(JavaDefaultReadWriteSuite.java:59)
[error] ...

mengxr added 2 commits November 12, 2015 12:57

include call site info in SparkContext.assertNotStopped error message
656953f

Revert "[SPARK-11672][ML] disable spark.ml read/write tests"
e130df6
This reverts commit 1a21be1.

mengxr changed the title ~~[WIP] [SPARK-11672] flaky spark.ml read/write tests~~[WIP] [SPARK-11672] [ML] flaky spark.ml read/write testsNov 12, 2015

mengxr added 2 commits November 12, 2015 15:57

clear active SQLContext before and after tests
62b09a3

Merge remote-tracking branch 'apache/master' into SPARK-11672.2
da857a3

mengxr changed the title ~~[WIP] [SPARK-11672] [ML] flaky spark.ml read/write tests~~[SPARK-11672] [ML] flaky spark.ml read/write testsNov 13, 2015

asfgit closed this in e71c075Nov 13, 2015

liancheng mentioned this pull request Nov 13, 2015
[SPARK-11678] [SQL] Partition discovery should stop at the root path of the table. #9651
Closed

Navigation Menu

Uh oh!

[SPARK-11672] [ML] flaky spark.ml read/write tests - #9677

[SPARK-11672] [ML] flaky spark.ml read/write tests#9677
mengxr wants to merge 4 commits into
apache:masterfrom
mengxr:SPARK-11672.2

mengxr commented Nov 12, 2015

Uh oh!

mengxr commented Nov 12, 2015

Uh oh!

mengxr commented Nov 12, 2015

Uh oh!

SparkQA commented Nov 12, 2015

Uh oh!

SparkQA commented Nov 12, 2015

Uh oh!

mengxr commented Nov 12, 2015

Uh oh!

mengxr commented Nov 12, 2015

Uh oh!

SparkQA commented Nov 13, 2015

Uh oh!

SparkQA commented Nov 13, 2015

Uh oh!

yhuai commented Nov 13, 2015

Uh oh!

mengxr commented Nov 13, 2015

Uh oh!

shivaram commented Nov 14, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

mengxr commented Nov 12, 2015

Uh oh!

mengxr commented Nov 12, 2015

Uh oh!

mengxr commented Nov 12, 2015

Uh oh!

SparkQA commented Nov 12, 2015

Uh oh!

SparkQA commented Nov 12, 2015

Uh oh!

mengxr commented Nov 12, 2015

Uh oh!

mengxr commented Nov 12, 2015

Uh oh!

SparkQA commented Nov 13, 2015

Uh oh!

SparkQA commented Nov 13, 2015

Uh oh!

yhuai commented Nov 13, 2015

Uh oh!

mengxr commented Nov 13, 2015

Uh oh!

shivaram commented Nov 14, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants