[Spark-3490] Disable SparkUI for tests (backport into 1.0) by andrewor14 · Pull Request #3959 · apache/spark

andrewor14 · 2015-01-08T23:37:56Z

Branch-1.2 #2363 (original)
Branch-1.1 #2415
Branch-1.0 #3959 (this PR)
Branch-0.9 #3961

SparkQA · 2015-01-08T23:42:47Z

Test build #25274 has started for PR 3959 at commit 5425314.

This patch merges cleanly.

SparkQA · 2015-01-09T00:56:10Z

Test build #25274 has finished for PR 3959 at commit 5425314.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-09T00:56:14Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25274/
Test FAILed.

Branch-1.2 #2363 (original) Branch-1.1 #2415 Branch-1.0 #3959 Branch-0.9 #3961 (this PR) Author: Andrew Or <andrew@databricks.com> Closes#3961 from andrewor14/ui-ports-0.9 and squashes the following commits: 8644997 [Andrew Or] Disable UI for tests

andrewor14 · 2015-01-09T18:25:56Z

retest this please

SparkQA · 2015-01-09T18:27:42Z

Test build #25330 has started for PR 3959 at commit 5425314.

This patch merges cleanly.

SparkQA · 2015-01-09T19:41:14Z

Test build #25330 has finished for PR 3959 at commit 5425314.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-09T19:41:18Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25330/
Test FAILed.

andrewor14 · 2015-01-09T19:57:45Z

Looks like the context cleaner suite is consistently failing. (It's failing outside of this PR too)

JoshRosen · 2015-01-09T20:12:03Z

We should also include two hotfixes that I made to disable the UI for processes launched by shell scripts (in SparkSubmitSuite and DriverSuite):

e24d3a9
0128398 (#3873).

andrewor14 · 2015-01-09T22:56:02Z

Ok, I have cherry-picked those commits, though they shouldn't affect the particular test that has been failing (ContextCleanerSuite). Let's just retest this please anyway.

SparkQA · 2015-01-09T22:57:45Z

Test build #25345 has started for PR 3959 at commit 5425314.

This patch merges cleanly.

SparkQA · 2015-01-10T00:07:38Z

Test build #25345 has finished for PR 3959 at commit 5425314.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-10T00:07:42Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25345/
Test FAILed.

JoshRosen · 2015-01-13T00:20:50Z

Jenkins, retest this please.

SparkQA · 2015-01-13T00:22:48Z

Test build #25436 has started for PR 3959 at commit 5425314.

This patch merges cleanly.

SparkQA · 2015-01-13T01:44:10Z

Test build #25436 has finished for PR 3959 at commit 5425314.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-13T01:44:14Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25436/
Test FAILed.

JoshRosen · 2015-01-15T19:58:37Z

Jenkins, retest this please.

SparkQA · 2015-01-15T20:02:53Z

Test build #25613 has started for PR 3959 at commit 5425314.

This patch merges cleanly.

SparkQA · 2015-01-15T21:13:02Z

Test build #25613 has finished for PR 3959 at commit 5425314.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-15T21:13:06Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25613/
Test FAILed.

JoshRosen · 2015-01-15T21:33:44Z

I logged into Jenkins to look at the core/unit-tests.log file and noticed a few exceptions that might explain this test failure:

15/01/15 13:12:44.892 ERROR ExecutorRunner: Error running executor
java.io.IOException: Cannot run program "/home/jenkins/workspace/SparkPullRequestBuilder/core/./bin/compute-classpath.sh" (in directory "."): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
at org.apache.spark.util.Utils$.executeAndGetOutput(Utils.scala:759)
at org.apache.spark.deploy.worker.CommandUtils$.buildJavaOpts(CommandUtils.scala:72)
at org.apache.spark.deploy.worker.CommandUtils$.buildCommandSeq(CommandUtils.scala:37)
at org.apache.spark.deploy.worker.ExecutorRunner.getCommandSeq(ExecutorRunner.scala:110)
at org.apache.spark.deploy.worker.ExecutorRunner.fetchAndRunExecutor(ExecutorRunner.scala:125)
at org.apache.spark.deploy.worker.ExecutorRunner$$anon$1.run(ExecutorRunner.scala:58)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
... 6 more
15/01/15 13:12:44.893 INFO Master: Registering worker localhost:57914 with 1 cores, 512.0 MB RAM
15/01/15 13:12:44.893 INFO Master: Launching executor app-20150115131244-0000/1 on worker worker-20150115131244-localhost-57914
15/01/15 13:12:44.893 INFO Worker: Successfully registered with master spark://localhost:54168
15/01/15 13:12:44.894 INFO Worker: Executor app-20150115131244-0000/0 finished with state FAILED message java.io.IOException: Cannot run program "/home/jenkins/workspace/SparkPullRequestBuilder/core/./bin/compute-classpath.sh" (in directory "."): error=2, No such file or directory
15/01/15 13:12:44.894 INFO Master: Removing executor app-20150115131244-0000/0 because it is FAILED
15/01/15 13:12:44.894 INFO Master: Launching executor app-20150115131244-0000/2 on worker worker-20150115131244-localhost-46755
15/01/15 13:12:44.894 INFO Worker: Asked to launch executor app-20150115131244-0000/1 for ContextCleanerSuite
15/01/15 13:12:44.895 INFO Worker: Asked to launch executor app-20150115131244-0000/2 for ContextCleanerSuite
15/01/15 13:12:44.897 ERROR ExecutorRunner: Error running executor
java.io.IOException: Cannot run program "/home/jenkins/workspace/SparkPullRequestBuilder/core/./bin/compute-classpath.sh" (in directory "."): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
at org.apache.spark.util.Utils$.executeAndGetOutput(Utils.scala:759)
at org.apache.spark.deploy.worker.CommandUtils$.buildJavaOpts(CommandUtils.scala:72)
at org.apache.spark.deploy.worker.CommandUtils$.buildCommandSeq(CommandUtils.scala:37)
at org.apache.spark.deploy.worker.ExecutorRunner.getCommandSeq(ExecutorRunner.scala:110)
at org.apache.spark.deploy.worker.ExecutorRunner.fetchAndRunExecutor(ExecutorRunner.scala:125)
at org.apache.spark.deploy.worker.ExecutorRunner$$anon$1.run(ExecutorRunner.scala:58)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
... 6 more

and

15/01/15 13:12:59.527 ERROR ExecutorRunner: Error running executor
java.io.IOException: Failed to create directory ./work/app-20150115131259-0000/0
at org.apache.spark.deploy.worker.ExecutorRunner.fetchAndRunExecutor(ExecutorRunner.scala:121)
at org.apache.spark.deploy.worker.ExecutorRunner$$anon$1.run(ExecutorRunner.scala:58)
15/01/15 13:12:59.528 INFO Worker: Executor app-20150115131259-0000/0 finished with state FAILED message java.io.IOException: Failed to create directory ./work/app-20150115131259-0000/0
15/01/15 13:12:59.531 INFO Master: Removing executor app-20150115131259-0000/0 because it is FAILED
15/01/15 13:12:59.531 INFO Master: Launching executor app-20150115131259-0000/1 on worker worker-20150115131259-localhost-52186
15/01/15 13:12:59.532 INFO Worker: Asked to launch executor app-20150115131259-0000/1 for test
15/01/15 13:12:59.533 ERROR ExecutorRunner: Error running executor
java.io.IOException: Failed to create directory ./work/app-20150115131259-0000/1
at org.apache.spark.deploy.worker.ExecutorRunner.fetchAndRunExecutor(ExecutorRunner.scala:121)
at org.apache.spark.deploy.worker.ExecutorRunner$$anon$1.run(ExecutorRunner.scala:58)
15/01/15 13:12:59.534 INFO Worker: Executor app-20150115131259-0000/1 finished with state FAILED message java.io.IOException: Failed to create directory ./work/app-20150115131259-0000/1
15/01/15 13:12:59.535 INFO Master: Removing executor app-20150115131259-0000/1 because it is FAILED
15/01/15 13:12:59.535 INFO Master: Launching executor app-20150115131259-0000/2 on worker worker-20150115131259-localhost-52186
15/01/15 13:12:59.535 WARN Master: Got status update for unknown executor app-20150115131259-0000/1
15/01/15 13:12:59.536 INFO Worker: Asked to launch executor app-20150115131259-0000/2 for test
15/01/15 13:12:59.536 ERROR ExecutorRunner: Error running executor
[...]

Kind of hard to figure out which tests were hitting this (since we don't have any helpful Log4J messages to let us know which test was running when log messages were written), but I'm going to assume that this is the failed test.

Looks like maybe this is a path / environment variable / CWD issue. I'll investigate.

andrewor14 · 2015-01-15T22:35:07Z

Interesting. Thanks for looking into this more @JoshRosen. This looks a lot like a Spark home issue that I've seen in the past, where the executors on different machines inherit the same Spark home from the driver and use this Spark home to call bin/compute-classpath.sh. That problem is fixed and not really applicable here, however.

tdas · 2015-02-04T06:29:58Z

Whats the state of this PR? Is it still relevant?

andrewor14 · 2015-02-06T19:52:44Z

yes, the tests are still not passing I believe, test this please

SparkQA · 2015-02-06T19:58:09Z

Test build #26933 has started for PR 3959 at commit 5425314.

This patch merges cleanly.

SparkQA · 2015-02-06T21:14:30Z

Test build #26933 has finished for PR 3959 at commit 5425314.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-02-06T21:14:34Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26933/
Test FAILed.

JoshRosen · 2015-02-08T07:16:05Z

Jenkins, retest this please.

SparkQA · 2015-02-08T07:18:02Z

Test build #27032 has started for PR 3959 at commit 5425314.

This patch merges cleanly.

SparkQA · 2015-02-08T08:33:20Z

Test build #27032 has finished for PR 3959 at commit 5425314.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-02-08T08:33:25Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27032/
Test FAILed.

Disable UI for tests
5425314

This was referenced Jan 9, 2015
[Spark-3490] Disable SparkUI for tests (backport into 0.9) #3961
Closed
[SPARK-3490] Disable SparkUI for tests (backport into 1.1) #2415
Closed
[Spark-3490] Disable SparkUI for tests #2363
Closed

andrewor14 closed this Mar 3, 2015

andrewor14 deleted the ui-ports branch March 3, 2015 01:06

Uh oh!

Conversation

andrewor14 commented Jan 8, 2015

Uh oh!

SparkQA commented Jan 8, 2015

Uh oh!

SparkQA commented Jan 9, 2015

Uh oh!

AmplabJenkins commented Jan 9, 2015

Uh oh!

andrewor14 commented Jan 9, 2015

Uh oh!

SparkQA commented Jan 9, 2015

Uh oh!

SparkQA commented Jan 9, 2015

Uh oh!

AmplabJenkins commented Jan 9, 2015

Uh oh!

andrewor14 commented Jan 9, 2015

Uh oh!

JoshRosen commented Jan 9, 2015

Uh oh!

andrewor14 commented Jan 9, 2015

Uh oh!

SparkQA commented Jan 9, 2015

Uh oh!

SparkQA commented Jan 10, 2015

Uh oh!

AmplabJenkins commented Jan 10, 2015

Uh oh!

JoshRosen commented Jan 13, 2015

Uh oh!

SparkQA commented Jan 13, 2015

Uh oh!

SparkQA commented Jan 13, 2015

Uh oh!

AmplabJenkins commented Jan 13, 2015

Uh oh!

JoshRosen commented Jan 15, 2015

Uh oh!

SparkQA commented Jan 15, 2015

Uh oh!

SparkQA commented Jan 15, 2015

Uh oh!

AmplabJenkins commented Jan 15, 2015

Uh oh!

JoshRosen commented Jan 15, 2015

Uh oh!

andrewor14 commented Jan 15, 2015

Uh oh!

tdas commented Feb 4, 2015

Uh oh!

andrewor14 commented Feb 6, 2015

Uh oh!

SparkQA commented Feb 6, 2015

Uh oh!

SparkQA commented Feb 6, 2015

Uh oh!

AmplabJenkins commented Feb 6, 2015

Uh oh!

JoshRosen commented Feb 8, 2015

Uh oh!

SparkQA commented Feb 8, 2015

Uh oh!

SparkQA commented Feb 8, 2015

Uh oh!

AmplabJenkins commented Feb 8, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants