Uh oh!
There was an error while loading. Please reload this page.
[SPARK-1896] Respect spark.master (and --master) before MASTER in spark-shell - #846
Closed
andrewor14 wants to merge 1 commit into
Closed
[SPARK-1896] Respect spark.master (and --master) before MASTER in spark-shell#846andrewor14 wants to merge 1 commit into
andrewor14 wants to merge 1 commit into
Conversation
AmplabJenkins
commented
May 21, 2014
Merged build triggered. |
AmplabJenkins
commented
May 21, 2014
Merged build started. |
AmplabJenkins
commented
May 21, 2014
Merged build finished. |
AmplabJenkins
commented
May 21, 2014
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15122/ |
andrewor14
commented
May 21, 2014
ContributorAuthor
Jenkins test this please |
AmplabJenkins
commented
May 21, 2014
Merged build triggered. |
AmplabJenkins
commented
May 21, 2014
Merged build started. |
AmplabJenkins
commented
May 21, 2014
Merged build finished. All automated tests passed. |
AmplabJenkins
commented
May 21, 2014
All automated tests passed. |
tdas
commented
May 23, 2014
Contributor
LGTM. Merging this. Thanks! |
asfgit pushed a commit
that referenced
this pull request
May 23, 2014
…rk-shell The hierarchy for configuring the Spark master in the shell is as follows: ``` MASTER > --master > spark.master (spark-defaults.conf) ``` This is inconsistent with the way we run normal applications, which is: ``` --master > spark.master (spark-defaults.conf) > MASTER ``` I was trying to run a shell locally on a standalone cluster launched through the ec2 scripts, which automatically set `MASTER` in spark-env.sh. It was surprising to me that `--master` didn't take effect, considering that this is the way we tell users to set their masters [here](http://people.apache.org/~pwendell/spark-1.0.0-rc7-docs/scala-programming-guide.html#initializing-spark). Author: Andrew Or <andrewor14@gmail.com> Closes#846 from andrewor14/shell-master and squashes the following commits: 2cb81c9 [Andrew Or] Respect spark.master before MASTER in REPL (cherry picked from commit cce7745) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
pdeyhim pushed a commit
to pdeyhim/spark-1
that referenced
this pull request
Jun 25, 2014
…rk-shell The hierarchy for configuring the Spark master in the shell is as follows: ``` MASTER > --master > spark.master (spark-defaults.conf) ``` This is inconsistent with the way we run normal applications, which is: ``` --master > spark.master (spark-defaults.conf) > MASTER ``` I was trying to run a shell locally on a standalone cluster launched through the ec2 scripts, which automatically set `MASTER` in spark-env.sh. It was surprising to me that `--master` didn't take effect, considering that this is the way we tell users to set their masters [here](http://people.apache.org/~pwendell/spark-1.0.0-rc7-docs/scala-programming-guide.html#initializing-spark). Author: Andrew Or <andrewor14@gmail.com> Closesapache#846 from andrewor14/shell-master and squashes the following commits: 2cb81c9 [Andrew Or] Respect spark.master before MASTER in REPL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The hierarchy for configuring the Spark master in the shell is as follows:
This is inconsistent with the way we run normal applications, which is:
I was trying to run a shell locally on a standalone cluster launched through the ec2 scripts, which automatically set
MASTERin spark-env.sh. It was surprising to me that--masterdidn't take effect, considering that this is the way we tell users to set their masters here.