Set master URL configuration in scala example

[ganeshkrishnan1]

This is the Spark Scala example which was missing setting a master URL in Spark Session

Unit tested. Changes affect examples and documentation only

There is no UI change. It's a minor change in the example scala files.

Need to set master url to SparkSession for the example to run

[rxin]

LGTM - can you clean up the pr description to remove the messages from the template?

[hhbyyh]

@rxin Shall we set default master for all the examples, or this is a special case?

[srowen]

Ping @getintouchapp

[ganeshkrishnan1]

Am I supposed to do anything here? I cleaned up the comments and I don't have permission to do anything else beyond that.

[srowen]

Yes thanks for updating the description. The title could be better. Yuhao also asked if this is something that looks like it needs to be done for more examples?

[ganeshkrishnan1]

Fixed the title. Yes other examples are missing the master url config too. I will add them and create a pull request for the rest of the files once this is merged.

[ganeshkrishnan1]

btw, for the rest of the files its not a major issue because they don't turn up in documentation. This one does at http://spark.apache.org/docs/latest/sql-programming-guide.html#starting-point-sparksession

[srowen]

I think it's right to do this all at once, rather than in pieces.

I must say I recall I opened a PR like this a long long time ago and Matei said the master was excluded on purpose because it was intended to be set by the environment running the example, which made some sense.

I wonder if the logic is still the same or not? that is, if it's a runnable example, do we want to not override the master that the runner might set? or for doc-only example code, is it that we do need to show master being set programmatically?

[ganeshkrishnan1]

It does make sense to load master url dynamically thru environment or command line while running the example. However the documentation example should be a valid piece of code and not dependent on other hidden variables.

I vote that we merge this pull request for the sake of clarity in documentation but for other examples we leave as is, since they could load the master URL dynamically

[srowen]

OK are there other such examples that are of the same form? it makes sense to change any case like this in one go.

[SparkQA]

Test build #3333 has finished for PR 15411 at commit 5324034.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

Hm, so I see several more examples that get included in documentation that don't set master. I am not sure that is a salient difference, because in general, when writing your own app you would not hard-code a master in the code either. The examples evidently don't generally set master for this reason. Therefore I'm not sure we shoudl make this change.