[SPARK-5951][YARN] Remove unreachable driver memory properties in yarn client mode

[zuxqoj]

Remove unreachable driver memory properties in yarn client mode

[AmplabJenkins]

Can one of the admins verify this patch?

[sryza]

LGTM. This seems in line with SPARK-1953's eradication of these properties for yarn-client mode.

[srowen]

Does @andrewor14 's comment at #3590 (comment) affect this PR? should more stuff be removed for 1.4?

[zuxqoj]

@srowen
Does @andrewor14 's comment at #3590 (comment) affect this PR?
no, he is talking about removing deprecated properties (SPARK_MASTER_MEMORY and spark.master.memory) from spark in 1.4

[andrewor14]

ok to test

[SparkQA]

Test build #27894 has started for PR 4730 at commit 977dc96.

• This patch merges cleanly.

[andrewor14]

@zuxqoj Thanks for this PR. Actually I'd suggest that we just remove all the deprecated properties here. Calling a Spark executor a "Worker" is just as misleading as calling a Spark driver a "Master".

[SparkQA]

Test build #27894 has finished for PR 4730 at commit 977dc96.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27894/
Test PASSed.

[andrewor14]

I just realized that this patch doesn't actually remove support for those deprecated configs; they were already removed in a patch #3607 that was merged beforehand. What this patch does is just to stop showing the deprecation messages for these configs that were no longer even supported.

LGTM I'm merging this into master and 1.3. Removing support for other deprecated env vars and configs does seem like a different issue.