Skip to content

[SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is needed only when use… - #18061

Closed
wujianping10043419 wants to merge 1 commit into
apache:masterfrom
wujianping10043419:spark-master-20170519
Closed

[SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is needed only when use…#18061
wujianping10043419 wants to merge 1 commit into
apache:masterfrom
wujianping10043419:spark-master-20170519

Conversation

@wujianping10043419

Copy link
Copy Markdown

…rDeployMode is 'client'. If userDeployMode is not 'client', the launcher shouldn't care about DRIVER_EXTRA_JAVA_OPTIONS, nor should it exit abnormally because DRIVER_EXTRA_JAVA_OPTIONS contains Xmx configuration.

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

Please review http://spark.apache.org/contributing.html before opening a pull request.

…rDeployMode is 'client'. If userDeployMode is not 'client', the launcher shouldn't care about DRIVER_EXTRA_JAVA_OPTIONS, nor should it exit abnormally because DRIVER_EXTRA_JAVA_OPTIONS contains Xmx configuration.
@AmplabJenkins

Copy link
Copy Markdown

Can one of the admins verify this patch?

@srowen

Copy link
Copy Markdown
Member

Although this could be moved, probably, it's not really worth bothering. It isn't actually redundant. If you specify driver memory settings in non-client mode, that should also be an error. Please close this.

@wujianping10043419

Copy link
Copy Markdown
Author

@srowen
The DRIVER_EXTRA_JAVA_OPTIONS is from a file. Maybe, Setting the parameter is an error. But Spark doesn't use it in non-client mode, nobody should pay for food who doesn't eat,so I think the code is not robust.

@srowen

Copy link
Copy Markdown
Member

The check is truly trivial, and the check is not actually irrelevant to non-client mode.

@wujianping10043419

Copy link
Copy Markdown
Author

@srowen But the check result in the abnormal exit, because an irrelevant parameters.

@srowen

Copy link
Copy Markdown
Member

This check is still correct for cluster mode. You should set memory with --driver-memory.

@srowensrowen mentioned this pull request Jun 7, 2017
@asfgitasfgit closed this in b771fedJun 8, 2017
zifeif2 pushed a commit to zifeif2/spark that referenced this pull request Nov 22, 2025
# What changes were proposed in this pull request?
This PR proposes to close stale PRs, mostly the same instances with apache#18017Closesapache#11459Closesapache#13833Closesapache#13720Closesapache#12506Closesapache#12456Closesapache#12252Closesapache#17689Closesapache#17791Closesapache#18163Closesapache#17640Closesapache#17926Closesapache#18163Closesapache#12506Closesapache#18044Closesapache#14036Closesapache#15831Closesapache#14461Closesapache#17638Closesapache#18222
Added:
Closesapache#18045Closesapache#18061Closesapache#18010Closesapache#18041Closesapache#18124Closesapache#18130Closesapache#12217
Added:
Closesapache#16291Closesapache#17480Closesapache#14995
Added:
Closesapache#12835Closesapache#17141
## How was this patch tested?
N/A
Author: hyukjinkwon <gurwls223@gmail.com>
Closesapache#18223 from HyukjinKwon/close-stale-prs.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wujianping10043419@AmplabJenkins@srowen