Skip to content

[SPARK-23449][K8S] Preserve extraJavaOptions ordering - #20628

Closed
andrusha wants to merge 1 commit into
apache:masterfrom
andrusha:patch-2
Closed

[SPARK-23449][K8S] Preserve extraJavaOptions ordering#20628
andrusha wants to merge 1 commit into
apache:masterfrom
andrusha:patch-2

Conversation

@andrusha

@andrushaandrusha commented Feb 16, 2018

Copy link
Copy Markdown
Contributor

For some JVM options, like -XX:+UnlockExperimentalVMOptions ordering is necessary.

What changes were proposed in this pull request?

Keep original extraJavaOptions ordering, when passing them through environment variables inside the Docker container.

How was this patch tested?

Ran base branch a couple of times and checked startup command in logs. Ordering differed every time. Added sorting, ordering was consistent to what user had in extraJavaOptions.

For some JVM options, like `-XX:+UnlockExperimentalVMOptions` ordering is necessary.
@andrushaandrusha changed the title Preserve extraJavaOptions ordering[SPARK-23449] Preserve extraJavaOptions orderingFeb 16, 2018
@andrushaandrusha changed the title [SPARK-23449] Preserve extraJavaOptions ordering[SPARK-23449][K8S] Preserve extraJavaOptions orderingFeb 16, 2018

@jiangxb1987jiangxb1987 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiangxb1987

Copy link
Copy Markdown
Contributor

also cc @vanzin to verify this change.

@vanzin

Copy link
Copy Markdown
Contributor

ok to test

@SparkQA

Copy link
Copy Markdown

Test build #87680 has finished for PR 20628 at commit 6759e9e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@vanzin

Copy link
Copy Markdown
Contributor

LGTM, pending tests. I hope we can just use spark-submit here at some point.

@vanzin

Copy link
Copy Markdown
Contributor

Merging to master / 2.3.

asfgit pushed a commit that referenced this pull request Feb 26, 2018
For some JVM options, like `-XX:+UnlockExperimentalVMOptions` ordering is necessary.
## What changes were proposed in this pull request?
Keep original `extraJavaOptions` ordering, when passing them through environment variables inside the Docker container.
## How was this patch tested?
Ran base branch a couple of times and checked startup command in logs. Ordering differed every time. Added sorting, ordering was consistent to what user had in `extraJavaOptions`.
Author: Andrew Korzhuev <korzhuev@andrusha.me>
Closes#20628 from andrusha/patch-2.
(cherry picked from commit 185f5bc)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
@andrusha
andrusha deleted the patch-2 branch March 13, 2018 14:24
andrusha added a commit to andrusha/spark that referenced this pull request Apr 5, 2018
For some JVM options, like `-XX:+UnlockExperimentalVMOptions` ordering is necessary.
## What changes were proposed in this pull request?
Keep original `extraJavaOptions` ordering, when passing them through environment variables inside the Docker container.
## How was this patch tested?
Ran base branch a couple of times and checked startup command in logs. Ordering differed every time. Added sorting, ordering was consistent to what user had in `extraJavaOptions`.
Author: Andrew Korzhuev <korzhuev@andrusha.me>
Closesapache#20628 from andrusha/patch-2.
(cherry picked from commit 185f5bc)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
peter-toth pushed a commit to peter-toth/spark that referenced this pull request Oct 6, 2018
For some JVM options, like `-XX:+UnlockExperimentalVMOptions` ordering is necessary.
## What changes were proposed in this pull request?
Keep original `extraJavaOptions` ordering, when passing them through environment variables inside the Docker container.
## How was this patch tested?
Ran base branch a couple of times and checked startup command in logs. Ordering differed every time. Added sorting, ordering was consistent to what user had in `extraJavaOptions`.
Author: Andrew Korzhuev <korzhuev@andrusha.me>
Closesapache#20628 from andrusha/patch-2.
(cherry picked from commit 185f5bc)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@andrusha@jiangxb1987@vanzin@SparkQA