Skip to content

TEZ-4299: Default java opts cause jdk11 to fail - #116

Merged
abstractdog merged 1 commit into
apache:masterfrom
abstractdog:TEZ-4299
Apr 5, 2021
Merged

TEZ-4299: Default java opts cause jdk11 to fail#116
abstractdog merged 1 commit into
apache:masterfrom
abstractdog:TEZ-4299

Conversation

@abstractdog

Copy link
Copy Markdown
Contributor

No description provided.

@abstractdog

abstractdog commented Apr 2, 2021

Copy link
Copy Markdown
ContributorAuthor

@jteagles: could you please take a look? the patch solves the problem with automatic java version detection
it assumes that the tez client runs on the same java version as AM/tasks (because the resolution of the launch opts is done there), I think this is a reasonable assumption.

btw: I used earlier experience from HIVE-23938

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 33sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall13m 18smaster passed
+1 💚compile0m 34smaster passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚compile0m 33smaster passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚checkstyle1m 11smaster passed
+1 💚javadoc0m 45smaster passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚javadoc0m 36smaster passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+0 🆗spotbugs1m 32sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 30smaster passed
_ Patch Compile Tests _
+1 💚mvninstall0m 20sthe patch passed
+1 💚compile0m 20sthe patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚javac0m 20sthe patch passed
+1 💚compile0m 19sthe patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚javac0m 20sthe patch passed
+1 💚checkstyle0m 13sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 21sthe patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚javadoc0m 22sthe patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚findbugs0m 54sthe patch passed
_ Other Tests _
+1 💚unit1m 55stez-api in the patch passed.
+1 💚asflicense0m 14sThe patch does not generate ASF License warnings.
24m 54s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/2/artifact/out/Dockerfile
GITHUB PR#116
JIRA IssueTEZ-4299
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux f2a627bf7a66 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 2dcbe0b
Default JavaPrivate Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/2/testReport/
Max. process+thread count401 (vs. ulimit of 5500)
modulesC: tez-api U: tez-api
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/2/console
versionsgit=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

public static final String TEZ_AM_LAUNCH_CMD_OPTS = TEZ_AM_PREFIX + "launch.cmd-opts";
public static final String TEZ_AM_LAUNCH_CMD_OPTS_DEFAULT =
"-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC";
public static final String TEZ_AM_LAUNCH_CMD_OPTS_JDK9_DEFAULT =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we create TEZ_AM_LAUNCH_CMD_OPTS_JDK8_DEFAULT with the origin value of TEZ_AM_LAUNCH_CMD_OPTS_DEFAULT.
Then we could statically assign TEZ_AM_LAUNCH_CMD_OPTS_DEFAULT to correct value of JDK8 or JDK9+. This would remove the runtime detection.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, I'm doing it in the next commit

}

private static int getJavaVersion() {
String javaVersionString = System.getProperty("java.version");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally detected the presence of java.lang.runtime.Version class to avoid parsing the version string. Knowing the actual version could be more useful in the future though so let's try to keep this for now.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec12m 53sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall13m 23smaster passed
+1 💚compile0m 35smaster passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚compile0m 34smaster passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚checkstyle1m 12smaster passed
+1 💚javadoc0m 45smaster passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚javadoc0m 36smaster passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+0 🆗spotbugs1m 28sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 25smaster passed
_ Patch Compile Tests _
+1 💚mvninstall0m 19sthe patch passed
+1 💚compile0m 22sthe patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚javac0m 22sthe patch passed
+1 💚compile0m 19sthe patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚javac0m 19sthe patch passed
+1 💚checkstyle0m 13sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 22sthe patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚javadoc0m 22sthe patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚findbugs0m 56sthe patch passed
_ Other Tests _
+1 💚unit1m 56stez-api in the patch passed.
+1 💚asflicense0m 15sThe patch does not generate ASF License warnings.
37m 22s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/3/artifact/out/Dockerfile
GITHUB PR#116
JIRA IssueTEZ-4299
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux ca86ebb4b3d1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 2dcbe0b
Default JavaPrivate Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/3/testReport/
Max. process+thread count327 (vs. ulimit of 5500)
modulesC: tez-api U: tez-api
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/3/console
versionsgit=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdog

Copy link
Copy Markdown
ContributorAuthor

@jteagles: are you fine with merging this?

@jteaglesjteagles left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Let's put this in master and branch-0.9

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@abstractdog@hadoop-yetus@jteagles