Skip to content

TEZ-4677: Upgrades Tez to compile and run with JDK 25 - #482

Open
maheshrajus wants to merge 7 commits into
apache:masterfrom
maheshrajus:TEZ-4677
Open

TEZ-4677: Upgrades Tez to compile and run with JDK 25#482
maheshrajus wants to merge 7 commits into
apache:masterfrom
maheshrajus:TEZ-4677

Conversation

@maheshrajus

@maheshrajusmaheshrajus commented Apr 15, 2026

Copy link
Copy Markdown
Contributor
  • Upgrades Tez source compilation target to JDK 25
  • Updates Yetus CI Docker image to JDK 25 for pre-commit testing

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/1/console in case of problems.

@tez-yetus

This comment was marked as outdated.

@maheshrajus

Copy link
Copy Markdown
ContributorAuthor

It looks like the Jenkinsfile and Dockerfile have changed and been optimized in the latest commits. I need to rebase with the latest changes and check on this task.

@Aggarwal-Raghav, if we want to test Tez with JDK 25, which fields in the latest Jenkinsfile and Dockerfile do we need to modify? As far as I can see in the latest changes, there are no fields related to the JDK version.

@Aggarwal-Raghav

Aggarwal-Raghav commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

@maheshrajus , please check this.

  • build-tools/docker/Dockerfile basically the eclipse-temurin:21-jdk-noble
  • parent pom
<javaVersion>21</javaVersion>
<maven.compiler.release>21</maven.compiler.release>
  • in tez-dist/src/docker/Dockerfile docker image eclipse-temurin:21-jdk-ubi9-minimaloptional
  • build.yml is already handled in recent commits for github actions

Comment threadtez-plugins/tez-aux-services/pom.xml Outdated
@maheshrajusmaheshrajus changed the title [WIP] TEZ-4677: JDK25 testing in yetusTEZ-4677: Upgrades Tez to compile and run with JDK 25Jun 26, 2026
@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/2/console in case of problems.

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/3/console in case of problems.

Comment threadpom.xml Outdated
@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/4/console in case of problems.

@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/5/console in case of problems.

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor

@maheshrajus , now the spotbugs have failed because its incompatible with jdk25. I'll investigate it in some time to see which is better approach (maven-shading or apache pom version) from my perspective and share my findings as both have some tradeoffs in terms of error

@Aggarwal-Raghav

Aggarwal-Raghav commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

This much of the patch should be enough. It is passing in local. Regarding ubi10 (RHEL10) in docker image, I don't have any strong opinion on that. Also update the README.md or other docs/md where JDK version, spotbugs tools etc. version are mentioned. For example:

tez/README.md

Lines 49 to 53 in 17d1549

* JDK 21+
* Maven 3.9.14 or later
* spotbugs 4.9.3 or later (if running spotbugs)
* ProtocolBuffer 3.25.5
* Hadoop 3.x

jdk25.patch

@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/6/console in case of problems.

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/7/console in case of problems.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec4m 12sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+0 🆗shelldocs0m 0sShelldocs was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 7 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep0m 39sMaven dependency ordering for branch
+1 💚mvninstall3m 6smaster passed
+1 💚compile2m 41smaster passed
+1 💚checkstyle2m 17smaster passed
+1 💚javadoc2m 39smaster passed
-1 ❌spotbugs0m 29s/branch-spotbugs-tez-runtime-internals.txttez-runtime-internals in master failed.
-1 ❌spotbugs0m 29s/branch-spotbugs-tez-runtime-library.txttez-runtime-library in master failed.
-1 ❌spotbugs0m 33s/branch-spotbugs-tez-dag.txttez-dag in master failed.
-1 ❌spotbugs0m 22s/branch-spotbugs-tez-tests.txttez-tests in master failed.
-1 ❌spotbugs0m 21s/branch-spotbugs-root.txtroot in master failed.
+0 🆗spotbugs0m 19sbranch/tez-dist no spotbugs output file (spotbugsXml.xml)
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 27sthe patch passed
+1 💚codespell1m 1sNo new issues.
+1 💚compile2m 33sthe patch passed
+1 💚javac2m 33sroot generated 0 new + 748 unchanged - 22 fixed = 748 total (was 770)
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 19stez-runtime-internals: The patch generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2)
+1 💚checkstyle0m 21stez-runtime-library: The patch generated 0 new + 43 unchanged - 2 fixed = 43 total (was 45)
+1 💚checkstyle0m 23sThe patch passed checkstyle in tez-dag
+1 💚checkstyle0m 14stez-tests: The patch generated 0 new + 72 unchanged - 2 fixed = 72 total (was 74)
+1 💚checkstyle0m 9sThe patch passed checkstyle in tez-dist
+1 💚checkstyle0m 35sroot: The patch generated 0 new + 139 unchanged - 6 fixed = 139 total (was 145)
+1 💚hadolint0m 0sNo new issues.
+1 💚markdownlint0m 1sNo new issues.
+1 💚shellcheck0m 0sNo new issues.
+1 💚javadoc0m 15stez-runtime-internals in the patch passed.
+1 💚javadoc0m 25stez-runtime-library generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1)
+1 💚javadoc0m 20stez-dag in the patch passed.
+1 💚javadoc0m 15stez-tests in the patch passed.
+1 💚javadoc0m 18stez-dist in the patch passed.
+1 💚javadoc1m 0sroot generated 0 new + 5 unchanged - 1 fixed = 5 total (was 6)
+0 🆗spotbugs0m 17stez-dist has no data from spotbugs
_ Other Tests _
+1 💚unit60m 59sroot in the patch passed.
+1 💚asflicense2m 11sThe patch does not generate ASF License warnings.
102m 25s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/7/artifact/out/Dockerfile
Optional Testsdupname compile unit asflicense codespell detsecrets hadolint shellcheck shelldocs javac javadoc xmllint markdownlint spotbugs checkstyle
unameLinux 83053a08fe07 5.15.0-179-generic #189-Ubuntu SMP Tue May 5 18:20:56 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitytez-personality.sh
git revisionmaster / 17d1549
Default JavaEclipse Adoptium-25.0.3+9-LTS
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/7/testReport/
Max. process+thread count2106 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-runtime-library tez-dag tez-tests tez-dist . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/7/console
versionsgit=2.43.0 maven=3.9.15 hadolint=2.14.0 codespell=2.4.1 markdownlint=0.46.0 shellcheck=0.11.0
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Comment threadtez-tests/src/test/java/org/apache/tez/test/TestInput.java
@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor

LGTM +1 for pom, Dockerfile changes. JDK25 changes look good. For checkstyle plese consult with a committer/PMC whether to do in this PR or separate?

@maheshrajus

Copy link
Copy Markdown
ContributorAuthor

@Aggarwal-Raghav All checkstyle issues related to jdk 25 so better to include in this PR itself.

@abstractdog Could you please review the PR at your convenience?
Thank you !

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/8/console in case of problems.

@maheshrajus

Copy link
Copy Markdown
ContributorAuthor

@abstractdog spotbugs checked internally and they are running fine.

jdk25_spotbugs_report.txt

Could you please review the PR at your convenience?
Thank you !

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec3m 13sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+0 🆗shelldocs0m 0sShelldocs was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 7 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep0m 36sMaven dependency ordering for branch
+1 💚mvninstall2m 54smaster passed
+1 💚compile2m 31smaster passed
+1 💚checkstyle2m 10smaster passed
+1 💚javadoc2m 31smaster passed
-1 ❌spotbugs0m 27s/branch-spotbugs-tez-runtime-internals.txttez-runtime-internals in master failed.
-1 ❌spotbugs0m 27s/branch-spotbugs-tez-runtime-library.txttez-runtime-library in master failed.
-1 ❌spotbugs0m 32s/branch-spotbugs-tez-dag.txttez-dag in master failed.
-1 ❌spotbugs0m 26s/branch-spotbugs-tez-tests.txttez-tests in master failed.
-1 ❌spotbugs0m 22s/branch-spotbugs-root.txtroot in master failed.
+0 🆗spotbugs0m 18sbranch/tez-dist no spotbugs output file (spotbugsXml.xml)
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 9sthe patch passed
+1 💚codespell0m 49sNo new issues.
+1 💚compile2m 28sthe patch passed
+1 💚javac2m 28sroot generated 0 new + 747 unchanged - 22 fixed = 747 total (was 769)
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 18stez-runtime-internals: The patch generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2)
+1 💚checkstyle0m 21stez-runtime-library: The patch generated 0 new + 41 unchanged - 2 fixed = 41 total (was 43)
+1 💚checkstyle0m 22sThe patch passed checkstyle in tez-dag
+1 💚checkstyle0m 17stez-tests: The patch generated 0 new + 72 unchanged - 2 fixed = 72 total (was 74)
+1 💚checkstyle0m 14sThe patch passed checkstyle in tez-dist
+1 💚checkstyle0m 35sroot: The patch generated 0 new + 136 unchanged - 6 fixed = 136 total (was 142)
+1 💚hadolint0m 1sNo new issues.
+1 💚markdownlint0m 1sNo new issues.
+1 💚shellcheck0m 0sNo new issues.
+1 💚javadoc0m 19stez-runtime-internals in the patch passed.
+1 💚javadoc0m 23stez-runtime-library generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1)
+1 💚javadoc0m 20stez-dag in the patch passed.
+1 💚javadoc0m 18stez-tests in the patch passed.
+1 💚javadoc0m 17stez-dist in the patch passed.
+1 💚javadoc0m 55sroot generated 0 new + 5 unchanged - 1 fixed = 5 total (was 6)
+0 🆗spotbugs0m 20stez-dist has no data from spotbugs
_ Other Tests _
+1 💚unit60m 6sroot in the patch passed.
+1 💚asflicense2m 6sThe patch does not generate ASF License warnings.
98m 53s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/8/artifact/out/Dockerfile
Optional Testsdupname compile unit asflicense codespell detsecrets hadolint shellcheck shelldocs javac javadoc xmllint markdownlint spotbugs checkstyle
unameLinux e706a9a8f8dd 5.15.0-179-generic #189-Ubuntu SMP Tue May 5 18:20:56 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitytez-personality.sh
git revisionmaster / d53e67d
Default JavaEclipse Adoptium-25.0.3+9-LTS
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/8/testReport/
Max. process+thread count1484 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-runtime-library tez-dag tez-tests tez-dist . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/8/console
versionsgit=2.43.0 maven=3.9.15 hadolint=2.14.0 codespell=2.4.1 markdownlint=0.46.0 shellcheck=0.11.0
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@maheshrajus

Copy link
Copy Markdown
ContributorAuthor

@abstractdog@ayushtkn
Could you please review the PR at your convenience?
Thank you !

@abstractdog

Copy link
Copy Markdown
Contributor

is this expected in spotbugs?

[INFO] --- spotbugs:4.9.3.0:spotbugs (default-cli) @ hadoop-shim ---
[INFO] Fork Value is true
[java] The following errors occurred during analysis:
[java] Error scanning java/lang/Object for referenced classes
[java] java.lang.IllegalArgumentException: Unsupported class file major version 69

as far as I know, if you change the Dockerfile, it's picked up, so the new spotbugs is supposed to be used

@maheshrajus

Copy link
Copy Markdown
ContributorAuthor

@abstractdog Yeah, it should pick SPOTBUGS_VERSION="4.10.2" since I set that same version in the Dockerfile, but it is taking spotbugs:4.9.3.0:spotbugs and running the checks with that instead. We previously faced a similar issue with JDK 17 where Yetus did not respect the updated SpotBugs version. I tested locally with SPOTBUGS_VERSION="4.10.2" and didn't find any issues. We also merged and raised a dummy PR to verify this behavior.

@abstractdog

Copy link
Copy Markdown
Contributor

@abstractdog Yeah, it should pick SPOTBUGS_VERSION="4.10.2" since I set that same version in the Dockerfile, but it is taking spotbugs:4.9.3.0:spotbugs and running the checks with that instead. We previously faced a similar issue with JDK 17 where Yetus did not respect the updated SpotBugs version. I tested locally with SPOTBUGS_VERSION="4.10.2" and didn't find any issues. We also merged and raised a dummy PR to verify this behavior.

thanks @maheshrajus

Error: Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.6.2:enforce (enforce-java-version) on project tez: Error: Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion failed with message:
Error: Detected JDK /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/21.0.11-10.0.LTS/arm64 is version 21.0.11 which is not in the allowed range [25,).
Error: -> [Help 1]

I believe the Java 21-related GitHub workflows should be removed once we assume JDK 25 minimum support

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/9/console in case of problems.

@maheshrajus

Copy link
Copy Markdown
ContributorAuthor

I believe the Java 21-related GitHub workflows should be removed once we assume JDK 25 minimum support

@abstractdog i just kept jdk 25 in github workflows.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec36m 41sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+0 🆗shelldocs0m 0sShelldocs was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 7 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep0m 31sMaven dependency ordering for branch
+1 💚mvninstall3m 6smaster passed
+1 💚compile2m 45smaster passed
+1 💚checkstyle2m 9smaster passed
+1 💚javadoc2m 9smaster passed
-1 ❌spotbugs0m 29s/branch-spotbugs-tez-runtime-internals.txttez-runtime-internals in master failed.
-1 ❌spotbugs0m 27s/branch-spotbugs-tez-runtime-library.txttez-runtime-library in master failed.
-1 ❌spotbugs0m 34s/branch-spotbugs-tez-dag.txttez-dag in master failed.
-1 ❌spotbugs0m 23s/branch-spotbugs-tez-tests.txttez-tests in master failed.
-1 ❌spotbugs0m 18s/branch-spotbugs-root.txtroot in master failed.
+0 🆗spotbugs0m 19sbranch/tez-dist no spotbugs output file (spotbugsXml.xml)
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall2m 23sthe patch passed
+1 💚codespell1m 3sNo new issues.
+1 💚compile2m 37sthe patch passed
+1 💚javac2m 37sroot generated 0 new + 747 unchanged - 22 fixed = 747 total (was 769)
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 19stez-runtime-internals: The patch generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2)
+1 💚checkstyle0m 20stez-runtime-library: The patch generated 0 new + 41 unchanged - 2 fixed = 41 total (was 43)
+1 💚checkstyle0m 19sThe patch passed checkstyle in tez-dag
+1 💚checkstyle0m 17stez-tests: The patch generated 0 new + 72 unchanged - 2 fixed = 72 total (was 74)
+1 💚checkstyle0m 13sThe patch passed checkstyle in tez-dist
+1 💚checkstyle0m 37sroot: The patch generated 0 new + 136 unchanged - 6 fixed = 136 total (was 142)
+1 💚hadolint0m 1sNo new issues.
+1 💚markdownlint0m 2sNo new issues.
+1 💚shellcheck0m 0sNo new issues.
+1 💚yamllint0m 0sNo new issues.
+1 💚javadoc0m 16stez-runtime-internals in the patch passed.
+1 💚javadoc0m 25stez-runtime-library generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1)
+1 💚javadoc0m 20stez-dag in the patch passed.
+1 💚javadoc0m 19stez-tests in the patch passed.
+1 💚javadoc0m 18stez-dist in the patch passed.
+1 💚javadoc0m 55sroot generated 0 new + 5 unchanged - 1 fixed = 5 total (was 6)
+0 🆗spotbugs0m 18stez-dist has no data from spotbugs
_ Other Tests _
+1 💚unit61m 38sroot in the patch passed.
+1 💚asflicense2m 7sThe patch does not generate ASF License warnings.
134m 23s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/9/artifact/out/Dockerfile
Optional Testsdupname compile unit asflicense codespell detsecrets hadolint shellcheck shelldocs yamllint javac javadoc xmllint markdownlint spotbugs checkstyle
unameLinux 0556b97520bf 5.15.0-179-generic #189-Ubuntu SMP Tue May 5 18:20:56 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitytez-personality.sh
git revisionmaster / 41145de
Default JavaEclipse Adoptium-25.0.3+9-LTS
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/9/testReport/
Max. process+thread count1458 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-runtime-library tez-dag tez-tests tez-dist . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/9/console
versionsgit=2.43.0 maven=3.9.15 hadolint=2.14.0 codespell=2.4.1 markdownlint=0.46.0 shellcheck=0.11.0 yamllint=1.38.0
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@maheshrajus@tez-yetus@Aggarwal-Raghav@abstractdog