Skip to content

TEZ-4609: JDK-17: Fix ByteBuffer mark incompatibility in PipelinedSorter. - #397

Merged
ayushtkn merged 1 commit into
apache:masterfrom
ayushtkn:TEZ-4609
Mar 24, 2025
Merged

TEZ-4609: JDK-17: Fix ByteBuffer mark incompatibility in PipelinedSorter.#397
ayushtkn merged 1 commit into
apache:masterfrom
ayushtkn:TEZ-4609

Conversation

@ayushtkn

@ayushtknayushtkn commented Mar 24, 2025

Copy link
Copy Markdown
Member

The ByteBuffer.mark() method returns ByteBuffer itself in Java 8, but starting with JDK 9+, it returns a more specific subclass (like HeapByteBuffer or DirectByteBuffer), which is incompatible with method signatures expecting ByteBuffer.

When running the same code in JDK 17, this mismatch triggers a NoSuchMethodError due to binary incompatibility.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec26m 49sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall11m 16smaster passed
+1 💚compile0m 36smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚compile0m 33smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚checkstyle1m 2smaster passed
+1 💚javadoc0m 39smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 30smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+0 🆗spotbugs1m 19sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 17smaster passed
_ Patch Compile Tests _
+1 💚mvninstall0m 21sthe patch passed
+1 💚compile0m 22sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javac0m 22sthe patch passed
+1 💚compile0m 20sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚javac0m 20sthe patch passed
+1 💚checkstyle0m 15sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 18sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 17sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚findbugs0m 54sthe patch passed
_ Other Tests _
+1 💚unit5m 37stez-runtime-library in the patch passed.
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
52m 26s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-397/1/artifact/out/Dockerfile
GITHUB PR#397
JIRA IssueTEZ-4609
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux e83271e24bdb 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 3aa2aa1
Default JavaPrivate Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-397/1/testReport/
Max. process+thread count1090 (vs. ulimit of 5500)
modulesC: tez-runtime-library U: tez-runtime-library
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-397/1/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdog

Copy link
Copy Markdown
Contributor

is it about to compile on JDK8 and run on JDK17 scenario?
it fails on this line according to Jira (line 949)

 reserved.mark();

please confirm if the error is because:

  1. the compiler saw a version of ByteBuffer.mark() that returns a ByteBuffer.
  2. at runtime, the JVM is using a different version of ByteBuffer where mark() exists but returns something else (as you mentioned, HeapByteBuffer)

@ayushtkn

Copy link
Copy Markdown
MemberAuthor

yes, exactly

@abstractdogabstractdog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ayushtkn
ayushtkn merged commit c34ec16 into apache:masterMar 24, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@ayushtkn@tez-yetus@abstractdog