Skip to content

HBASE-29193: Allow ZstdByteBuffDecompressor to take direct ByteBuffer as input and heap ByteBuffer as output, or vice versa - #6806

Merged
ndimiduk merged 1 commit into
apache:masterfrom
HubSpot:HBASE-29193/ZstdByteBuffDecompressor-more-flexible
Apr 17, 2025
Merged

HBASE-29193: Allow ZstdByteBuffDecompressor to take direct ByteBuffer as input and heap ByteBuffer as output, or vice versa#6806
ndimiduk merged 1 commit into
apache:masterfrom
HubSpot:HBASE-29193/ZstdByteBuffDecompressor-more-flexible

Conversation

@charlesconnell

Copy link
Copy Markdown
Contributor

At present, ZstdByteBuffDecompressor is limited to decompressing from a direct ByteBuffer to another direct ByteBuffer, or from a heap ByteBuffer to another heap ByteBuffer. Following a change I contributed to the zstd-jni project, this limitation can be removed.

… as input and heap ByteBuffer as output, or vice versa
@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@charlesconnell

Copy link
Copy Markdown
ContributorAuthor

Test failures are unrelated

Comment threadpom.xml
<lz4.version>1.8.0</lz4.version>
<snappy.version>1.1.10.4</snappy.version>
<zstd-jni.version>1.5.5-2</zstd-jni.version>
<zstd-jni.version>1.5.7-2</zstd-jni.version>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit surprised that we expose these libraries to downstream applications.

@ndimiduk

Copy link
Copy Markdown
Member

I kicked off another build.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 33sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for branch
+1 💚mvninstall3m 14smaster passed
+1 💚compile7m 56smaster passed
+1 💚checkstyle1m 14smaster passed
+1 💚spotbugs7m 43smaster passed
+1 💚spotless0m 45sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for patch
+1 💚mvninstall3m 3sthe patch passed
+1 💚compile7m 55sthe patch passed
+1 💚javac7m 55sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle1m 11sthe patch passed
+1 💚xmllint0m 0sNo new issues.
+1 💚spotbugs8m 0sthe patch passed
+1 💚hadoopcheck11m 56sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚spotless0m 43spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 19sThe patch does not generate ASF License warnings.
62m 36s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6806/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6806
JIRA IssueHBASE-29193
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless xmllint
unameLinux 732de22ded75 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f8660bd
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count191 (vs. ulimit of 30000)
modulesC: hbase-compression/hbase-compression-zstd . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6806/2/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 32sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 18sMaven dependency ordering for branch
+1 💚mvninstall3m 48smaster passed
+1 💚compile2m 46smaster passed
+1 💚javadoc3m 4smaster passed
+1 💚shadedjars7m 28sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗mvndep0m 20sMaven dependency ordering for patch
+1 💚mvninstall3m 57sthe patch passed
+1 💚compile2m 48sthe patch passed
+1 💚javac2m 48sthe patch passed
+1 💚javadoc2m 49sthe patch passed
+1 💚shadedjars7m 15spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit327m 15sroot in the patch passed.
369m 50s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6806/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6806
JIRA IssueHBASE-29193
Optional Testsjavac javadoc unit compile shadedjars
unameLinux d7b790fe93c1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f8660bd
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6806/2/testReport/
Max. process+thread count8146 (vs. ulimit of 30000)
modulesC: hbase-compression/hbase-compression-zstd . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6806/2/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@ndimiduk
ndimiduk merged commit e56e14b into apache:masterApr 17, 2025
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 17, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 17, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 17, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 17, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit that referenced this pull request Apr 17, 2025
… as input and heap ByteBuffer as output, or vice versa (#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
@ndimiduk
ndimiduk deleted the HBASE-29193/ZstdByteBuffDecompressor-more-flexible branch April 17, 2025 15:09
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 22, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 22, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 22, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit that referenced this pull request Apr 23, 2025
… as input and heap ByteBuffer as output, or vice versa (#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 23, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit that referenced this pull request Apr 23, 2025
… as input and heap ByteBuffer as output, or vice versa (#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit that referenced this pull request Apr 23, 2025
… as input and heap ByteBuffer as output, or vice versa (#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
mokai87 pushed a commit to mokai87/hbase that referenced this pull request Aug 7, 2025
… as input and heap ByteBuffer as output, or vice versa (apache#6806)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@charlesconnell@Apache-HBase@ndimiduk