Skip to content

HBASE-29658: Add RISC-V Vector (RVV) optimizations - #7388

Open
gpg12 wants to merge 4 commits into
apache:masterfrom
gpg12:rvv-optimization_master
Open

HBASE-29658: Add RISC-V Vector (RVV) optimizations#7388
gpg12 wants to merge 4 commits into
apache:masterfrom
gpg12:rvv-optimization_master

Conversation

@gpg12

@gpg12gpg12 commented Oct 15, 2025

Copy link
Copy Markdown

Description

This PR introduces RISC-V Vector (RVV) optimizations to Apache HBase to significantly improve performance on RISC-V platforms. The optimizations target three critical areas:

  1. LZ4 Compression/Decompression - Vectorized hash computation and dictionary operations
  2. BloomFilter Operations - Parallel bit manipulation and batch processing
  3. Scan Query Operations - Vectorized byte comparisons and prefix matching

Key Features

  • JNI integration with native C implementations using RVV intrinsics
  • Conditional compilation ensuring compatibility with non-RISC-V platforms
  • Runtime detection with graceful fallback mechanisms
  • Comprehensive test coverage and performance benchmarks
  • Backward compatibility maintained for existing deployments

Core Design Principles

  1. Conditional Compilation: Use #if defined(__riscv) && defined(__riscv_vector) to ensure RVV code only compiles on supported platforms
  2. Runtime Detection: Implement graceful fallback when RVV is unavailable
  3. Backward Compatibility: No impact on existing deployments or non-RISC-V platforms
  4. Performance Monitoring: Built-in metrics collection and performance analysis

JNI Integration Strategy

The implementation uses JNI to call native C functions that utilize RISC-V vector instructions for:

  • Vectorized memory operations (memcmp, memcpy)
  • Parallel hash computation for BloomFilter operations
  • Batch processing for improved throughput

Closes #HBASE-29658

@Apache-HBase

Apache-HBase commented Oct 15, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 33sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+0 🆗mvndep0m 17sMaven dependency ordering for branch
+1 💚mvninstall3m 37smaster passed
+1 💚compile4m 28smaster passed
+1 💚checkstyle1m 0smaster passed
+1 💚spotbugs2m 27smaster passed
+1 💚spotless0m 53sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
-1 ❌mvninstall0m 38s/patch-mvninstall-root.txtroot in the patch failed.
-1 ❌compile0m 15s/patch-compile-hbase-common.txthbase-common in the patch failed.
-1 ❌compile0m 39s/patch-compile-hbase-server.txthbase-server in the patch failed.
-1 ❌compile0m 13s/patch-compile-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
-1 ❌cc0m 15s/patch-compile-hbase-common.txthbase-common in the patch failed.
-1 ❌cc0m 39s/patch-compile-hbase-server.txthbase-server in the patch failed.
-1 ❌cc0m 13s/patch-compile-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
-1 ❌golang0m 15s/patch-compile-hbase-common.txthbase-common in the patch failed.
-1 ❌golang0m 39s/patch-compile-hbase-server.txthbase-server in the patch failed.
-1 ❌golang0m 13s/patch-compile-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
-0 ⚠️javac0m 15s/patch-compile-hbase-common.txthbase-common in the patch failed.
-0 ⚠️javac0m 39s/patch-compile-hbase-server.txthbase-server in the patch failed.
-0 ⚠️javac0m 13s/patch-compile-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
-0 ⚠️blanks0m 0s/blanks-eol.txtThe patch has 48 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️checkstyle0m 10s/buildtool-patch-checkstyle-hbase-common.txtThe patch fails to run checkstyle in hbase-common
-0 ⚠️checkstyle0m 8s/buildtool-patch-checkstyle-hbase-server.txtThe patch fails to run checkstyle in hbase-server
-0 ⚠️checkstyle0m 5s/buildtool-patch-checkstyle-hbase-compression_hbase-compression-lz4.txtThe patch fails to run checkstyle in hbase-compression-lz4
-1 ❌spotbugs0m 12s/patch-spotbugs-hbase-common.txthbase-common in the patch failed.
-1 ❌spotbugs0m 30s/patch-spotbugs-hbase-server.txthbase-server in the patch failed.
-1 ❌spotbugs0m 12s/patch-spotbugs-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
-1 ❌hadoopcheck0m 43sThe patch causes 212 errors with Hadoop v3.3.6.
-1 ❌hadoopcheck1m 26sThe patch causes 212 errors with Hadoop v3.4.1.
-1 ❌spotless0m 7spatch has 56 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
-1 ❌asflicense0m 21s/results-asflicense.txtThe patch generated 14 ASF License warnings.
20m 16s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#7388
JIRA IssueHBASE-29658
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless cc golang
unameLinux 25242804c1c1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 9396d72
Default JavaEclipse Adoptium-17.0.11+9
hadoopcheckhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-javac-3.3.6.txt
hadoopcheckhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-javac-3.4.1.txt
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count84 (vs. ulimit of 30000)
modulesC: hbase-common hbase-server hbase-compression/hbase-compression-lz4 U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Oct 15, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 56sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 21sMaven dependency ordering for branch
+1 💚mvninstall5m 11smaster passed
+1 💚compile2m 37smaster passed
+1 💚javadoc1m 25smaster passed
+1 💚shadedjars9m 5sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗mvndep0m 15sMaven dependency ordering for patch
-1 ❌mvninstall1m 9s/patch-mvninstall-root.txtroot in the patch failed.
-1 ❌compile0m 25s/patch-compile-hbase-common.txthbase-common in the patch failed.
-1 ❌compile0m 50s/patch-compile-hbase-server.txthbase-server in the patch failed.
-1 ❌compile0m 25s/patch-compile-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
-0 ⚠️javac0m 25s/patch-compile-hbase-common.txthbase-common in the patch failed.
-0 ⚠️javac0m 50s/patch-compile-hbase-server.txthbase-server in the patch failed.
-0 ⚠️javac0m 25s/patch-compile-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
-0 ⚠️javadoc0m 25s/patch-javadoc-hbase-common.txthbase-common in the patch failed.
-0 ⚠️javadoc0m 29s/patch-javadoc-hbase-server.txthbase-server in the patch failed.
-0 ⚠️javadoc0m 23s/patch-javadoc-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
-1 ❌shadedjars1m 30spatch has 212 errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit0m 22s/patch-unit-hbase-common.txthbase-common in the patch failed.
-1 ❌unit0m 44s/patch-unit-hbase-server.txthbase-server in the patch failed.
-1 ❌unit0m 22s/patch-unit-hbase-compression_hbase-compression-lz4.txthbase-compression-lz4 in the patch failed.
28m 42s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#7388
JIRA IssueHBASE-29658
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 0a58c2361429 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 9396d72
Default JavaEclipse Adoptium-17.0.11+9
shadedjarshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-jdk17-hadoop3-check/output/patch-shadedjars.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/testReport/
Max. process+thread count81 (vs. ulimit of 30000)
modulesC: hbase-common hbase-server hbase-compression/hbase-compression-lz4 U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@gpg12gpg12 closed this Oct 15, 2025
@gpg12gpg12 reopened this Oct 21, 2025
@Apache-HBase

Apache-HBase commented Oct 21, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 15sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+0 🆗codespell0m 1scodespell was not available.
+0 🆗detsecrets0m 1sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+0 🆗mvndep0m 21sMaven dependency ordering for branch
+1 💚mvninstall5m 38smaster passed
+1 💚compile15m 28smaster passed
+1 💚checkstyle2m 58smaster passed
+1 💚spotbugs17m 17smaster passed
+1 💚spotless1m 14sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 🆗mvndep0m 13sMaven dependency ordering for patch
-1 ❌mvninstall1m 1s/patch-mvninstall-root.txtroot in the patch failed.
-1 ❌compile1m 6s/patch-compile-root.txtroot in the patch failed.
-1 ❌cc1m 6s/patch-compile-root.txtroot in the patch failed.
-1 ❌golang1m 6s/patch-compile-root.txtroot in the patch failed.
-0 ⚠️javac1m 6s/patch-compile-root.txtroot in the patch failed.
-0 ⚠️blanks0m 0s/blanks-eol.txtThe patch has 56 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️checkstyle0m 17s/buildtool-patch-checkstyle-root.txtThe patch fails to run checkstyle in root
-1 ❌spotbugs0m 16s/patch-spotbugs-hbase-common.txthbase-common in the patch failed.
-1 ❌spotbugs0m 56s/patch-spotbugs-hbase-server.txthbase-server in the patch failed.
-1 ❌spotbugs3m 33s/patch-spotbugs-root.txtroot in the patch failed.
-1 ❌hadoopcheck1m 10sThe patch causes 16 errors with Hadoop v3.3.6.
-1 ❌hadoopcheck2m 16sThe patch causes 16 errors with Hadoop v3.4.1.
-1 ❌spotless0m 10spatch has 63 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
-1 ❌asflicense0m 45s/results-asflicense.txtThe patch generated 14 ASF License warnings.
56m 44s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#7388
JIRA IssueHBASE-29658
Optional Testsdupname asflicense codespell detsecrets spotless javac spotbugs checkstyle compile hadoopcheck hbaseanti cc golang
unameLinux 77e8e15e3d97 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e8500d4
Default JavaEclipse Adoptium-17.0.11+9
hadoopcheckhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-javac-3.3.6.txt
hadoopcheckhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-javac-3.4.1.txt
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count163 (vs. ulimit of 30000)
modulesC: hbase-common hbase-server hbase-compression/hbase-compression-lz4 . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Oct 21, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 15sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 26sMaven dependency ordering for branch
+1 💚mvninstall5m 58smaster passed
+1 💚compile4m 27smaster passed
+1 💚javadoc5m 23smaster passed
+1 💚shadedjars10m 6sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗mvndep0m 17sMaven dependency ordering for patch
-1 ❌mvninstall1m 5s/patch-mvninstall-root.txtroot in the patch failed.
-1 ❌compile0m 55s/patch-compile-root.txtroot in the patch failed.
-0 ⚠️javac0m 55s/patch-compile-root.txtroot in the patch failed.
-0 ⚠️javadoc0m 13s/patch-javadoc-hbase-common.txthbase-common in the patch failed.
-0 ⚠️javadoc0m 40s/results-javadoc-javadoc-hbase-server.txthbase-server generated 3 new + 63 unchanged - 0 fixed = 66 total (was 63)
-0 ⚠️javadoc0m 16s/patch-javadoc-root.txtroot in the patch failed.
-1 ❌shadedjars0m 9spatch has 7 errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit0m 53s/patch-unit-root.txtroot in the patch failed.
32m 54s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#7388
JIRA IssueHBASE-29658
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 8497561e89d5 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e8500d4
Default JavaEclipse Adoptium-17.0.11+9
shadedjarshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-jdk17-hadoop3-check/output/patch-shadedjars.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/testReport/
Max. process+thread count148 (vs. ulimit of 30000)
modulesC: hbase-common hbase-server hbase-compression/hbase-compression-lz4 . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@gpg12
gpg12 marked this pull request as draft October 22, 2025 12:03
@gpg12
gpg12 marked this pull request as ready for review October 22, 2025 12:04
@Apache9

Apache9 commented Nov 4, 2025

Copy link
Copy Markdown
Contributor

The PR is really big...

Is it possible to contribute it with several smaller pieces so it is easier for us to review it?

Thanks.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@gpg12@Apache-HBase@Apache9