Skip to content

HBASE-29857 better handling NPE in BucketCache - #7685

Merged
wchevreuil merged 5 commits into
apache:masterfrom
teamconfx:fix-hbase-29857-master
Feb 3, 2026
Merged

HBASE-29857 better handling NPE in BucketCache #7685
wchevreuil merged 5 commits into
apache:masterfrom
teamconfx:fix-hbase-29857-master

Conversation

@teamconfx

@teamconfxteamconfx commented Jan 28, 2026

Copy link
Copy Markdown
Contributor

This PR is for JIRA issue: https://issues.apache.org/jira/browse/HBASE-29857

The original JIRA issue is found in HBase v2.6.3, and fixed by HBASE-28839.
I tested with the same workload that I see the bug from v2.6.3 and verify the fix works in both master and branch-2.6.

However the current code still not handle NPE when persistence file is empty or corrupted explicitly.
This PR proposes a new checker to better catch the null from BucketCacheProtos.BucketCacheEntry.parseDelimitedFrom(in);.

@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
vaijosh
vaijosh suggested changes Jan 29, 2026
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
@Apache-HBase

This comment has been minimized.

Sign in to view
@vaijosh

vaijosh commented Jan 29, 2026

Copy link
Copy Markdown
Contributor

Thanks @teamconfx
Changes LGTM.

@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

Apache-HBase commented Jan 29, 2026

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 12sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall3m 21smaster passed
+1 💚compile3m 32smaster passed
+1 💚checkstyle1m 4smaster passed
+1 💚spotbugs1m 52smaster passed
+1 💚spotless0m 49sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall3m 33sthe patch passed
+1 💚compile2m 48sthe patch passed
+1 💚javac2m 48sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 48sthe patch passed
+1 💚spotbugs1m 23sthe patch passed
+1 💚hadoopcheck8m 53sPatch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚spotless0m 36spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 10sThe patch does not generate ASF License warnings.
34m 45s
SubsystemReport/Notes
DockerClientAPI=1.53 ServerAPI=1.53 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#7685
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 09ad049fb39f 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8423bce
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count84 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/3/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil
wchevreuil reviewed Jan 29, 2026
View reviewed changes

@wchevreuilwchevreuil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a few minor comments. Apart from those, can we add UTs for these conditions?

Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java Outdated
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
@Apache-HBase

Apache-HBase commented Jan 29, 2026

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 12sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 20smaster passed
+1 💚compile0m 46smaster passed
+1 💚javadoc0m 22smaster passed
+1 💚shadedjars4m 26sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 17sthe patch passed
+1 💚compile0m 46sthe patch passed
+1 💚javac0m 46sthe patch passed
+1 💚javadoc0m 21sthe patch passed
+1 💚shadedjars4m 26spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit215m 24s/patch-unit-hbase-server.txthbase-server in the patch failed.
235m 17s
SubsystemReport/Notes
DockerClientAPI=1.53 ServerAPI=1.53 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#7685
Optional Testsjavac javadoc unit compile shadedjars
unameLinux d1f2b9254e00 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8423bce
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/3/testReport/
Max. process+thread count5911 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/3/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jan 29, 2026

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec1m 24sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall2m 35smaster passed
+1 💚compile2m 35smaster passed
+1 💚checkstyle0m 47smaster passed
+1 💚spotbugs1m 15smaster passed
+1 💚spotless0m 40sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 13sthe patch passed
+1 💚compile2m 38sthe patch passed
+1 💚javac2m 38sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 45sthe patch passed
+1 💚spotbugs1m 19sthe patch passed
+1 💚hadoopcheck8m 29sPatch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚spotless0m 33spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 9sThe patch does not generate ASF License warnings.
30m 31s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#7685
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 29577dda8773 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f64f0ed
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/4/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jan 30, 2026

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 11sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 40smaster passed
+1 💚compile0m 46smaster passed
+1 💚javadoc0m 24smaster passed
+1 💚shadedjars4m 26sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 18sthe patch passed
+1 💚compile0m 46sthe patch passed
+1 💚javac0m 46sthe patch passed
+1 💚javadoc0m 22sthe patch passed
+1 💚shadedjars4m 23spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit216m 35s/patch-unit-hbase-server.txthbase-server in the patch failed.
236m 44s
SubsystemReport/Notes
DockerClientAPI=1.53 ServerAPI=1.53 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#7685
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 6ec5fc974b1a 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f64f0ed
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/4/testReport/
Max. process+thread count6217 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/4/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@teamconfx

teamconfx commented Jan 30, 2026

Copy link
Copy Markdown
ContributorAuthor

@wchevreuil I added a UT TestBucketCacheEmptyPersistence that verifies that BucketCache recovers gracefully when the persistence file contains only magic bytes without actual cache data. It tests both persistence formats (PBUF for
non-chunked and V2UF for chunked) by creating truncated files, then verifying the cache initializes successfully, has an empty backing map, and remains functional for caching operations.

@Apache-HBase

Apache-HBase commented Jan 30, 2026

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec1m 21sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 1scodespell was not available.
+0 🆗detsecrets0m 1sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall2m 48smaster passed
+1 💚compile2m 32smaster passed
+1 💚checkstyle0m 45smaster passed
+1 💚spotbugs1m 14smaster passed
+1 💚spotless0m 38sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 11sthe patch passed
+1 💚compile2m 28sthe patch passed
+1 💚javac2m 28sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 44sthe patch passed
+1 💚spotbugs1m 17sthe patch passed
+1 💚hadoopcheck8m 22sPatch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚spotless0m 32spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 8sThe patch does not generate ASF License warnings.
30m 8s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#7685
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux c83c2b0900a9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0e82df2
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count84 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/5/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jan 30, 2026

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 11sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 15smaster passed
+1 💚compile0m 46smaster passed
+1 💚javadoc0m 21smaster passed
+1 💚shadedjars4m 19sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 16sthe patch passed
+1 💚compile0m 45sthe patch passed
+1 💚javac0m 45sthe patch passed
+1 💚javadoc0m 21sthe patch passed
+1 💚shadedjars4m 20spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit211m 12s/patch-unit-hbase-server.txthbase-server in the patch failed.
230m 36s
SubsystemReport/Notes
DockerClientAPI=1.53 ServerAPI=1.53 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/5/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#7685
Optional Testsjavac javadoc unit compile shadedjars
unameLinux b36d4ce3efcb 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0e82df2
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/5/testReport/
Max. process+thread count5666 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7685/5/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@wchevreuil
wchevreuil merged commit 9e8bdf4 into apache:masterFeb 3, 2026
1 check failed
wchevreuil pushed a commit that referenced this pull request Feb 3, 2026
@teamconfx@wchevreuil
HBASE-29857better handling NPE in BucketCache (#7685)
f569b0d
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Singed-off-by: Peng Lu <lupeng@apache.org>
Reviewed-by: Vaibhav Joshi <vjoshi@cloudera.com>
wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request Feb 3, 2026
@teamconfx@wchevreuil
HBASE-29857 better handling NPE in BucketCache (apache#7685)
011fd32
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Singed-off-by: Peng Lu <lupeng@apache.org>
Reviewed-by: Vaibhav Joshi <vjoshi@cloudera.com>
wchevreuil pushed a commit that referenced this pull request Feb 4, 2026
@teamconfx@wchevreuil
HBASE-29857better handling NPE in BucketCache (#7685)
02a63ea
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Singed-off-by: Peng Lu <lupeng@apache.org>
Reviewed-by: Vaibhav Joshi <vjoshi@cloudera.com>
wchevreuil pushed a commit that referenced this pull request Feb 4, 2026
@teamconfx@wchevreuil
HBASE-29857better handling NPE in BucketCache (#7685)
f60cb68
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Singed-off-by: Peng Lu <lupeng@apache.org>
Reviewed-by: Vaibhav Joshi <vjoshi@cloudera.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wchevreuilwchevreuilwchevreuil approved these changes
@guluo2016guluo2016guluo2016 approved these changes
+1 more reviewer
@vaijoshvaijoshvaijosh approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@teamconfx@Apache-HBase@vaijosh@wchevreuil@guluo2016