Skip to content

HBASE-27990. BucketCache causes ArithmeticException due to improper blockSize value checking - #5389

Merged
Apache9 merged 2 commits into
apache:masterfrom
teamconfx:HBASE-27990
Mar 28, 2024
Merged

HBASE-27990. BucketCache causes ArithmeticException due to improper blockSize value checking#5389
Apache9 merged 2 commits into
apache:masterfrom
teamconfx:HBASE-27990

Conversation

@teamconfx

@teamconfxteamconfx commented Sep 11, 2023

Copy link
Copy Markdown
Contributor

Description of PR

https://issues.apache.org/jira/browse/HBASE-27990
This PR adds a precheck for hbase.blockcache.minblocksize such that it must be positive.

How was this patch tested?

(1) set hbase.blockcache.minblocksize=0
(2) run org.apache.hadoop.hbase.io.hfile.TestCacheConfig#testBucketCacheConfigL1L2Setup
The test throws a IllegalArgumentException rather than a division by zero error.

@Apache-HBase

Apache-HBase commented Sep 11, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 35sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall2m 50smaster passed
+1 💚compile2m 24smaster passed
+1 💚checkstyle0m 38smaster passed
+1 💚spotless0m 44sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 29smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 33sthe patch passed
+1 💚compile2m 22sthe patch passed
+1 💚javac2m 22sthe patch passed
+1 💚checkstyle0m 36sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck9m 6sPatch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚spotless0m 41spatch has no errors when running spotless:check.
+1 💚spotbugs1m 37sthe patch passed
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
31m 37s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5389
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux d63d4f71ca8f 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 97d512b
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count79 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Sep 11, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 12sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 47smaster passed
+1 💚compile0m 48smaster passed
+1 💚shadedjars4m 38sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 27smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 36sthe patch passed
+1 💚compile0m 46sthe patch passed
+1 💚javac0m 46sthe patch passed
+1 💚shadedjars4m 37spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sthe patch passed
_ Other Tests _
-1 ❌unit220m 46shbase-server in the patch failed.
242m 38s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5389
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 10c2c318659b 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 97d512b
Default JavaEclipse Adoptium-11.0.17+8
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/testReport/
Max. process+thread count4823 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Sep 11, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 25sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 36smaster passed
+1 💚compile0m 36smaster passed
+1 💚shadedjars5m 3sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 23smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 20sthe patch passed
+1 💚compile0m 39sthe patch passed
+1 💚javac0m 39sthe patch passed
+1 💚shadedjars5m 1spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 22sthe patch passed
_ Other Tests _
+1 💚unit222m 32shbase-server in the patch passed.
244m 30s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5389
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 40986c7b59e0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 97d512b
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/testReport/
Max. process+thread count4458 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9
Apache9 reviewed Sep 23, 2023
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java Outdated
public BucketCache(String ioEngineName, long capacity, int blockSize, int[] bucketSizes,
int writerThreadNum, int writerQLen, String persistencePath, int ioErrorsTolerationDuration,
Configuration conf) throws IOException {
Preconditions.checkArgument(blockSize > 0, "BucketCache capacity can not be less than 0");

@Apache9Apache9Sep 23, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to include the incorrect blockSize value in the error message.

@teamconfxteamconfxMar 20, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@Apache-HBase

Apache-HBase commented Mar 20, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 29sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 17smaster passed
+1 💚compile0m 53smaster passed
+1 💚shadedjars5m 37sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 55sthe patch passed
+1 💚compile0m 53sthe patch passed
+1 💚javac0m 53sthe patch passed
+1 💚shadedjars5m 39spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sthe patch passed
_ Other Tests _
-1 ❌unit12m 6shbase-server in the patch failed.
34m 14s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#5389
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 6bda447d1d7d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0763a74
Default JavaEclipse Adoptium-17.0.10+7
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/testReport/
Max. process+thread count1828 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Mar 20, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec5m 3sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 38smaster passed
+1 💚compile3m 19smaster passed
+1 💚checkstyle0m 41smaster passed
+1 💚spotless0m 49sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 51smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 59sthe patch passed
+1 💚compile3m 13sthe patch passed
+1 💚javac3m 13sthe patch passed
-0 ⚠️checkstyle0m 44shbase-server: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck5m 46sPatch does not cause any errors with Hadoop 3.3.6.
-1 ❌spotless1m 6spatch has 23 errors when running spotless:check, run spotless:apply to fix.
+1 💚spotbugs2m 52sthe patch passed
_ Other Tests _
+1 💚asflicense0m 22sThe patch does not generate ASF License warnings.
44m 11s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5389
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux a741436ffe9f 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0763a74
Default JavaEclipse Adoptium-11.0.17+8
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count77 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Mar 21, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 45sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 38smaster passed
+1 💚compile1m 6smaster passed
+1 💚shadedjars5m 15sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 28smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 50sthe patch passed
+1 💚compile0m 49sthe patch passed
+1 💚javac0m 49sthe patch passed
+1 💚shadedjars5m 13spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sthe patch passed
_ Other Tests _
+1 💚unit221m 10shbase-server in the patch passed.
247m 26s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5389
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 515bd80d7a5e 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0763a74
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/testReport/
Max. process+thread count5453 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Mar 21, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec3m 55sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 56smaster passed
+1 💚compile0m 49smaster passed
+1 💚shadedjars5m 37sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 33smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 17sthe patch passed
+1 💚compile0m 48sthe patch passed
+1 💚javac0m 48sthe patch passed
+1 💚shadedjars5m 36spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 24sthe patch passed
_ Other Tests _
-1 ❌unit232m 47shbase-server in the patch failed.
261m 23s
SubsystemReport/Notes
DockerClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5389
Optional Testsjavac javadoc unit shadedjars compile
unameLinux aab8c6c5f434 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0763a74
Default JavaTemurin-1.8.0_352-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/testReport/
Max. process+thread count4368 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9
Apache9 merged commit 5a0c4de into apache:masterMar 28, 2024
Apache9 pushed a commit that referenced this pull request Mar 28, 2024
@teamconfx@Apache9
HBASE-27990BucketCache causes ArithmeticException due to improper bl…
81e4385
…ockSize value checking (#5389)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5a0c4de)
Apache9 pushed a commit that referenced this pull request Mar 28, 2024
@teamconfx@Apache9
HBASE-27990BucketCache causes ArithmeticException due to improper bl…
bb094dc
…ockSize value checking (#5389)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5a0c4de)
Apache9 pushed a commit that referenced this pull request Mar 28, 2024
@teamconfx@Apache9
HBASE-27990BucketCache causes ArithmeticException due to improper bl…
c057037
…ockSize value checking (#5389)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5a0c4de)
Apache9 pushed a commit that referenced this pull request Mar 28, 2024
@teamconfx@Apache9
HBASE-27990BucketCache causes ArithmeticException due to improper bl…
9e1c1cb
…ockSize value checking (#5389)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5a0c4de)
Apache9 pushed a commit that referenced this pull request Mar 28, 2024
@teamconfx@Apache9
HBASE-27990BucketCache causes ArithmeticException due to improper bl…
e2d7ec9
…ockSize value checking (#5389)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5a0c4de)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Apache9Apache9Apache9 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@teamconfx@Apache-HBase@Apache9