Skip to content

HBASE-26681 Introduce a little RAMBuffer for bucketcache to reduce gc and improve throughput - #4043

Merged
Apache9 merged 3 commits into
apache:branch-1from
YutSean:HBASE-26681
Aug 2, 2022
Merged

HBASE-26681 Introduce a little RAMBuffer for bucketcache to reduce gc and improve throughput#4043
Apache9 merged 3 commits into
apache:branch-1from
YutSean:HBASE-26681

Conversation

@YutSean

Copy link
Copy Markdown
Contributor

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec7m 59sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-0 ⚠️test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚mvninstall14m 28sbranch-1 passed
+1 💚compile0m 46sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚compile0m 51sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚checkstyle1m 57sbranch-1 passed
+1 💚shadedjars3m 38sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 49sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 43sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗spotbugs3m 20sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs3m 17sbranch-1 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 19sthe patch passed
+1 💚compile0m 47sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javac0m 47sthe patch passed
+1 💚compile0m 52sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚javac0m 52sthe patch passed
-1 ❌checkstyle1m 44shbase-server: The patch generated 1 new + 16 unchanged - 0 fixed = 17 total (was 16)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚shadedjars3m 20spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck8m 2sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚javadoc0m 33sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 43sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
-1 ❌findbugs3m 16shbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
_ Other Tests _
-1 ❌unit157m 15shbase-server in the patch failed.
+1 💚asflicense0m 34sThe patch does not generate ASF License warnings.
217m 57s
ReasonTests
FindBugsmodule:hbase-server
Integral division result cast to double or float in new org.apache.hadoop.hbase.io.hfile.bucket.BufferedBucketCache(String, long, int, int[], int, int, String, int, Configuration) At BufferedBucketCache.java:double or float in new org.apache.hadoop.hbase.io.hfile.bucket.BufferedBucketCache(String, long, int, int[], int, int, String, int, Configuration) At BufferedBucketCache.java:[line 68]
Failed junit testshadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint
hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/Dockerfile
GITHUB PR#4043
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 3c8b22b2c05c 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4043/out/precommit/personality/provided.sh
git revisionbranch-1 / a3e7d36
Default JavaAzul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/diff-checkstyle-hbase-server.txt
findbugshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/new-findbugs-hbase-server.html
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/testReport/
Max. process+thread count4708 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/console
versionsgit=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 6sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-0 ⚠️test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚mvninstall12m 48sbranch-1 passed
+1 💚compile0m 45sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚compile0m 51sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚checkstyle1m 55sbranch-1 passed
+1 💚shadedjars3m 40sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 51sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 44sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗spotbugs3m 22sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs3m 18sbranch-1 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 20sthe patch passed
+1 💚compile0m 45sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javac0m 45sthe patch passed
+1 💚compile0m 51sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚javac0m 51sthe patch passed
-1 ❌checkstyle1m 44shbase-server: The patch generated 1 new + 16 unchanged - 0 fixed = 17 total (was 16)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚shadedjars3m 23spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck6m 58sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚javadoc0m 32sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 43sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚findbugs3m 11sthe patch passed
_ Other Tests _
-1 ❌unit164m 13shbase-server in the patch failed.
+1 💚asflicense0m 38sThe patch does not generate ASF License warnings.
215m 12s
ReasonTests
Failed junit testshadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint
hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
hadoop.hbase.TestPartialResultsFromClientSide
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/artifact/out/Dockerfile
GITHUB PR#4043
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 2e27082b906b 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4043/out/precommit/personality/provided.sh
git revisionbranch-1 / a3e7d36
Default JavaAzul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/artifact/out/diff-checkstyle-hbase-server.txt
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/testReport/
Max. process+thread count4509 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/console
versionsgit=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 32sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-0 ⚠️test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚mvninstall13m 58sbranch-1 passed
+1 💚compile0m 59sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚compile1m 3sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚checkstyle2m 23sbranch-1 passed
+1 💚shadedjars4m 16sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc1m 0sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 57sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗spotbugs3m 49sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs3m 44sbranch-1 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 54sthe patch passed
+1 💚compile0m 52sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javac0m 52sthe patch passed
+1 💚compile0m 58sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚javac0m 58sthe patch passed
+1 💚checkstyle2m 0sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚shadedjars4m 19spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck8m 48sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚javadoc0m 38sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 51sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚findbugs3m 59sthe patch passed
_ Other Tests _
-1 ❌unit189m 59shbase-server in the patch failed.
+1 💚asflicense0m 37sThe patch does not generate ASF License warnings.
250m 11s
ReasonTests
Failed junit testshadoop.hbase.replication.TestReplicationSource
hadoop.hbase.namequeues.TestNamedQueueRecorder
hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
hadoop.hbase.snapshot.TestExportSnapshot
hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/3/artifact/out/Dockerfile
GITHUB PR#4043
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 93328c06d304 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4043/out/precommit/personality/provided.sh
git revisionbranch-1 / a3e7d36
Default JavaAzul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/3/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/3/testReport/
Max. process+thread count4578 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/3/console
versionsgit=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@YutSean

Copy link
Copy Markdown
ContributorAuthor

The failed UTs passed locally though.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 54sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-0 ⚠️test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚mvninstall10m 16sbranch-1 passed
+1 💚compile0m 43sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚compile0m 50sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚checkstyle1m 46sbranch-1 passed
+1 💚shadedjars3m 21sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 48sbranch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 42sbranch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗spotbugs3m 9sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs3m 6sbranch-1 passed
_ Patch Compile Tests _
+1 💚mvninstall2m 10sthe patch passed
+1 💚compile0m 46sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javac0m 46sthe patch passed
+1 💚compile0m 49sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚javac0m 49sthe patch passed
+1 💚checkstyle1m 31sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚shadedjars3m 2spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck5m 17sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚javadoc0m 32sthe patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚javadoc0m 41sthe patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚findbugs2m 59sthe patch passed
_ Other Tests _
+1 💚unit130m 27shbase-server in the patch passed.
+1 💚asflicense0m 41sThe patch does not generate ASF License warnings.
179m 0s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/Dockerfile
GITHUB PR#4043
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 59e24a57f1fc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4043/out/precommit/personality/provided.sh
git revisionbranch-1 / a2406f3
Default JavaAzul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/testReport/
Max. process+thread count4549 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/console
versionsgit=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9
Apache9 merged commit 0a0a474 into apache:branch-1Aug 2, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@YutSean@Apache-HBase@Apache9