Skip to content

HBASE-26988 dynamic configuration of loadbalance.bytable - #4384

Merged
apurtell merged 2 commits into
apache:masterfrom
d-c-manning:HBASE-26988
Apr 29, 2022
Merged

HBASE-26988 dynamic configuration of loadbalance.bytable#4384
apurtell merged 2 commits into
apache:masterfrom
d-c-manning:HBASE-26988

Conversation

@d-c-manning

@d-c-manningd-c-manning commented Apr 29, 2022

Copy link
Copy Markdown
Contributor

HBASE-26988

Some tests call conf.unset(HConstants.HBASE_MASTER_LOADBALANCE_BYTABLE); but that only works if dynamic reconfiguration will reset the value. Without this fix (or test fixes), the tests only pass if some ordering constraints are guaranteed.

@Apache-HBase

Apache-HBase commented Apr 29, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 9sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall2m 24smaster passed
+1 💚compile0m 22smaster passed
+1 💚checkstyle0m 12smaster passed
+1 💚spotbugs0m 25smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 11sthe patch passed
+1 💚compile0m 21sthe patch passed
+1 💚javac0m 21sthe patch passed
+1 💚checkstyle0m 10sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck11m 37sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs0m 27sthe patch passed
_ Other Tests _
+1 💚asflicense0m 8sThe patch does not generate ASF License warnings.
24m 13s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4384
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 60c7663ef5d7 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f4eb907
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count64 (vs. ulimit of 30000)
modulesC: hbase-balancer U: hbase-balancer
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 29, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 19sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 37smaster passed
+1 💚compile0m 18smaster passed
+1 💚shadedjars3m 40sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 14smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 35sthe patch passed
+1 💚compile0m 15sthe patch passed
+1 💚javac0m 15sthe patch passed
+1 💚shadedjars3m 50spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 13sthe patch passed
_ Other Tests _
+1 💚unit11m 28shbase-balancer in the patch passed.
27m 16s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4384
Optional Testsjavac javadoc unit shadedjars compile
unameLinux b97bde627e8f 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f4eb907
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/testReport/
Max. process+thread count193 (vs. ulimit of 30000)
modulesC: hbase-balancer U: hbase-balancer
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 29, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 54sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 30smaster passed
+1 💚compile0m 21smaster passed
+1 💚shadedjars4m 53sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 13smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 46sthe patch passed
+1 💚compile0m 14sthe patch passed
+1 💚javac0m 14sthe patch passed
+1 💚shadedjars4m 39spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 11sthe patch passed
_ Other Tests _
+1 💚unit11m 44shbase-balancer in the patch passed.
31m 21s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4384
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 4d00d1238faa 5.4.0-1068-aws #72~18.04.1-Ubuntu SMP Thu Mar 3 08:49:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f4eb907
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/testReport/
Max. process+thread count165 (vs. ulimit of 30000)
modulesC: hbase-balancer U: hbase-balancer
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 29, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 14sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall2m 10smaster passed
+1 💚compile0m 21smaster passed
+1 💚checkstyle0m 10smaster passed
+1 💚spotbugs0m 22smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 13sthe patch passed
+1 💚compile0m 21sthe patch passed
+1 💚javac0m 21sthe patch passed
+1 💚checkstyle0m 9sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck11m 28sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs0m 28sthe patch passed
_ Other Tests _
+1 💚asflicense0m 9sThe patch does not generate ASF License warnings.
23m 47s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4384
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 5f7e93bcf1e2 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f4eb907
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count64 (vs. ulimit of 30000)
modulesC: hbase-balancer U: hbase-balancer
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 29, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 14sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 42smaster passed
+1 💚compile0m 15smaster passed
+1 💚shadedjars3m 40sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 14smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 41sthe patch passed
+1 💚compile0m 15sthe patch passed
+1 💚javac0m 15sthe patch passed
+1 💚shadedjars3m 42spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 12sthe patch passed
_ Other Tests _
+1 💚unit11m 28shbase-balancer in the patch passed.
27m 10s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4384
Optional Testsjavac javadoc unit shadedjars compile
unameLinux b2b38c852b13 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f4eb907
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/testReport/
Max. process+thread count190 (vs. ulimit of 30000)
modulesC: hbase-balancer U: hbase-balancer
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 29, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 47sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 45smaster passed
+1 💚compile0m 15smaster passed
+1 💚shadedjars4m 43sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 11smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 18sthe patch passed
+1 💚compile0m 11sthe patch passed
+1 💚javac0m 11sthe patch passed
+1 💚shadedjars3m 53spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 9sthe patch passed
_ Other Tests _
+1 💚unit11m 6shbase-balancer in the patch passed.
27m 7s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4384
Optional Testsjavac javadoc unit shadedjars compile
unameLinux f212f8bb9ce0 5.4.0-1068-aws #72~18.04.1-Ubuntu SMP Thu Mar 3 08:49:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / f4eb907
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/testReport/
Max. process+thread count163 (vs. ulimit of 30000)
modulesC: hbase-balancer U: hbase-balancer
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@apurtell
apurtell merged commit 4a33ed1 into apache:masterApr 29, 2022
@d-c-manning
d-c-manning deleted the HBASE-26988 branch April 29, 2022 21:43
@d-c-manningd-c-manning mentioned this pull request Apr 29, 2022
Merged
asfgit pushed a commit that referenced this pull request Apr 29, 2022
@d-c-manning@apurtell
HBASE-26988dynamic configuration of loadbalance.bytable (#4384)
adc9d07
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
asfgit pushed a commit that referenced this pull request Apr 29, 2022
@d-c-manning@apurtell
HBASE-26988dynamic configuration of loadbalance.bytable (#4384)
afb8afd
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@apurtellapurtellapurtell approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@d-c-manning@Apache-HBase@apurtell