Skip to content

HBASE-25775 Use a special balancer to deal with maintenance mode - #3161

Merged
Apache9 merged 1 commit into
apache:masterfrom
Apache9:HBASE-25775
Apr 16, 2021
Merged

HBASE-25775 Use a special balancer to deal with maintenance mode#3161
Apache9 merged 1 commit into
apache:masterfrom
Apache9:HBASE-25775

Conversation

@Apache9

@Apache9Apache9 commented Apr 15, 2021

Copy link
Copy Markdown
Contributor

No description provided.

@Apache9Apache9 self-assigned this Apr 15, 2021
@Apache9
Apache9 requested a review from saintstackApril 15, 2021 03:55
@Apache-HBase

Apache-HBase commented Apr 15, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 9sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 19smaster passed
+1 💚compile3m 21smaster passed
+1 💚checkstyle1m 14smaster passed
+1 💚spotbugs2m 17smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 59sthe patch passed
+1 💚compile3m 21sthe patch passed
+1 💚javac3m 21sthe patch passed
+1 💚checkstyle1m 13shbase-server: The patch generated 0 new + 145 unchanged - 4 fixed = 145 total (was 149)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 43sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 24sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
51m 30s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3161
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux b93f4431a6f7 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c5b0989
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9Apache9 mentioned this pull request Apr 15, 2021
Merged
@Apache-HBase

Apache-HBase commented Apr 15, 2021

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 30sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 35smaster passed
+1 💚compile1m 13smaster passed
+1 💚shadedjars7m 48sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 43smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 12sthe patch passed
+1 💚compile1m 11sthe patch passed
+1 💚javac1m 11sthe patch passed
+1 💚shadedjars7m 46spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 39sthe patch passed
_ Other Tests _
-1 ❌unit140m 17shbase-server in the patch failed.
171m 0s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3161
Optional Testsjavac javadoc unit shadedjars compile
unameLinux db8b5a6d013f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c5b0989
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/testReport/
Max. process+thread count3840 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 15, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 36sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 19smaster passed
+1 💚compile1m 1smaster passed
+1 💚shadedjars8m 24sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 0sthe patch passed
+1 💚compile1m 4sthe patch passed
+1 💚javac1m 4sthe patch passed
+1 💚shadedjars8m 26spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sthe patch passed
_ Other Tests _
+1 💚unit213m 38shbase-server in the patch passed.
246m 40s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3161
Optional Testsjavac javadoc unit shadedjars compile
unameLinux f2e4be6a1474 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c5b0989
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/testReport/
Max. process+thread count3539 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 15, 2021

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 48sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 30smaster passed
+1 💚compile1m 6smaster passed
+1 💚shadedjars8m 30sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 39smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 12sthe patch passed
+1 💚compile1m 6sthe patch passed
+1 💚javac1m 6sthe patch passed
+1 💚shadedjars8m 30spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 38sthe patch passed
_ Other Tests _
-1 ❌unit13m 46shbase-server in the patch failed.
45m 48s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3161
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 075dab014d63 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c5b0989
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/2/testReport/
Max. process+thread count674 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 15, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 56smaster passed
+1 💚compile3m 12smaster passed
+1 💚checkstyle1m 7smaster passed
+1 💚spotbugs2m 9smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 37sthe patch passed
+1 💚compile3m 7sthe patch passed
+1 💚javac3m 7sthe patch passed
+1 💚checkstyle1m 7shbase-server: The patch generated 0 new + 145 unchanged - 4 fixed = 145 total (was 149)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck17m 56sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 29sthe patch passed
_ Other Tests _
+1 💚asflicense0m 16sThe patch does not generate ASF License warnings.
48m 5s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3161
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux fb83a8948b98 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c5b0989
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count96 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Apr 15, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 41sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 32smaster passed
+1 💚compile3m 28smaster passed
+1 💚checkstyle1m 17smaster passed
+1 💚spotbugs2m 13smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 33sthe patch passed
+1 💚compile3m 29sthe patch passed
+1 💚javac3m 29sthe patch passed
+1 💚checkstyle1m 16shbase-server: The patch generated 0 new + 145 unchanged - 4 fixed = 145 total (was 149)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck24m 9sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 29sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
58m 37s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3161
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 43f7bd0df3ff 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c5b0989
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/3/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil
wchevreuil approved these changes Apr 15, 2021
View reviewed changes

@wchevreuilwchevreuil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, +1.

As an additional thought, should we consider disable balancing altogether during maintenance mode? After all, in this mode cluster capability will already be limited and is supposed to last for short time, requiring a restart to move it out of safe mode.

@Apache-HBase

Apache-HBase commented Apr 15, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 30sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 29smaster passed
+1 💚compile1m 11smaster passed
+1 💚shadedjars7m 46sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 42smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 16sthe patch passed
+1 💚compile1m 11sthe patch passed
+1 💚javac1m 11sthe patch passed
+1 💚shadedjars7m 46spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 42sthe patch passed
_ Other Tests _
+1 💚unit139m 16shbase-server in the patch passed.
169m 52s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3161
Optional Testsjavac javadoc unit shadedjars compile
unameLinux aec406eedde3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c5b0989
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/3/testReport/
Max. process+thread count4110 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9

Apache9 commented Apr 15, 2021

Copy link
Copy Markdown
ContributorAuthor

LGTM, +1.

As an additional thought, should we consider disable balancing altogether during maintenance mode? After all, in this mode cluster capability will already be limited and is supposed to last for short time, requiring a restart to move it out of safe mode.

Currectly we have a skipRegionManagementAction method, in this method, if maintenance mode is on, we will true, then we will skip some actions such as balancing or normalizing.

@Apache-HBase

Apache-HBase commented Apr 15, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 15sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 58smaster passed
+1 💚compile1m 3smaster passed
+1 💚shadedjars8m 25sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 4sthe patch passed
+1 💚compile1m 4sthe patch passed
+1 💚javac1m 4sthe patch passed
+1 💚shadedjars8m 28spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sthe patch passed
_ Other Tests _
+1 💚unit215m 19shbase-server in the patch passed.
246m 27s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3161
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 9f891271ed26 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c5b0989
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/3/testReport/
Max. process+thread count3358 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3161/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9
Apache9 merged commit bf78246 into apache:masterApr 16, 2021
Apache9 added a commit that referenced this pull request Apr 16, 2021
@Apache9
HBASE-25775Use a special balancer to deal with maintenance mode (#3161)
a177fb4
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Apache9 added a commit that referenced this pull request Apr 16, 2021
@Apache9
HBASE-25775Use a special balancer to deal with maintenance mode (#3161)
1444cdf
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Apache9 added a commit that referenced this pull request Apr 16, 2021
@Apache9
HBASE-25775Use a special balancer to deal with maintenance mode (#3161)
a15fd5c
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wchevreuilwchevreuilwchevreuil approved these changes
@saintstacksaintstackAwaiting requested review from saintstack

Assignees

@Apache9Apache9

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Apache9@Apache-HBase@wchevreuil