Skip to content

HBASE-23078 BaseLoadBalancer should consider region replicas when ran… - #663

Merged
infraio merged 1 commit into
apache:masterfrom
infraio:HBASE-23078
Sep 29, 2019
Merged

HBASE-23078 BaseLoadBalancer should consider region replicas when ran…#663
infraio merged 1 commit into
apache:masterfrom
infraio:HBASE-23078

Conversation

@infraio

@infraioinfraio commented Sep 26, 2019

Copy link
Copy Markdown
Contributor

…domAssignment and roundRobinAssignment

saintstack
saintstack approved these changes Sep 26, 2019
View reviewed changes

@saintstacksaintstack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup.

How you think this changes the behavior of the balancer?

Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java Outdated
TableName tableName = regionInfo.getTable();
if (services != null &&
services.getTableDescriptors().get(tableName).getRegionReplication() > 1) {
hasRegionReplica = true;

@saintstacksaintstackSep 26, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good

@Apache-HBase

Apache-HBase commented Sep 26, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec1m 12sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ master Compile Tests _
💚mvninstall5m 56smaster passed
💚compile0m 55smaster passed
💚checkstyle1m 33smaster passed
💚shadedjars5m 9sbranch has no errors when building our shaded downstream artifacts.
💚javadoc0m 38smaster passed
💙spotbugs4m 30sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs4m 27smaster passed
_ Patch Compile Tests _
💚mvninstall5m 31sthe patch passed
💚compile0m 59sthe patch passed
💚javac0m 59sthe patch passed
💔checkstyle1m 28shbase-server: The patch generated 2 new + 61 unchanged - 0 fixed = 63 total (was 61)
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars5m 3spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck17m 25sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚javadoc0m 37sthe patch passed
💚findbugs4m 40sthe patch passed
_ Other Tests _
💔unit234m 32shbase-server in the patch failed.
💚asflicense0m 30sThe patch does not generate ASF License warnings.
297m 34s
ReasonTests
Failed junit testshadoop.hbase.master.TestRegionPlacement2
hadoop.hbase.master.balancer.TestBaseLoadBalancer
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/1/artifact/out/Dockerfile
GITHUB PR#663
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 0460cea06f1d 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-663/out/precommit/personality/provided.sh
git revisionmaster / f0dddd1
Default Java1.8.0_181
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/1/artifact/out/diff-checkstyle-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/1/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/1/testReport/
Max. process+thread count4963 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/1/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

ramkrish86
ramkrish86 reviewed Sep 27, 2019
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java Outdated
for (RegionInfo regionInfo : regions) {
TableName tableName = regionInfo.getTable();
if (services != null &&
services.getTableDescriptors().get(tableName).getRegionReplication() > 1) {

@ramkrish86ramkrish86Sep 27, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the descriptors are anyway cached right? If so good.

@infraioinfraioSep 28, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in new patch. Will only getTableDescriptors once now.

@ramkrish86

ramkrish86 commented Sep 27, 2019

Copy link
Copy Markdown
Contributor

Looks good to me.

@Apache-HBase

Apache-HBase commented Sep 27, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec2m 28sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ master Compile Tests _
💚mvninstall8m 3smaster passed
💚compile1m 14smaster passed
💚checkstyle1m 40smaster passed
💚shadedjars6m 15sbranch has no errors when building our shaded downstream artifacts.
💚javadoc0m 52smaster passed
💙spotbugs5m 22sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs5m 20smaster passed
_ Patch Compile Tests _
💚mvninstall7m 27sthe patch passed
💚compile1m 18sthe patch passed
💚javac1m 18sthe patch passed
💚checkstyle1m 38sthe patch passed
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars6m 16spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck23m 2sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚javadoc0m 48sthe patch passed
💚findbugs6m 27sthe patch passed
_ Other Tests _
💔unit240m 23shbase-server in the patch failed.
💚asflicense0m 26sThe patch does not generate ASF License warnings.
322m 18s
ReasonTests
Failed junit testshadoop.hbase.master.TestRegionPlacement2
hadoop.hbase.master.TestMasterShutdown
hadoop.hbase.master.balancer.TestBaseLoadBalancer
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/2/artifact/out/Dockerfile
GITHUB PR#663
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux b9e768eaa24c 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-663/out/precommit/personality/provided.sh
git revisionmaster / 3250a80
Default Java1.8.0_181
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/2/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/2/testReport/
Max. process+thread count5097 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/2/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Sep 27, 2019

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
💙reexec0m 35sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ master Compile Tests _
💚mvninstall5m 26smaster passed
💚compile0m 56smaster passed
💚checkstyle1m 21smaster passed
💚shadedjars4m 31sbranch has no errors when building our shaded downstream artifacts.
💚javadoc0m 38smaster passed
💙spotbugs4m 1sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs3m 59smaster passed
_ Patch Compile Tests _
💚mvninstall4m 55sthe patch passed
💚compile0m 56sthe patch passed
💚javac0m 56sthe patch passed
💚checkstyle1m 19sthe patch passed
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars4m 32spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck15m 30sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚javadoc0m 35sthe patch passed
💚findbugs4m 10sthe patch passed
_ Other Tests _
💚unit157m 12shbase-server in the patch passed.
💚asflicense0m 35sThe patch does not generate ASF License warnings.
213m 34s
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/3/artifact/out/Dockerfile
GITHUB PR#663
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 44d0e592d884 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-663/out/precommit/personality/provided.sh
git revisionmaster / 3250a80
Default Java1.8.0_181
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/3/testReport/
Max. process+thread count4855 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/3/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@infraio

infraio commented Sep 28, 2019

Copy link
Copy Markdown
ContributorAuthor

Add the region replica UT back.

@Apache-HBase

Apache-HBase commented Sep 28, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec2m 10sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 3 new or modified test files.
_ master Compile Tests _
💚mvninstall7m 43smaster passed
💚compile1m 2smaster passed
💚checkstyle1m 35smaster passed
💚shadedjars5m 21sbranch has no errors when building our shaded downstream artifacts.
💚javadoc0m 41smaster passed
💙spotbugs4m 40sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs4m 37smaster passed
_ Patch Compile Tests _
💚mvninstall5m 39sthe patch passed
💚compile1m 4sthe patch passed
💚javac1m 4sthe patch passed
💔checkstyle1m 34shbase-server: The patch generated 1 new + 71 unchanged - 0 fixed = 72 total (was 71)
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars5m 17spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck18m 20sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚javadoc0m 34sthe patch passed
💚findbugs4m 32sthe patch passed
_ Other Tests _
💔unit244m 16shbase-server in the patch failed.
💚asflicense0m 28sThe patch does not generate ASF License warnings.
312m 1s
ReasonTests
Failed junit testshadoop.hbase.master.TestMasterShutdown
hadoop.hbase.client.TestAsyncRegionAdminApi2
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/4/artifact/out/Dockerfile
GITHUB PR#663
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux ce172c12f057 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-663/out/precommit/personality/provided.sh
git revisionmaster / ef79b40
Default Java1.8.0_181
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/4/artifact/out/diff-checkstyle-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/4/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/4/testReport/
Max. process+thread count5062 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/4/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Sep 29, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec1m 15sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 3 new or modified test files.
_ master Compile Tests _
💚mvninstall5m 59smaster passed
💚compile0m 57smaster passed
💚checkstyle1m 32smaster passed
💚shadedjars5m 5sbranch has no errors when building our shaded downstream artifacts.
💚javadoc0m 40smaster passed
💙spotbugs4m 25sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs4m 23smaster passed
_ Patch Compile Tests _
💚mvninstall5m 30sthe patch passed
💚compile0m 56sthe patch passed
💚javac0m 56sthe patch passed
💚checkstyle1m 28sthe patch passed
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars5m 3spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck17m 9sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚javadoc0m 35sthe patch passed
💚findbugs4m 32sthe patch passed
_ Other Tests _
💔unit254m 8shbase-server in the patch failed.
💚asflicense0m 26sThe patch does not generate ASF License warnings.
316m 28s
ReasonTests
Failed junit testshadoop.hbase.master.assignment.TestRegionBypass
hadoop.hbase.util.TestFromClientSide3WoUnsafe
hadoop.hbase.master.procedure.TestRestoreSnapshotProcedure
hadoop.hbase.master.TestSplitWALManager
hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/5/artifact/out/Dockerfile
GITHUB PR#663
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 672d38183af9 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-663/out/precommit/personality/provided.sh
git revisionmaster / 8363681
Default Java1.8.0_181
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/5/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/5/testReport/
Max. process+thread count5006 (vs. ulimit of 10000)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/5/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@infraio

infraio commented Sep 29, 2019

Copy link
Copy Markdown
ContributorAuthor

The failed ut not related and passed.

@infraio
infraio merged commit 1170f28 into apache:masterSep 29, 2019
@infraio
infraio deleted the HBASE-23078 branch September 29, 2019 09:36
asfgit pushed a commit that referenced this pull request Sep 29, 2019
@infraio
HBASE-23078BaseLoadBalancer should consider region replicas when ran…
9f703fc
…domAssignment and roundRobinAssignment (#663)
Signed-off-by: stack <stack@apache.org>
asfgit pushed a commit that referenced this pull request Sep 29, 2019
@infraio
HBASE-23078BaseLoadBalancer should consider region replicas when ran…
c50e3f3
…domAssignment and roundRobinAssignment (#663)
Signed-off-by: stack <stack@apache.org>
infraio added a commit to infraio/hbase that referenced this pull request Aug 17, 2020
@infraio@Apache9
HBASE-23078 BaseLoadBalancer should consider region replicas when ran…
2dafc24
…domAssignment and roundRobinAssignment (apache#663)
Signed-off-by: stack <stack@apache.org>
symat pushed a commit to symat/hbase that referenced this pull request Feb 17, 2021
@infraio
HBASE-23078 BaseLoadBalancer should consider region replicas when ran…
b7e692e
…domAssignment and roundRobinAssignment (apache#663)
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit c50e3f3)
Change-Id: I596ee594082021941f21ceb1b8150fc85ccb513b
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ramkrish86ramkrish86ramkrish86 left review comments
@saintstacksaintstacksaintstack approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@infraio@Apache-HBase@ramkrish86@saintstack