Skip to content

HBASE-24195 : Admin.getRegionServers() should return live servers exc… - #1523

Merged
virajjasani merged 2 commits into
apache:masterfrom
virajjasani:HBASE-24195-master
Apr 16, 2020
Merged

HBASE-24195 : Admin.getRegionServers() should return live servers exc…#1523
virajjasani merged 2 commits into
apache:masterfrom
virajjasani:HBASE-24195-master

Conversation

@virajjasani

Copy link
Copy Markdown
Contributor

…luding decom RS optionally

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 28sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 39sMaven dependency ordering for branch
+1 💚mvninstall3m 26smaster passed
+1 💚checkstyle1m 34smaster passed
+1 💚spotbugs2m 55smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
+1 💚mvninstall3m 9sthe patch passed
-0 ⚠️checkstyle0m 28shbase-client: The patch generated 1 new + 28 unchanged - 0 fixed = 29 total (was 28)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck10m 41sPatch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
+1 💚spotbugs3m 16sthe patch passed
_ Other Tests _
+1 💚asflicense0m 26sThe patch does not generate ASF License warnings.
35m 27s
SubsystemReport/Notes
DockerClient=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1523
Optional Testsdupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
unameLinux 217b48260cf1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 38954ed
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
Max. process+thread count94 (vs. ulimit of 12500)
modulesC: hbase-client hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 33sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 39sMaven dependency ordering for branch
+1 💚mvninstall3m 31smaster passed
+1 💚compile1m 18smaster passed
+1 💚shadedjars4m 49sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 58smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 15sMaven dependency ordering for patch
+1 💚mvninstall3m 22sthe patch passed
+1 💚compile1m 19sthe patch passed
+1 💚javac1m 19sthe patch passed
+1 💚shadedjars4m 49spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 57sthe patch passed
_ Other Tests _
+1 💚unit0m 56shbase-client in the patch passed.
+1 💚unit138m 0shbase-server in the patch passed.
163m 46s
SubsystemReport/Notes
DockerClient=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#1523
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 5359120d89b7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 38954ed
Default Java1.8.0_232
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/testReport/
Max. process+thread count3949 (vs. ulimit of 12500)
modulesC: hbase-client hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 33sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 38sMaven dependency ordering for branch
+1 💚mvninstall4m 15smaster passed
+1 💚compile1m 37smaster passed
+1 💚shadedjars5m 42sbranch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 28shbase-client in master failed.
-0 ⚠️javadoc0m 39shbase-server in master failed.
_ Patch Compile Tests _
+0 🆗mvndep0m 15sMaven dependency ordering for patch
+1 💚mvninstall4m 1sthe patch passed
+1 💚compile1m 33sthe patch passed
+1 💚javac1m 33sthe patch passed
+1 💚shadedjars6m 13spatch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 32shbase-client in the patch failed.
-0 ⚠️javadoc0m 52shbase-server in the patch failed.
_ Other Tests _
+1 💚unit1m 21shbase-client in the patch passed.
+1 💚unit137m 56shbase-server in the patch passed.
169m 3s
SubsystemReport/Notes
DockerClient=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#1523
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 2dcb4f35cbea 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 38954ed
Default Java2020-01-14
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/testReport/
Max. process+thread count3781 (vs. ulimit of 12500)
modulesC: hbase-client hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@liuml07liuml07 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks useful. +1 (non-binding) after comments are addressed.

waitForServerDecom(serverToDecommission, true);

Assert.assertEquals(2, ADMIN.getRegionServers(true).size());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add Assert.assertEquals(3, ADMIN.getRegionServers(false).size()); for paranoid reviewers (cough cough me).

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, definitely

ADMIN.recommissionRegionServer(serverToDecommission, Collections.emptyList());

Assert.assertEquals(3, ADMIN.getRegionServers(true).size());
waitForServerDecom(serverToDecommission, false);

@liuml07liuml07Apr 16, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why wait for decommission this server in the end?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking the assertion Assert.assertEquals(3, ADMIN.getRegionServers(true).size()); should have made sure all RS are fine now. I understand in case that is not enough, you explicitly make sure the mini cluster is ready for other test cases. OK, good.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why wait for decommission this server in the end?

Actually, we are waiting for server to be recommissioned (maybe method name not correct, last arg determines). Let me change the order.

return allServers;
}
List<ServerName> decommissionedRegionServers = listDecommissionedRegionServers();
if (CollectionUtils.isNotEmpty(decommissionedRegionServers)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 if (CollectionUtils.isNotEmpty(decommissionedRegionServers)) {
allServers = new ArrayList<>(allServers);
allServers.removeIf(decommissionedRegionServers::contains);
return Collections.unmodifiableList(allServers);
}
return allServers;

to

 return allServers.stream()
.filter(s -> !decommissionedRegionServers.contains(s))
.collect(ImmutableList.toImmutableList());

I think this will save one time full copy (previous code copy twice, and this copies just once). Need to confirm.

@virajjasanivirajjasaniApr 16, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason behind copy (new ArrayList<>(allServers)) was making allServers unmodifiable list. So ImmutableList.toImmutableList() seems to be solving our problem.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 8sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 37sMaven dependency ordering for branch
+1 💚mvninstall4m 52smaster passed
+1 💚checkstyle1m 58smaster passed
+1 💚spotbugs3m 50smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall4m 31sthe patch passed
+1 💚checkstyle2m 2sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck14m 32sPatch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
+1 💚spotbugs4m 10sthe patch passed
_ Other Tests _
+1 💚asflicense0m 25sThe patch does not generate ASF License warnings.
47m 38s
SubsystemReport/Notes
DockerClient=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1523
Optional Testsdupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
unameLinux 35274c71e872 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 94b4101
Max. process+thread count95 (vs. ulimit of 12500)
modulesC: hbase-client hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 31sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 35sMaven dependency ordering for branch
+1 💚mvninstall3m 23smaster passed
+1 💚compile1m 18smaster passed
+1 💚shadedjars4m 45sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 59smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for patch
+1 💚mvninstall3m 12sthe patch passed
+1 💚compile1m 19sthe patch passed
+1 💚javac1m 19sthe patch passed
+1 💚shadedjars4m 46spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 56sthe patch passed
_ Other Tests _
+1 💚unit0m 53shbase-client in the patch passed.
+1 💚unit117m 35shbase-server in the patch passed.
142m 54s
SubsystemReport/Notes
DockerClient=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#1523
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 02d3226261b1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 94b4101
Default Java1.8.0_232
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/testReport/
Max. process+thread count4334 (vs. ulimit of 12500)
modulesC: hbase-client hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 30sMaven dependency ordering for branch
+1 💚mvninstall4m 30smaster passed
+1 💚compile1m 36smaster passed
+1 💚shadedjars5m 55sbranch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 26shbase-client in master failed.
-0 ⚠️javadoc0m 38shbase-server in master failed.
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall4m 17sthe patch passed
+1 💚compile1m 41sthe patch passed
+1 💚javac1m 41sthe patch passed
+1 💚shadedjars6m 44spatch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 27shbase-client in the patch failed.
-0 ⚠️javadoc0m 42shbase-server in the patch failed.
_ Other Tests _
+1 💚unit1m 16shbase-client in the patch passed.
+1 💚unit174m 14shbase-server in the patch passed.
205m 33s
SubsystemReport/Notes
DockerClient=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#1523
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 6a18cbfd0a7b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 94b4101
Default Java2020-01-14
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/testReport/
Max. process+thread count3029 (vs. ulimit of 12500)
modulesC: hbase-client hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1523/2/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@virajjasani
virajjasani merged commit d212dc4 into apache:masterApr 16, 2020
@virajjasani
virajjasani deleted the HBASE-24195-master branch April 16, 2020 14:43
virajjasani added a commit that referenced this pull request Apr 16, 2020
#1523)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
virajjasani added a commit that referenced this pull request Apr 16, 2020
#1523)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
ddupg pushed a commit to ddupg/hbase that referenced this pull request Apr 20, 2020
apache#1523)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@virajjasani@Apache-HBase@liuml07@HorizonNet