Skip to content

HBASE-22819 Automatically migrate the rs group config for table after… - #498

Merged
Apache9 merged 1 commit into
apache:HBASE-22514from
Apache9:HBASE-22819
Aug 25, 2019
Merged

HBASE-22819 Automatically migrate the rs group config for table after…#498
Apache9 merged 1 commit into
apache:HBASE-22514from
Apache9:HBASE-22819

Conversation

@Apache9

@Apache9Apache9 commented Aug 15, 2019

Copy link
Copy Markdown
Contributor

@Apache-HBase

Apache-HBase commented Aug 15, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec93Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep40Maven dependency ordering for branch
+1mvninstall313HBASE-22514 passed
+1compile73HBASE-22514 passed
+1checkstyle96HBASE-22514 passed
+1shadedjars271branch has no errors when building our shaded downstream artifacts.
+1javadoc53HBASE-22514 passed
0spotbugs269Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs315HBASE-22514 passed
_ Patch Compile Tests _
0mvndep14Maven dependency ordering for patch
+1mvninstall289the patch passed
+1compile78the patch passed
+1javac78the patch passed
-1checkstyle73hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1whitespace0The patch has no whitespace issues.
+1shadedjars270patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck974Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc52the patch passed
+1findbugs337the patch passed
_ Other Tests _
+1unit175hbase-common in the patch passed.
-1unit18795hbase-server in the patch failed.
+1asflicense94The patch does not generate ASF License warnings.
22854
ReasonTests
Failed junit testshadoop.hbase.security.access.TestSnapshotScannerHDFSAclController
hadoop.hbase.client.TestSnapshotTemporaryDirectory
hadoop.hbase.rsgroup.TestMigrateRSGroupInfo
hadoop.hbase.namespace.TestNamespaceAuditor
hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory
hadoop.hbase.client.TestCloneSnapshotFromClientNormal
hadoop.hbase.client.TestAdmin1
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/1/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 20e31fc53cb5 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 07fe41d
Default Java1.8.0_181
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/1/artifact/out/diff-checkstyle-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/1/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/1/testReport/
Max. process+thread count4934 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/1/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

sunhelly
sunhelly reviewed Aug 16, 2019
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java Outdated
.setRegionServerGroup(groupInfo.getName()).build();
try {
pendingProcIds.add(
masterServices.modifyTable(tableName, newTd, HConstants.NO_NONCE, HConstants.NO_NONCE));

@sunhellysunhellyAug 16, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just updating the meta row of the table to add group info, avoiding reopening regions when modify table?

@sunhellysunhellyAug 16, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I thought about it again. New implementation will make tables all in the default group until migrating them to their belongs groups. So reopening maybe necessary.

@Apache9Apache9Aug 16, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is the safest way...

@Apache-HBase

Apache-HBase commented Aug 16, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec42Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep38Maven dependency ordering for branch
+1mvninstall430HBASE-22514 passed
+1compile101HBASE-22514 passed
+1checkstyle138HBASE-22514 passed
+1shadedjars352branch has no errors when building our shaded downstream artifacts.
+1javadoc68HBASE-22514 passed
0spotbugs346Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs405HBASE-22514 passed
_ Patch Compile Tests _
0mvndep18Maven dependency ordering for patch
+1mvninstall404the patch passed
+1compile97the patch passed
+1javac97the patch passed
+1checkstyle137the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedjars360patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck1232Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc64the patch passed
+1findbugs425the patch passed
_ Other Tests _
+1unit212hbase-common in the patch passed.
-1unit8899hbase-server in the patch failed.
+1asflicense47The patch does not generate ASF License warnings.
13972
ReasonTests
Failed junit testshadoop.hbase.security.token.TestZKSecretWatcher
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/2/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux d1fa5b5d721d 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 07fe41d
Default Java1.8.0_181
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/2/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/2/testReport/
Max. process+thread count4344 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/2/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 16, 2019

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
0reexec88Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep36Maven dependency ordering for branch
+1mvninstall361HBASE-22514 passed
+1compile87HBASE-22514 passed
+1checkstyle121HBASE-22514 passed
+1shadedjars311branch has no errors when building our shaded downstream artifacts.
+1javadoc64HBASE-22514 passed
0spotbugs292Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs347HBASE-22514 passed
_ Patch Compile Tests _
0mvndep17Maven dependency ordering for patch
+1mvninstall351the patch passed
+1compile87the patch passed
+1javac87the patch passed
+1checkstyle117the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedjars306patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck1064Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc59the patch passed
+1findbugs355the patch passed
_ Other Tests _
+1unit181hbase-common in the patch passed.
+1unit12187hbase-server in the patch passed.
+1asflicense46The patch does not generate ASF License warnings.
16631
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/3/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 1ea80eb328bc 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 07fe41d
Default Java1.8.0_181
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/3/testReport/
Max. process+thread count4217 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/3/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 16, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec87Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep41Maven dependency ordering for branch
+1mvninstall454HBASE-22514 passed
+1compile104HBASE-22514 passed
+1checkstyle124HBASE-22514 passed
+1shadedjars387branch has no errors when building our shaded downstream artifacts.
+1javadoc72HBASE-22514 passed
0spotbugs321Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs380HBASE-22514 passed
_ Patch Compile Tests _
0mvndep18Maven dependency ordering for patch
+1mvninstall409the patch passed
+1compile96the patch passed
+1javac96the patch passed
+1checkstyle130the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedjars372patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck1232Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc63the patch passed
+1findbugs404the patch passed
_ Other Tests _
+1unit196hbase-common in the patch passed.
-1unit18556hbase-server in the patch failed.
+1asflicense153The patch does not generate ASF License warnings.
23831
ReasonTests
Failed junit testshadoop.hbase.rsgroup.TestMigrateRSGroupInfo
hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient
hadoop.hbase.client.TestSnapshotTemporaryDirectory
hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory
hadoop.hbase.tool.TestSecureBulkLoadHFiles
hadoop.hbase.snapshot.TestFlushSnapshotFromClient
hadoop.hbase.client.TestCloneSnapshotFromClientNormal
hadoop.hbase.namespace.TestNamespaceAuditor
hadoop.hbase.tool.TestBulkLoadHFiles
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/5/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux bbf774a75aeb 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 32a2015
Default Java1.8.0_181
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/5/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/5/testReport/
Max. process+thread count4500 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/5/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 18, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec61Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep35Maven dependency ordering for branch
+1mvninstall329HBASE-22514 passed
+1compile79HBASE-22514 passed
+1checkstyle104HBASE-22514 passed
+1shadedjars283branch has no errors when building our shaded downstream artifacts.
+1javadoc55HBASE-22514 passed
0spotbugs272Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs322HBASE-22514 passed
_ Patch Compile Tests _
0mvndep15Maven dependency ordering for patch
+1mvninstall367the patch passed
+1compile79the patch passed
+1javac79the patch passed
+1checkstyle101the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedjars317patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck1005Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc54the patch passed
+1findbugs340the patch passed
_ Other Tests _
+1unit172hbase-common in the patch passed.
-1unit16315hbase-server in the patch failed.
+1asflicense55The patch does not generate ASF License warnings.
20536
ReasonTests
Failed junit testshadoop.hbase.rsgroup.TestMigrateRSGroupInfo
hadoop.hbase.client.TestAsyncTableGetMultiThreaded
hadoop.hbase.regionserver.TestSplitTransactionOnCluster
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/6/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux a283e04fde9b 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 8660b52
Default Java1.8.0_181
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/6/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/6/testReport/
Max. process+thread count4705 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/6/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 18, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec534Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep83Maven dependency ordering for branch
+1mvninstall769HBASE-22514 passed
+1compile587HBASE-22514 passed
+1checkstyle295HBASE-22514 passed
+1shadedjars832branch has no errors when building our shaded downstream artifacts.
+1javadoc141HBASE-22514 passed
0spotbugs82Used deprecated FindBugs config; considering switching to SpotBugs.
-1findbugs1130hbase-server in HBASE-22514 failed.
_ Patch Compile Tests _
0mvndep115Maven dependency ordering for patch
+1mvninstall627the patch passed
+1compile116the patch passed
+1javac116the patch passed
+1checkstyle136the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedjars384patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck1543Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc77the patch passed
-1findbugs686hbase-server in the patch failed.
_ Other Tests _
+1unit205hbase-common in the patch passed.
-1unit2699hbase-server in the patch failed.
+1asflicense62The patch does not generate ASF License warnings.
11844
ReasonTests
Failed junit testshadoop.hbase.client.TestAsyncAccessControlAdminApi
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/7/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 34365d222f30 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 8660b52
Default Java1.8.0_181
findbugshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/7/artifact/out/branch-findbugs-hbase-server.txt
findbugshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/7/artifact/out/patch-findbugs-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/7/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/7/testReport/
Max. process+thread count748 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/7/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 20, 2019

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
0reexec100Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 1 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep35Maven dependency ordering for branch
+1mvninstall357HBASE-22514 passed
+1compile93HBASE-22514 passed
+1checkstyle124HBASE-22514 passed
+1shadedjars334branch has no errors when building our shaded downstream artifacts.
+1javadoc66HBASE-22514 passed
0spotbugs293Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs345HBASE-22514 passed
_ Patch Compile Tests _
0mvndep15Maven dependency ordering for patch
+1mvninstall331the patch passed
+1compile86the patch passed
+1javac86the patch passed
+1checkstyle115the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedjars304patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck1045Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc58the patch passed
+1findbugs408the patch passed
_ Other Tests _
+1unit203hbase-common in the patch passed.
+1unit9506hbase-server in the patch passed.
+1asflicense41The patch does not generate ASF License warnings.
13989
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/9/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux a15ee86a38fa 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 8660b52
Default Java1.8.0_181
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/9/testReport/
Max. process+thread count4031 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/9/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

infraio
infraio reviewed Aug 20, 2019
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java Outdated
TableDescriptor newTd = TableDescriptorBuilder.newBuilder(oldTd)
.setRegionServerGroup(groupInfo.getName()).build();
try {
pendingProcIds.add(

@infraioinfraioAug 20, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the user which used RSGroup feature a lot, when user restart master with new code, this migrate thing will reopen all regions... This is not friendly for user? Can we finish this job by a tool? Only use the tool to change the TableDescriptor first, then upgrade cluster to new version code.

@Apache9Apache9Aug 20, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think doing things automatically will be more friendly? As users may forget to user the tool and all the table configs will be ignored...

@infraioinfraioAug 20, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need a new issue to add document for this? And add a config for maxConcurrency? If user decide to upgrade, user can use this config to reduce the impact for availability.

@Apache-HBase

Apache-HBase commented Aug 23, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec111Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 2 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep39Maven dependency ordering for branch
+1mvninstall364HBASE-22514 passed
+1compile89HBASE-22514 passed
+1checkstyle118HBASE-22514 passed
+1shadedjars287branch has no errors when building our shaded downstream artifacts.
+1javadoc56HBASE-22514 passed
0spotbugs291Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs343HBASE-22514 passed
_ Patch Compile Tests _
0mvndep16Maven dependency ordering for patch
+1mvninstall305the patch passed
+1compile77the patch passed
+1javac77the patch passed
+1checkstyle107the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedjars289patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck982Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc56the patch passed
+1findbugs351the patch passed
_ Other Tests _
+1unit172hbase-common in the patch passed.
-1unit15202hbase-server in the patch failed.
+1asflicense57The patch does not generate ASF License warnings.
19429
ReasonTests
Failed junit testshadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated
hadoop.hbase.client.TestFromClientSide3
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/12/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux fe66daec900b 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 4c570ed
Default Java1.8.0_181
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/12/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/12/testReport/
Max. process+thread count4720 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/12/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

infraio
infraio reviewed Aug 24, 2019
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java Outdated
infraio
infraio reviewed Aug 24, 2019
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupUtil.java
private static Optional<RSGroupInfo> getRSGroupInfo(MasterServices master, GetRSGroup getter,
/**
* Will try to get the rsgroup from {@link TableDescriptor} first, and then try to get the rsgroup
* from the {@link NamespaceDescriptor}. If still not present, return empty.

@infraioinfraioAug 24, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems the impl didn't get rsgroup from NamespaceDescriptor?

@Apache9Apache9Aug 24, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See below, you need to show all the code, include the unmodified pieces.

infraio
infraio reviewed Aug 24, 2019
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java Outdated
infraio
infraio reviewed Aug 24, 2019
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java
LOG.info("Start migrating table rs group config");
while (!masterServices.isStopped()) {
Collection<RSGroupInfo> groups = holder.groupName2Group.values();
boolean hasTables = groups.stream().anyMatch(r -> !r.getTables().isEmpty());

@infraioinfraioAug 24, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When restart master again, how to avoid migrate again?

@Apache9Apache9Aug 24, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the above method, we have lots of checks, and once we finish migrate one group, we will update the RSGroupInfo to remove all the tables in it.

@Apache-HBase

Apache-HBase commented Aug 24, 2019

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
0reexec113Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1hbaseanti0Patch does not have any anti-patterns.
+1@author0The patch does not contain any @author tags.
+1test4tests0The patch appears to include 2 new or modified test files.
_ HBASE-22514 Compile Tests _
0mvndep37Maven dependency ordering for branch
+1mvninstall391HBASE-22514 passed
+1compile102HBASE-22514 passed
+1checkstyle128HBASE-22514 passed
+1shadedjars344branch has no errors when building our shaded downstream artifacts.
+1javadoc63HBASE-22514 passed
0spotbugs293Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs344HBASE-22514 passed
_ Patch Compile Tests _
0mvndep16Maven dependency ordering for patch
+1mvninstall336the patch passed
+1compile97the patch passed
+1javac97the patch passed
+1checkstyle122the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedjars345patch has no errors when building our shaded downstream artifacts.
+1hadoopcheck1185Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1javadoc58the patch passed
+1findbugs353the patch passed
_ Other Tests _
+1unit184hbase-common in the patch passed.
+1unit14674hbase-server in the patch passed.
+1asflicense51The patch does not generate ASF License warnings.
19420
SubsystemReport/Notes
DockerClient=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/14/artifact/out/Dockerfile
GITHUB PR#498
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 46f9ecc28129 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-498/out/precommit/personality/provided.sh
git revisionHBASE-22514 / 02f48d5
Default Java1.8.0_181
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/14/testReport/
Max. process+thread count4754 (vs. ulimit of 10000)
modulesC: hbase-common hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-498/14/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache9
Apache9 merged commit 24f4913 into apache:HBASE-22514Aug 25, 2019
asfgit pushed a commit that referenced this pull request Aug 25, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
4741f66
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Aug 27, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
548e8c4
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Sep 10, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
fbdafde
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Sep 11, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
23b6a27
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Sep 11, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
3cb0d3e
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Sep 16, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
5d652cc
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Sep 17, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
a1930d5
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Sep 18, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
80d69c5
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Sep 20, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
363b97e
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Sep 24, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
87a1115
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Oct 5, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
a47d2b5
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Oct 28, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
af122d3
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Nov 5, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
c162c88
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Nov 10, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
3dc1b28
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Nov 13, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
db43492
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Dec 24, 2019
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
8dca9dc
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 2, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
61fa836
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 3, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
f682cc4
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 4, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
571fa3d
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 6, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
a96780f
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 7, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
23f95cc
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 8, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
d102331
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 9, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
ed6c740
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 12, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
f1cba5c
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 20, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
4e70293
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 22, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
72ecdce
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 25, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
0924d69
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 26, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
34d3983
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
asfgit pushed a commit that referenced this pull request Feb 29, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
7512445
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Apache9 added a commit that referenced this pull request Mar 2, 2020
@Apache9
HBASE-22819Automatically migrate the rs group config for table afterH…
b3399ef
…BASE-22695 (#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
thangTang pushed a commit to thangTang/hbase that referenced this pull request Apr 16, 2020
@Apache9@thangTang
HBASE-22819 Automatically migrate the rs group config for table after…
b38a793
… HBASE-22695 (apache#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
thangTang pushed a commit to thangTang/hbase that referenced this pull request Apr 16, 2020
@Apache9@thangTang
HBASE-22819 Automatically migrate the rs group config for table after…
9573010
… HBASE-22695 (apache#498)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sunhellysunhellysunhelly left review comments
@infraioinfraioinfraio approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Apache9@Apache-HBase@infraio@sunhelly