HBASE-27903 Skip submitting Split/Merge procedure when split/merge is… by NihalJain · Pull Request #6169 · apache/hbase

NihalJain · 2024-08-20T19:17:43Z

… disabled at table level

Fail fast by adding a check before even submitting a procedure
Update test cases to assert for expected exception post this change
Remove deprecated method mergeRegionsAsync's usage in test
Make use of RegionInfo.getShortNameToLog instead of logging complete region info
Update comments in procedure implementation

… disabled at table level - Fail fast by adding a check before even submitting a procedure - Update test cases to assert for expected exception post this change - Make use of RegionInfo.getShortNameToLog instead of logging complete region info - Update comments in procedure implementation

NihalJain · 2024-08-20T19:26:41Z

Verified the change in standalone mode. The code works as expected i.e. the procedure is not even submitted and we fail fast with this patch and as a result no procedure rollback happens.

Shell output with fix:

hbase:004:0> merge_region '299fda3a5e8630098a46363b0581fb9a', 'e0c5f133ae0f23ce56296b8c763fa00f'
ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: Merge of [299fda3a5e8630098a46363b0581fb9a, e0c5f133ae0f23ce56296b8c763fa00f] failed as region merge is disabled for the table
at org.apache.hadoop.hbase.master.HMaster.mergeRegions(HMaster.java:2269)
at org.apache.hadoop.hbase.master.MasterRpcServices.mergeTableRegions(MasterRpcServices.java:942)
at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:444)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)
For usage try 'help "merge_region"'
Took 0.2516 seconds
hbase:006:0> split 'testcreatetablewithsplitdisableparameter','30'
ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: Split region testcreatetablewithsplitdisableparameter,,1724177100721.42bc211e7063bb1d8fa7619a50a98c3a. failed as region split is disabled for the table
at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:2303)
at org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:954)
at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:444)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)
For usage try 'help "split"'
Took 0.0204 seconds

Shell output without fix:

hbase:010:0> merge_region '62b765354fdb007d571b98b5e6b5542e.', '212a918d92b35bf3afe270962bedb350'
ERROR: Merge of [{ENCODED => 62b765354fdb007d571b98b5e6b5542e, NAME => 'testCreateTableWithMergeDisableParameter,,1724176086193.62b765354fdb007d571b98b5e6b5542e.', STARTKEY => '', ENDKEY => 'a'}, {ENCODED => 212a918d92b35bf3afe270962bedb350, NAME => 'testCreateTableWithMergeDisableParameter,a,1724176086193.212a918d92b35bf3afe270962bedb350.', STARTKEY => 'a', ENDKEY => ''}] failed as region merge is disabled for the table
For usage try 'help "merge_region"'
Took 0.2405 seconds
hbase:007:0> split 'testcreatetablewithsplitdisableparameter','30'
ERROR: Split region testcreatetablewithsplitdisableparameter,,1724176000059.66d8a2862968698d012927d9f9d4dbcc. failed as region split is disabled for the table
For usage try 'help "split"'
Took 0.3706 seconds

Logs with fix:

2024-08-20T23:34:41,735 WARN [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.HMaster: Merge is disabled for the table! Skipping merge of [299fda3a5e8630098a46363b0581fb9a, e0c5f133ae0f23ce56296b8c763fa00f]
2024-08-20T23:35:07,354 WARN [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.HMaster: Split is disabled for the table! Skipping split of {ENCODED => 42bc211e7063bb1d8fa7619a50a98c3a, NAME => 'testcreatetablewithsplitdisableparameter,,1724177100721.42bc211e7063bb1d8fa7619a50a98c3a.', STARTKEY => '', ENDKEY => ''}

Logs without fix:

2024-08-20T23:18:53,744 WARN [PEWorker-2] assignment.MergeTableRegionsProcedure: Merge is disabled for the table! Skipping merge of [{ENCODED => 62b765354fdb007d571b98b5e6b5542e, NAME => 'testCreateTableWithMergeDisableParameter,,1724176086193.62b765354fdb007d571b98b5e6b5542e.', STARTKEY => '', ENDKEY => 'a'}, {ENCODED => 212a918d92b35bf3afe270962bedb350, NAME => 'testCreateTableWithMergeDisableParameter,a,1724176086193.212a918d92b35bf3afe270962bedb350.', STARTKEY => 'a', ENDKEY => ''}]
2024-08-20T23:17:36,504 WARN [PEWorker-1] assignment.SplitTableRegionProcedure: pid=13, split is disabled for the table! Skipping split of {ENCODED => 66d8a2862968698d012927d9f9d4dbcc, NAME => 'testcreatetablewithsplitdisableparameter,,1724176000059.66d8a2862968698d012927d9f9d4dbcc.', STARTKEY => '', ENDKEY => ''}

Apache-HBase · 2024-08-20T19:57:16Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 26s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	3m 4s		master passed
+1 💚	compile	3m 1s		master passed
+1 💚	checkstyle	0m 35s		master passed
+1 💚	spotbugs	1m 33s		master passed
+1 💚	spotless	0m 46s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 52s		the patch passed
+1 💚	compile	3m 0s		the patch passed
+1 💚	javac	3m 0s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 35s		the patch passed
+1 💚	spotbugs	1m 38s		the patch passed
+1 💚	hadoopcheck	11m 1s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	0m 41s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 10s		The patch does not generate ASF License warnings.
		36m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6169/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6169
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux cd520052c879 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `a4ae7e7`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6169/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-08-20T23:34:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	3m 5s		master passed
+1 💚	compile	1m 0s		master passed
+1 💚	javadoc	0m 31s		master passed
+1 💚	shadedjars	5m 26s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 0s		the patch passed
+1 💚	compile	0m 59s		the patch passed
+1 💚	javac	0m 59s		the patch passed
+1 💚	javadoc	0m 29s		the patch passed
+1 💚	shadedjars	5m 24s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	228m 26s		hbase-server in the patch passed.
		253m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6169/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6169
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 691532b33634 5.4.0-192-generic #212-Ubuntu SMP Fri Jul 5 09:47:39 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `a4ae7e7`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6169/1/testReport/
Max. process+thread count	5026 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6169/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2024-08-25T14:59:25Z


 // check and ensure that region do not get merged
- Future<?> f = admin.mergeRegionsAsync(nameOfRegionA, nameOfRegionB, true);
+ Future<?> f = admin.mergeRegionsAsync(new byte[][] { nameOfRegionA, nameOfRegionB }, true);


Why we need this change? Because the method which takes two byte[] is deprecated?

Hi Duo, yes saw this deprecated usage in the test. mentioned same in description. let me know if should revert

No problem. Just want to confirm.

… disabled at table level (apache#6169) - Fail fast by adding a check before even submitting a procedure - Update test cases to assert for expected exception post this change - Remove deprecated method mergeRegionsAsync's usage in test - Make use of RegionInfo.getShortNameToLog instead of logging complete region info - Update comments in procedure implementation Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit 2ac657c)

… disabled at table level (#6169) (#6185) - Fail fast by adding a check before even submitting a procedure - Update test cases to assert for expected exception post this change - Remove deprecated method mergeRegionsAsync's usage in test - Make use of RegionInfo.getShortNameToLog instead of logging complete region info - Update comments in procedure implementation Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit 2ac657c)

… disabled at table level (#6186) (#6169) - Fail fast by adding a check before even submitting a procedure - Update test cases to assert for expected exception post this change - Remove deprecated method mergeRegionsAsync's usage in test - Make use of RegionInfo.getShortNameToLog instead of logging complete region info - Update comments in procedure implementation - Fix tests as per 2.x code base: here the HBTU creates an instance of HBaseAdmin whose methods splitRegionAsync/mergeRegionsAsync seem to propagate error immediately without a call to future.get() which is in contrast to how it default admin instance of HBTU works in 3.x. Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit 2ac657c)

… disabled at table level (apache#6186) (apache#6169) - Fail fast by adding a check before even submitting a procedure - Update test cases to assert for expected exception post this change - Remove deprecated method mergeRegionsAsync's usage in test - Make use of RegionInfo.getShortNameToLog instead of logging complete region info - Update comments in procedure implementation - Fix tests as per 2.x code base: here the HBTU creates an instance of HBaseAdmin whose methods splitRegionAsync/mergeRegionsAsync seem to propagate error immediately without a call to future.get() which is in contrast to how it default admin instance of HBTU works in 3.x. Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit e658496)

… disabled at table level (#6186) (#6169) (#6285) - Fail fast by adding a check before even submitting a procedure - Update test cases to assert for expected exception post this change - Remove deprecated method mergeRegionsAsync's usage in test - Make use of RegionInfo.getShortNameToLog instead of logging complete region info - Update comments in procedure implementation - Fix tests as per 2.x code base: here the HBTU creates an instance of HBaseAdmin whose methods splitRegionAsync/mergeRegionsAsync seem to propagate error immediately without a call to future.get() which is in contrast to how it default admin instance of HBTU works in 3.x. Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit e658496)

Apache9 reviewed Aug 25, 2024
View reviewed changes

Apache9 approved these changes Aug 26, 2024
View reviewed changes

NihalJain merged commit 2ac657c into apache:masterAug 30, 2024

NihalJain mentioned this pull request Aug 30, 2024
Backport "HBASE-27903 Skip submitting Split/Merge procedure when split/merge is disabled at table level" to branch-3 #6185
Merged

NihalJain mentioned this pull request Aug 30, 2024
Backport "HBASE-27903 Skip submitting Split/Merge procedure when split/merge is disabled at table level" to branch-2 #6186
Merged

Navigation Menu

Uh oh!

HBASE-27903 Skip submitting Split/Merge procedure when split/merge is… - #6169

HBASE-27903 Skip submitting Split/Merge procedure when split/merge is…#6169
NihalJain merged 1 commit into
apache:masterfrom
NihalJain:HBASE-27903

NihalJain commented Aug 20, 2024

Uh oh!

NihalJain commented Aug 20, 2024

Uh oh!

Apache-HBase commented Aug 20, 2024

Uh oh!

Apache-HBase commented Aug 20, 2024

Uh oh!

Apache9Aug 25, 2024

Uh oh!

NihalJainAug 25, 2024

Uh oh!

Apache9Aug 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

NihalJain commented Aug 20, 2024

Uh oh!

NihalJain commented Aug 20, 2024

Uh oh!

Apache-HBase commented Aug 20, 2024

Uh oh!

Apache-HBase commented Aug 20, 2024

Uh oh!

Apache9Aug 25, 2024

Choose a reason for hiding this comment

Uh oh!

NihalJainAug 25, 2024

Choose a reason for hiding this comment

Uh oh!

Apache9Aug 26, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants