Skip to content

HBASE-26726 Allow disable of region warmup before graceful move - #4085

Merged
apurtell merged 1 commit into
apache:masterfrom
apurtell:HBASE-26726
Feb 1, 2022
Merged

HBASE-26726 Allow disable of region warmup before graceful move#4085
apurtell merged 1 commit into
apache:masterfrom
apurtell:HBASE-26726

Conversation

@apurtell

@apurtellapurtell commented Jan 31, 2022

Copy link
Copy Markdown
Contributor

We have encountered two issues with region warmup before the assignment manager gracefully moves a region from one regionserver to another.

The first instance is HBASE-26722. Part of the failure chain is temporary double assignment like conditions where both the source and destination regionservers think they have exclusive rights to storefile management (as they should) but warmup opens a region before it is closed and then both regionservers take compaction related actions. While this can be remediated with more care to this case, it is unclear if warmup affords significant advantage. The motivation of the original commit in 2015 introducing this feature was avoidance of blockcache misses once region ownership transfer was advertised to clients. Depending on use case and additional default-false schema options (like preload) this could be valuable. Or not.

The second instance is in place upgrade from HBase 1 to HBase 2. In a scenario where regionservers have been replaced by HBase 2 versions, but the master is still HBase 1, the region warmup RPC request fails. This is not particularly harmful but indicates it will not be useful during the transition period.

We think it would be good to allow for warmup before move to be optionally disabled by a site configuration setting. In particular there have been many unrelated changes committed since 2015 and expectations of invariants in the contribution of the warmup-on-move feature have been invalidated. HBASE-26722 may be the only case, or it might not. This is a fairly trivial change.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 11sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 43smaster passed
+1 💚compile3m 36smaster passed
+1 💚checkstyle1m 12smaster passed
+1 💚spotbugs2m 20smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 21sthe patch passed
+1 💚compile3m 24sthe patch passed
+1 💚javac3m 24sthe patch passed
+1 💚checkstyle1m 11sthe patch passed
+1 💚whitespace0m 1sThe patch has no whitespace issues.
+1 💚hadoopcheck21m 51sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs2m 25sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
55m 8s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4085
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 0ba2af7d2bfc 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 351caa5
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 35sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 53smaster passed
+1 💚compile1m 19smaster passed
+1 💚shadedjars10m 9sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 46smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 21sthe patch passed
+1 💚compile1m 26sthe patch passed
+1 💚javac1m 26sthe patch passed
+1 💚shadedjars11m 45spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 59sthe patch passed
_ Other Tests _
-1 ❌unit183m 57shbase-server in the patch failed.
224m 23s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4085
Optional Testsjavac javadoc unit shadedjars compile
unameLinux bda0ce516242 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 351caa5
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/testReport/
Max. process+thread count4062 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec3m 48sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall6m 4smaster passed
+1 💚compile1m 34smaster passed
+1 💚shadedjars9m 43sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 47smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 16sthe patch passed
+1 💚compile1m 24sthe patch passed
+1 💚javac1m 24sthe patch passed
+1 💚shadedjars9m 57spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 44sthe patch passed
_ Other Tests _
+1 💚unit217m 35shbase-server in the patch passed.
258m 47s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4085
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a22173b22afe 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 351caa5
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/testReport/
Max. process+thread count3392 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4085/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@virajjasanivirajjasani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, failed tests not relevant

@apurtell

Copy link
Copy Markdown
ContributorAuthor

Thanks.
Agreed, the test failures are not relevant. The default preserves existing behavior, too.

@apurtell
apurtell merged commit 3da23c2 into apache:masterFeb 1, 2022
@apurtell
apurtell deleted the HBASE-26726 branch February 1, 2022 23:38
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@apurtell@Apache-HBase@virajjasani