Skip to content

HBASE-28774: Protect hbase:meta from any hotspotting regions by balancing them to different region servers - #6173

Draft
ranganathg wants to merge 4 commits into
apache:masterfrom
ranganathg:HBASE-28774
Draft

HBASE-28774: Protect hbase:meta from any hotspotting regions by balancing them to different region servers#6173
ranganathg wants to merge 4 commits into
apache:masterfrom
ranganathg:HBASE-28774

Conversation

@ranganathg

@ranganathgranganathg commented Aug 22, 2024

Copy link
Copy Markdown

HBASE-28774
The idea is to provide a configuration where by one can configure mutually exclusive tables. For example: hbase:meta,SYSTEM.CATALOG can be mutually exclusive tables. Anytime these 2 tables come together the new plan will be costly and would need a rebalancing.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@ranganathg
ranganathg marked this pull request as ready for review August 22, 2024 14:57

Map<ServerName, List<RegionInfo>> clusterState = this.cluster.clusterState;

for (Map.Entry<ServerName, List<RegionInfo>> entry : clusterState.entrySet()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us add comment about what we are doing and how it would impact the balancer decision with an example.

@ranganathg
ranganathg marked this pull request as draft August 23, 2024 08:06
@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

* on same RS
*/
@InterfaceAudience.Private public class MutuallyExclusiveTablesCostFunction extends CostFunction {
public static final String MUTUALLY_EXCLUSIVE_TABLES_KEY =

@Umeshkumar9414Umeshkumar9414Aug 27, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: good to add comments for config.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 47sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 11smaster passed
+1 💚compile0m 18smaster passed
+1 💚javadoc0m 15smaster passed
+1 💚shadedjars5m 21sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall3m 4sthe patch passed
+1 💚compile0m 16sthe patch passed
+1 💚javac0m 16sthe patch passed
+1 💚javadoc0m 12sthe patch passed
+1 💚shadedjars5m 19spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit10m 20s/patch-unit-hbase-balancer.txthbase-balancer in the patch failed.
30m 29s
SubsystemReport/Notes
DockerClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6173
JIRA IssueHBASE-28774
Optional Testsjavac javadoc unit compile shadedjars
unameLinux d6f9069d05cc 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / be1b2fd
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/testReport/
Max. process+thread count262 (vs. ulimit of 30000)
modulesC: hbase-balancer U: hbase-balancer
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 50sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall3m 50smaster passed
+1 💚compile0m 23smaster passed
+1 💚checkstyle0m 9smaster passed
+1 💚spotbugs0m 29smaster passed
+1 💚spotless0m 51sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall3m 27sthe patch passed
+1 💚compile0m 27sthe patch passed
+1 💚javac0m 27sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
-0 ⚠️checkstyle0m 10s/results-checkstyle-hbase-balancer.txthbase-balancer: The patch generated 15 new + 1 unchanged - 0 fixed = 16 total (was 1)
+1 💚spotbugs0m 40sthe patch passed
+1 💚hadoopcheck11m 56sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 ❌spotless0m 19spatch has 70 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 💚asflicense0m 8sThe patch does not generate ASF License warnings.
30m 31s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6173
JIRA IssueHBASE-28774
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 3881e6c1ab0c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / be1b2fd
Default JavaEclipse Adoptium-17.0.11+9
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count81 (vs. ulimit of 30000)
modulesC: hbase-balancer U: hbase-balancer
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@ranganathg@Apache-HBase@mnpoonia@Umeshkumar9414