Skip to content

HBASE-28750 Region normalizer should work in off peak if config - #6113

Open
wangxin97-02 wants to merge 1 commit into
apache:masterfrom
wangxin97-02:HBASE-28750
Open

HBASE-28750 Region normalizer should work in off peak if config#6113
wangxin97-02 wants to merge 1 commit into
apache:masterfrom
wangxin97-02:HBASE-28750

Conversation

@wangxin97-02

@wangxin97-02wangxin97-02 commented Jul 23, 2024

Copy link
Copy Markdown
Contributor

Region normalizer involves the splitting and merging of regions, which can cause jitter in online services, especially when there are many region normalizer plans. We should run this task during off peak hours if config.

@Apache-HBase

Apache-HBase commented Jul 23, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 33sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 2smaster passed
+1 💚compile0m 59smaster passed
+1 💚javadoc0m 31smaster passed
+1 💚shadedjars5m 23sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 55sthe patch passed
+1 💚compile0m 58sthe patch passed
+1 💚javac0m 58sthe patch passed
+1 💚javadoc0m 28sthe patch passed
+1 💚shadedjars5m 20spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit13m 26s/patch-unit-hbase-server.txthbase-server in the patch failed.
35m 25s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6113
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 9effec4ca0f6 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / aec62bc
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/testReport/
Max. process+thread count1669 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 23, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 37sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall3m 4smaster passed
+1 💚compile2m 56smaster passed
+1 💚checkstyle0m 36smaster passed
+1 💚spotbugs1m 34smaster passed
+1 💚spotless0m 46sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 51sthe patch passed
+1 💚compile2m 54sthe patch passed
+1 💚javac2m 54sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 37sthe patch passed
+1 💚spotbugs1m 39sthe patch passed
+1 💚hadoopcheck10m 32sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚spotless0m 44spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
35m 45s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6113
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 0e746eb74a97 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / aec62bc
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count83 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil
wchevreuil reviewed Jul 23, 2024
View reviewed changes

@wchevreuilwchevreuil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice addition. Is it possible add test for this condition in TestRegionNormalizerWorker?

Comment thread...e-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorker.java
}
}

private void sleepToNextHour() {

@wchevreuilwchevreuilJul 23, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Add an info log saying offpeak hours is configured and we'll wait for offpeak hours to continue normalising.

@wangxin97-02

wangxin97-02 commented Jul 26, 2024

Copy link
Copy Markdown
ContributorAuthor

This is a nice addition. Is it possible add test for this condition in TestRegionNormalizerWorker?

It is ok.

@Apache-HBase

Apache-HBase commented Jul 26, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 38sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall3m 5smaster passed
+1 💚compile2m 56smaster passed
+1 💚checkstyle0m 38smaster passed
+1 💚spotbugs1m 34smaster passed
+1 💚spotless0m 46sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 56sthe patch passed
+1 💚compile2m 55sthe patch passed
+1 💚javac2m 55sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
-0 ⚠️checkstyle0m 38s/results-checkstyle-hbase-server.txthbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚spotbugs1m 40sthe patch passed
+1 💚hadoopcheck10m 43sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 ❌spotless0m 39spatch has 23 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 💚asflicense0m 12sThe patch does not generate ASF License warnings.
35m 59s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6113
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux f5cf9d63837f 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 37b8af7
Default JavaEclipse Adoptium-17.0.11+9
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count85 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 26, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 36sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 4smaster passed
+1 💚compile0m 59smaster passed
+1 💚javadoc0m 31smaster passed
+1 💚shadedjars5m 25sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 58sthe patch passed
+1 💚compile0m 58sthe patch passed
+1 💚javac0m 58sthe patch passed
+1 💚javadoc0m 30sthe patch passed
+1 💚shadedjars5m 17spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit232m 20s/patch-unit-hbase-server.txthbase-server in the patch failed.
257m 23s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6113
Optional Testsjavac javadoc unit compile shadedjars
unameLinux ab85b4fb8cf5 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 37b8af7
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/testReport/
Max. process+thread count5251 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 28, 2024

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec4m 2sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 1sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall2m 56smaster passed
+1 💚compile2m 56smaster passed
+1 💚checkstyle0m 37smaster passed
+1 💚spotbugs1m 32smaster passed
+1 💚spotless0m 43sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 53sthe patch passed
+1 💚compile2m 55sthe patch passed
+1 💚javac2m 55sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
-0 ⚠️checkstyle0m 37s/results-checkstyle-hbase-server.txthbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚spotbugs1m 39sthe patch passed
+1 💚hadoopcheck10m 33sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 ❌spotless0m 39spatch has 23 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 💚asflicense0m 10sThe patch does not generate ASF License warnings.
38m 50s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6113
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 008f7cf66619 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e066a29
Default JavaEclipse Adoptium-17.0.11+9
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count83 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 28, 2024

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 32sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 57smaster passed
+1 💚compile0m 58smaster passed
+1 💚javadoc0m 30smaster passed
+1 💚shadedjars5m 23sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall3m 0sthe patch passed
+1 💚compile0m 59sthe patch passed
+1 💚javac0m 59sthe patch passed
+1 💚javadoc0m 29sthe patch passed
+1 💚shadedjars5m 23spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit223m 35shbase-server in the patch passed.
248m 27s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6113
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 42adfebb16c2 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e066a29
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/testReport/
Max. process+thread count5576 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@wangxin97-02

wangxin97-02 commented Aug 30, 2024

Copy link
Copy Markdown
ContributorAuthor

@wchevreuil hello, If you have time, please take a review again.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wchevreuilwchevreuilAwaiting requested review from wchevreuil

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@wangxin97-02@Apache-HBase@wchevreuil