Skip to content

HBASE-26556 IT and Chaos Monkey improvements - #3932

Merged
wchevreuil merged 2 commits into
apache:masterfrom
wchevreuil:HBASE-26556
Dec 14, 2021
Merged

HBASE-26556 IT and Chaos Monkey improvements#3932
wchevreuil merged 2 commits into
apache:masterfrom
wchevreuil:HBASE-26556

Conversation

@wchevreuil

@wchevreuilwchevreuil commented Dec 10, 2021

Copy link
Copy Markdown
Contributor

No description provided.

Wellington Chevreuil
HBASE-26556 IT and Chaos Monkey improvements
845cd02
Change-Id: I8e26c27c2f2f5600406c25de6fceb598c980f1d5
@Apache-HBase

Apache-HBase commented Dec 10, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 27sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 44smaster passed
+1 💚compile0m 30smaster passed
+1 💚shadedjars8m 20sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 21smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 29sthe patch passed
+1 💚compile0m 32sthe patch passed
-0 ⚠️javac0m 32shbase-it generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚shadedjars8m 12spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 18sthe patch passed
_ Other Tests _
+1 💚unit0m 56shbase-it in the patch passed.
30m 7s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3932
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 84c14a394c1a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / b5cf3cd
Default JavaAdoptOpenJDK-11.0.10+9
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-jdk11-hadoop3-check/output/diff-compile-javac-hbase-it.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/testReport/
Max. process+thread count698 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Dec 10, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec3m 56sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 13smaster passed
+1 💚compile0m 28smaster passed
+1 💚shadedjars8m 33sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 20smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 0sthe patch passed
+1 💚compile0m 29sthe patch passed
-0 ⚠️javac0m 29shbase-it generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚shadedjars8m 39spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 17sthe patch passed
_ Other Tests _
+1 💚unit0m 59shbase-it in the patch passed.
33m 7s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3932
Optional Testsjavac javadoc unit shadedjars compile
unameLinux c4b07340dcd5 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / b5cf3cd
Default JavaAdoptOpenJDK-1.8.0_282-b08
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-jdk8-hadoop3-check/output/diff-compile-javac-hbase-it.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/testReport/
Max. process+thread count689 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Dec 10, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 24sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall5m 13smaster passed
+1 💚compile0m 49smaster passed
+1 💚checkstyle0m 22smaster passed
+1 💚spotbugs0m 46smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 43sthe patch passed
+1 💚compile0m 50sthe patch passed
-0 ⚠️javac0m 50shbase-it generated 2 new + 101 unchanged - 2 fixed = 103 total (was 103)
-0 ⚠️checkstyle0m 18shbase-it: The patch generated 3 new + 19 unchanged - 24 fixed = 22 total (was 43)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck24m 47sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs0m 51sthe patch passed
_ Other Tests _
+1 💚asflicense0m 14sThe patch does not generate ASF License warnings.
50m 22s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3932
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 67f2b61e44f1 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / b5cf3cd
Default JavaAdoptOpenJDK-1.8.0_282-b08
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-it.txt
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

taklwu
taklwu reviewed Dec 10, 2021
View reviewed changes

@taklwutaklwu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one nit comment for javac and checkstyle

Comment threadhbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java
Comment threadhbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java
Comment thread...java/org/apache/hadoop/hbase/chaos/factories/ConfigurableSlowDeterministicMonkeyFactory.java Outdated
@Override
protected Action[] getHeavyWeightedActions() {
String actions = this.properties.getProperty(HEAVY_ACTIONS);
if( actions==null||actions.isEmpty() ){

@taklwutaklwuDec 10, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] javac and checkstyle seems failing, please see if you can fix it.

@wchevreuilwchevreuilDec 14, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack.

joshelser
joshelser approved these changes Dec 10, 2021
View reviewed changes

@joshelserjoshelser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling errors to fix.

Could you also consider adding some documentation as to how one would use the new policy? https://hbase.apache.org/book.html#maven.build.commands.integration.tests.destructive Ok to me if that is done separately

Comment thread...it/src/test/java/org/apache/hadoop/hbase/chaos/factories/SlowDeterministicMonkeyFactory.java Outdated
private long rollingBatchSuspendRSSleepTime;
private float rollingBatchSuspendtRSRatio;

protected Action[] getLightWeigthedActions(){

@joshelserjoshelserDec 10, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
protectedAction[] getLightWeigthedActions(){
protectedAction[] getLightWeightedActions(){

@wchevreuilwchevreuilDec 14, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack.

Comment thread...it/src/test/java/org/apache/hadoop/hbase/chaos/factories/SlowDeterministicMonkeyFactory.java Outdated
Comment thread...java/org/apache/hadoop/hbase/chaos/factories/ConfigurableSlowDeterministicMonkeyFactory.java
Wellington Chevreuil
addressing review comments
2ddf62f
Change-Id: If75dc3fa225cf9359e92bac0579bf46cbc3cf2ca
@Apache-HBase

Apache-HBase commented Dec 14, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 29sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 59smaster passed
+1 💚compile0m 33smaster passed
+1 💚shadedjars8m 48sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 22smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 30sthe patch passed
+1 💚compile0m 32sthe patch passed
-0 ⚠️javac0m 32shbase-it generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚shadedjars8m 24spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 18sthe patch passed
_ Other Tests _
+1 💚unit0m 57shbase-it in the patch passed.
31m 12s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3932
Optional Testsjavac javadoc unit shadedjars compile
unameLinux bd6f1c16bab5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 13f3b17
Default JavaAdoptOpenJDK-11.0.10+9
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/artifact/yetus-jdk11-hadoop3-check/output/diff-compile-javac-hbase-it.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/testReport/
Max. process+thread count677 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Dec 14, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 16sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 34smaster passed
+1 💚compile0m 27smaster passed
+1 💚shadedjars9m 4sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 18smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 18sthe patch passed
+1 💚compile0m 28sthe patch passed
-0 ⚠️javac0m 28shbase-it generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚shadedjars9m 5spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 15sthe patch passed
_ Other Tests _
+1 💚unit0m 57shbase-it in the patch passed.
31m 59s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3932
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 9dafd1d08aab 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 13f3b17
Default JavaAdoptOpenJDK-1.8.0_282-b08
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/artifact/yetus-jdk8-hadoop3-check/output/diff-compile-javac-hbase-it.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/testReport/
Max. process+thread count638 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Dec 14, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 11smaster passed
+1 💚compile0m 44smaster passed
+1 💚checkstyle0m 21smaster passed
+1 💚spotbugs0m 37smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 51sthe patch passed
+1 💚compile0m 42sthe patch passed
-0 ⚠️javac0m 42shbase-it generated 2 new + 101 unchanged - 2 fixed = 103 total (was 103)
+1 💚checkstyle0m 17shbase-it: The patch generated 0 new + 19 unchanged - 24 fixed = 19 total (was 43)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 22sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs0m 45sthe patch passed
_ Other Tests _
+1 💚asflicense0m 15sThe patch does not generate ASF License warnings.
39m 51s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3932
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 49c990609756 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 13f3b17
Default JavaAdoptOpenJDK-1.8.0_282-b08
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt
Max. process+thread count96 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

joshelser
joshelser approved these changes Dec 14, 2021
View reviewed changes

@joshelserjoshelser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed after last update. Looks good to me. We can update docs with how to use in follow-on.

@wchevreuil

wchevreuil commented Dec 14, 2021

Copy link
Copy Markdown
ContributorAuthor

Re-reviewed after last update. Looks good to me. We can update docs with how to use in follow-on.

Thanks @joshelser . Yeah, was just waiting the qa run before replying to you, I will submit an update to docs in another PR.

@wchevreuil
wchevreuil merged commit a36d41a into apache:masterDec 14, 2021
wchevreuil added a commit that referenced this pull request Dec 14, 2021
@wchevreuil
HBASE-26556IT and Chaos Monkey improvements (#3932)
5defd8c
Signed-off-by: Josh Elser <elserj@apache.org>
Reviewed-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
(cherry picked from commit a36d41a)
wchevreuil added a commit that referenced this pull request Dec 14, 2021
@wchevreuil
HBASE-26556IT and Chaos Monkey improvements (#3932)
55fdff5
Signed-off-by: Josh Elser <elserj@apache.org>
Reviewed-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
(cherry picked from commit a36d41a)
wchevreuil added a commit that referenced this pull request Dec 14, 2021
@wchevreuil
HBASE-26556IT and Chaos Monkey improvements (#3932)
0866975
Signed-off-by: Josh Elser <elserj@apache.org>
Reviewed-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
(cherry picked from commit a36d41a)
wchevreuil added a commit that referenced this pull request Dec 14, 2021
@wchevreuil
HBASE-26556IT and Chaos Monkey improvements (#3932)
dc1f546
Signed-off-by: Josh Elser <elserj@apache.org>
Reviewed-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
(cherry picked from commit a36d41a)
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
@wchevreuil@petersomogyi
HBASE-26556 IT and Chaos Monkey improvements (apache#3932)
03cbae9
Signed-off-by: Josh Elser <elserj@apache.org>
Reviewed-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
(cherry picked from commit a36d41a)
(cherry picked from commit 0866975)
Change-Id: Ie8d9f995268233ec6a140860bf4a61300d656fba
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@taklwutaklwutaklwu left review comments
@joshelserjoshelserjoshelser approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@wchevreuil@Apache-HBase@joshelser@taklwu