Skip to content

HBASE-27567 Introduce ChaosMonkey Action to print HDFS Cluster status - #4959

Merged
ndimiduk merged 3 commits into
apache:masterfrom
ndimiduk:27567-cm-action-dump-hdfs-status
Jan 16, 2023
Merged

HBASE-27567 Introduce ChaosMonkey Action to print HDFS Cluster status#4959
ndimiduk merged 3 commits into
apache:masterfrom
ndimiduk:27567-cm-action-dump-hdfs-status

Conversation

@ndimiduk

Copy link
Copy Markdown
Member

No description provided.

@Apache-HBase

This comment was marked as outdated.

@Apache-HBase

This comment was marked as outdated.

@Apache-HBase

This comment was marked as outdated.

@Apache-HBase

This comment was marked as outdated.

@Apache-HBase

This comment was marked as outdated.

@Apache-HBase

This comment was marked as outdated.


if (activeNamenode == null) {
throw new Exception("No active Name node found in zookeeper under " + hadoopHAZkNode);
} else {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why an else here? Seems not necessary,,,

}
// TODO: add more on NN, JNs, and ZK.
// TODO: Print how long process has been up.
LOG.info(sb.toString());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getLogger().info?

}

if (activeNamenode == null) {
throw new Exception("No active Name node found in zookeeper under " + hadoopHAZkNode);

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could just return instead of throwing.

@ndimiduk
ndimidukforce-pushed the 27567-cm-action-dump-hdfs-status branch from 8452ef8 to f8bbc92CompareJanuary 13, 2023 14:30
@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 42sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall2m 21smaster passed
+1 💚compile0m 14smaster passed
+1 💚shadedjars4m 1sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 10smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 7sthe patch passed
+1 💚compile0m 15sthe patch passed
+1 💚javac0m 15sthe patch passed
+1 💚shadedjars4m 5spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 11sthe patch passed
_ Other Tests _
+1 💚unit0m 37shbase-it in the patch passed.
15m 29s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4959/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4959
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 0fb6e65bd6bd 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 2a7c69d
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4959/3/testReport/
Max. process+thread count559 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4959/3/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 28sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 39smaster passed
+1 💚compile0m 23smaster passed
+1 💚shadedjars4m 47sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 15smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 0sthe patch passed
+1 💚compile0m 19sthe patch passed
+1 💚javac0m 19sthe patch passed
+1 💚shadedjars4m 13spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 13sthe patch passed
_ Other Tests _
+1 💚unit0m 45shbase-it in the patch passed.
19m 52s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4959/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4959
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 74a555a7426f 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 2a7c69d
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4959/3/testReport/
Max. process+thread count589 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4959/3/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 41sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall2m 58smaster passed
+1 💚compile0m 26smaster passed
+1 💚checkstyle0m 11smaster passed
+1 💚spotless0m 45sbranch has no errors when running spotless:check.
+1 💚spotbugs0m 25smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 36sthe patch passed
+1 💚compile0m 24sthe patch passed
+1 💚javac0m 24sthe patch passed
+1 💚checkstyle0m 9sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck10m 7sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 43spatch has no errors when running spotless:check.
+1 💚spotbugs0m 29sthe patch passed
_ Other Tests _
+1 💚asflicense0m 9sThe patch does not generate ASF License warnings.
26m 14s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4959/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4959
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux 13339363f0e7 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 2a7c69d
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count80 (vs. ulimit of 30000)
modulesC: hbase-it U: hbase-it
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4959/3/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@ndimiduk

Copy link
Copy Markdown
MemberAuthor

Do you have any further concerns @Apache9 ?

@Apache9Apache9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering what if we run ITs against S3AFileSystem, but since in ServerAndDependenciesKillingMonkeyFactory, we have already add RestartRandomDataNodeAction which is also HDFS specific, the PR here does not introduce new problems.

@ndimiduk

Copy link
Copy Markdown
MemberAuthor

We could run Chaos tests against a cluster configured with S3A storage, but in that case we would not run the HDFS chaos components.

I'd like to have a declarative configuration-driven chaos definition, instead of using these hard-coded factories, but I don't think that's been implemented.

@ndimiduk
ndimiduk merged commit 7ed2cb9 into apache:masterJan 16, 2023
@ndimiduk
ndimiduk deleted the 27567-cm-action-dump-hdfs-status branch January 16, 2023 13:40
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@ndimiduk@Apache-HBase@Apache9@Reidddddd