Skip to content

TEZ-4646: Periodic jstack collection for tez (tez.thread.dump.interval) only collects jstacks once. - #425

Merged
ayushtkn merged 2 commits into
apache:masterfrom
ayushtkn:TEZ-4646
Sep 2, 2025
Merged

TEZ-4646: Periodic jstack collection for tez (tez.thread.dump.interval) only collects jstacks once.#425
ayushtkn merged 2 commits into
apache:masterfrom
ayushtkn:TEZ-4646

Conversation

@ayushtkn

Copy link
Copy Markdown
Member

Added an option for initial delay & Fixed the periodic schedule so multiple jstacks are captured.
image

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec32m 11sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep1m 59sMaven dependency ordering for branch
+1 💚mvninstall10m 39smaster passed
+1 💚compile1m 42smaster passed
+1 💚checkstyle1m 24smaster passed
+1 💚javadoc1m 32smaster passed
+0 🆗spotbugs2m 16stez-api in master has 610 extant spotbugs warnings.
+0 🆗spotbugs0m 54stez-runtime-internals in master has 114 extant spotbugs warnings.
+0 🆗spotbugs0m 52stez-tests in master has 6 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗mvndep0m 8sMaven dependency ordering for patch
+1 💚mvninstall1m 10sthe patch passed
+1 💚codespell0m 31sNo new issues.
+1 💚compile1m 8sthe patch passed
+1 💚javac1m 8sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 30sthe patch passed
+1 💚javadoc0m 48sthe patch passed
-1 ❌spotbugs0m 59s/new-spotbugs-tez-runtime-internals.htmltez-runtime-internals generated 1 new + 113 unchanged - 1 fixed = 114 total (was 114)
_ Other Tests _
+1 💚unit2m 19stez-api in the patch passed.
+1 💚unit0m 52stez-runtime-internals in the patch passed.
+1 💚unit41m 44stez-tests in the patch passed.
+1 💚asflicense0m 32sThe patch does not generate ASF License warnings.
108m 41s
ReasonTests
SpotBugsmodule:tez-runtime-internals
Exception thrown in class org.apache.tez.runtime.TezThreadDumpHelper at new org.apache.tez.runtime.TezThreadDumpHelper(long, long, Configuration) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At TezThreadDumpHelper.java:at new org.apache.tez.runtime.TezThreadDumpHelper(long, long, Configuration) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At TezThreadDumpHelper.java:[line 71]
SubsystemReport/Notes
DockerClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-425/1/artifact/out/Dockerfile
GITHUB PR#425
Optional Testsdupname asflicense javac javadoc unit spotbugs checkstyle codespell detsecrets compile
unameLinux 0334a9746d14 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/tez-multibranch_PR-425/src/.yetus/personality.sh
git revisionmaster / 785bf8a
Default JavaUbuntu-21.0.8+9-Ubuntu-0ubuntu124.04.1
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-425/1/testReport/
Max. process+thread count974 (vs. ulimit of 5500)
modulesC: tez-api tez-runtime-internals tez-tests U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-425/1/console
versionsgit=2.43.0 maven=3.8.7 spotbugs=4.9.3 codespell=2.0.0
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@ayushtkn

Copy link
Copy Markdown
MemberAuthor

the spotbug warning isn't induced in this PR, flagging the already existing one. the count stays the same 114 (was 114)


try {
return new TezThreadDumpHelper(periodicThreadDumpFrequency, conf);
return new TezThreadDumpHelper(periodicThreadDumpFrequency, initialDelay, conf);

@abstractdogabstractdogSep 2, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about harmonizing the variable names from periodicThreadDumpFrequency + initialDelay to something like: threadDumpFrequency + threadDumpInitialDelay or even frequency + initialDelay to reflect that these variables will configure the very-same executor

also, if it's called something frequency at some point, let it be called the same in the function (instead of duration)

@abstractdog

Copy link
Copy Markdown
Contributor

looks good to me @ayushtkn, minor comment about naming

@abstractdogabstractdog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 40sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep1m 54sMaven dependency ordering for branch
+1 💚mvninstall10m 35smaster passed
+1 💚compile1m 51smaster passed
+1 💚checkstyle1m 32smaster passed
+1 💚javadoc1m 43smaster passed
+0 🆗spotbugs2m 37stez-api in master has 610 extant spotbugs warnings.
+0 🆗spotbugs1m 11stez-runtime-internals in master has 114 extant spotbugs warnings.
+0 🆗spotbugs0m 57stez-tests in master has 6 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗mvndep0m 9sMaven dependency ordering for patch
+1 💚mvninstall1m 21sthe patch passed
+1 💚codespell0m 36sNo new issues.
+1 💚compile1m 20sthe patch passed
+1 💚javac1m 20sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 36sthe patch passed
+1 💚javadoc0m 57sthe patch passed
-1 ❌spotbugs1m 8s/new-spotbugs-tez-runtime-internals.htmltez-runtime-internals generated 1 new + 113 unchanged - 1 fixed = 114 total (was 114)
_ Other Tests _
+1 💚unit2m 19stez-api in the patch passed.
+1 💚unit0m 53stez-runtime-internals in the patch passed.
+1 💚unit43m 41stez-tests in the patch passed.
+1 💚asflicense0m 41sThe patch does not generate ASF License warnings.
81m 43s
ReasonTests
SpotBugsmodule:tez-runtime-internals
Exception thrown in class org.apache.tez.runtime.TezThreadDumpHelper at new org.apache.tez.runtime.TezThreadDumpHelper(long, long, Configuration) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At TezThreadDumpHelper.java:at new org.apache.tez.runtime.TezThreadDumpHelper(long, long, Configuration) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At TezThreadDumpHelper.java:[line 72]
SubsystemReport/Notes
DockerClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-425/2/artifact/out/Dockerfile
GITHUB PR#425
Optional Testsdupname asflicense javac javadoc unit spotbugs checkstyle codespell detsecrets compile
unameLinux a5bf8f54ab2a 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/tez-multibranch_PR-425/src/.yetus/personality.sh
git revisionmaster / cc8f8fb
Default JavaUbuntu-21.0.8+9-Ubuntu-0ubuntu124.04.1
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-425/2/testReport/
Max. process+thread count1039 (vs. ulimit of 5500)
modulesC: tez-api tez-runtime-internals tez-tests U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-425/2/console
versionsgit=2.43.0 maven=3.8.7 spotbugs=4.9.3 codespell=2.0.0
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@ayushtkn
ayushtkn merged commit 6683866 into apache:masterSep 2, 2025
4 checks passed
Asmoday pushed a commit to arenadata/tez that referenced this pull request Apr 20, 2026
…al) only collects jstacks once. (apache#425). (Ayush Saxena, reviewed by Laszlo Bodor)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@ayushtkn@tez-yetus@abstractdog