Skip to content

TEZ-4344: Collect jstack periodically from all containers. - #299

Merged
abstractdog merged 12 commits into
apache:masterfrom
ayushtkn:TEZ-4344
Jul 18, 2023
Merged

TEZ-4344: Collect jstack periodically from all containers.#299
abstractdog merged 12 commits into
apache:masterfrom
ayushtkn:TEZ-4344

Conversation

@ayushtkn

Copy link
Copy Markdown
Member

SS:
image
Jstack:
image

@abstractdog

Copy link
Copy Markdown
Contributor

thanks for the initial patch @ayushtkn! can you take care of the same in the DAGAppMaster too?

@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

Comment threadtez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java Outdated
Comment threadtez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java Outdated
Comment threadtez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java Outdated
Change-Id: I8d8b13c03a76a61c053b851743c1d2426e2fdd7b
@tez-yetus

This comment was marked as outdated.

Change-Id: I0353a0dc983a5c02b7bf183d1d5541f15af388dd
Change-Id: Ib0167ef6b52faebc02876ab895f69d8c592a0aca
@tez-yetus

This comment was marked as outdated.

Change-Id: I3d484a6dcadd8f8b880cceb12cae48e22b2580a7
@tez-yetus

This comment was marked as outdated.

Comment threadtez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java Outdated
Comment threadtez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java Outdated
Comment threadtez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java Outdated
Comment threadtez-tests/src/test/java/org/apache/tez/test/TestTezJobs.java Outdated
@abstractdog

Copy link
Copy Markdown
Contributor

thanks @ayushtkn, left some minor comments, this is quite close!
can you please try this on a cluster (e.g. with Hive on Tez) to see how it's working in production? we can discuss the details offline

Change-Id: Ie16c5b180d4b19fef43144f48c279e7382965476
Change-Id: I2be6bf61b3925f61a68891d722a830b988f77263
Change-Id: I9d88c928f6591f5590f4bbe601364b8eddf5d8b4
@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

Change-Id: I33993adb8ebd811dae1b3e3eed3d75641fae2956
@tez-yetus

This comment was marked as outdated.

Comment threadtez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java Outdated
Comment threadtez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java Outdated
Change-Id: I4ab43d923511668c8e51e8a571bda153b6ccf102
Change-Id: If88667b9e978b648c8d3e85535feeb24201d9879
@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

Change-Id: I0fdeea8971edce4ff2886c757a05306438b96893
@tez-yetus

This comment was marked as outdated.

Change-Id: I4bf6b2c82f6d1455f5f816a07dbcfcd80b603652
@tez-yetus

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 45sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep5m 43sMaven dependency ordering for branch
+1 💚mvninstall8m 22smaster passed
+1 💚compile2m 26smaster passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚compile2m 8smaster passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚checkstyle2m 4smaster passed
+1 💚javadoc1m 54smaster passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javadoc1m 40smaster passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+0 🆗spotbugs0m 47sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs4m 17smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 9sMaven dependency ordering for patch
+1 💚mvninstall1m 26sthe patch passed
+1 💚compile1m 36sthe patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javac1m 36sthe patch passed
+1 💚compile1m 21sthe patch passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚javac1m 21sthe patch passed
+1 💚checkstyle0m 12sThe patch passed checkstyle in tez-api
+1 💚checkstyle0m 9stez-runtime-internals: The patch generated 0 new + 7 unchanged - 1 fixed = 7 total (was 8)
+1 💚checkstyle0m 21sThe patch passed checkstyle in tez-dag
+1 💚checkstyle0m 12sThe patch passed checkstyle in tez-tests
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 54sthe patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javadoc0m 53sthe patch passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚findbugs3m 44sthe patch passed
_ Other Tests _
+1 💚unit2m 16stez-api in the patch passed.
+1 💚unit0m 37stez-runtime-internals in the patch passed.
+1 💚unit4m 56stez-dag in the patch passed.
+1 💚unit39m 3stez-tests in the patch passed.
+1 💚asflicense0m 45sThe patch does not generate ASF License warnings.
89m 56s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-299/14/artifact/out/Dockerfile
GITHUB PR#299
JIRA IssueTEZ-4344
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux 9af5727e3a09 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 5038075
Default JavaPrivate Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-299/14/testReport/
Max. process+thread count1194 (vs. ulimit of 5500)
modulesC: tez-api tez-runtime-internals tez-dag tez-tests U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-299/14/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdog
abstractdog self-requested a review July 18, 2023 06:07
@abstractdog
abstractdog merged commit a9c5365 into apache:masterJul 18, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@ayushtkn@abstractdog@tez-yetus