Skip to content

HADOOP-19964. Restore TimedOutTestsListener thread dumps on test timeout. - #8682

Draft
joseluisll wants to merge 1 commit into
apache:trunkfrom
joseluisll:HADOOP-19964
Draft

HADOOP-19964. Restore TimedOutTestsListener thread dumps on test timeout.#8682
joseluisll wants to merge 1 commit into
apache:trunkfrom
joseluisll:HADOOP-19964

Conversation

@joseluisll

@joseluislljoseluisll commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Description of PR

JIRA: HADOOP-19964

GenericTestUtils.waitFor built a 16 KB thread dump into every
TimeoutException message, at 650 call sites. It now dumps to stderr the
moment the wait expires, while the threads are still hung, and the message
drops to one line. Sample failure report: 18,223 -> 1,642 bytes.

TimedOutTestsListener has registered nothing since HADOOP-19415 Part 4 moved
Surefire onto the JUnit Platform provider, which ignores the listener
property. It now implements TestExecutionListener, so the property the eight
poms already carry binds to it once the provider accepts platform listeners
(SUREFIRE-1639, apache/maven-surefire#3438). Dormant until then. No pom is
touched.

  • -Dhadoop.test.timedout.dump=false disables dumps.
  • -Dhadoop.test.timedout.dump.limit (default 5) caps them per JVM. Both entry
    points share the budget.
  • The message records "Thread dump printed to stderr." only when one was.

Not covered: a fork killed at forkedProcessTimeoutInSeconds (Shutdown.KILL
-> Runtime.halt()). HADOOP-19950 captures those.

Release note: waitFor's TimeoutException message no longer carries the
thread dump. Nothing in the Hadoop tree asserted on it; downstream code that
does will need adjusting.

Contains content generated by Claude Code.

How was this patch tested?

mvn test -pl hadoop-common-project/hadoop-common -Dtest='TestTimedOutTestsListener,TestGenericTestUtils'
— 15 tests, all pass.

TestTimedOutTestsListener covers deadlock detection, timeout-failure
detection, the off switch, the dump limit and its elision notice,
dumpForTimeout's return value, and waitFor's exact message with a dump
printed, with dumps off, and with the budget spent — driving the real
waitFor. Checkstyle clean on both listener files; the two violations the 2012
test file carried are fixed.

For code changes:

  • Does the title of this PR start with the corresponding JIRA issue id?
  • Object storage: integration tests — N/A
  • New dependencies licensed compatibly with ASF 2.0 — N/A, none added
  • LICENSE, LICENSE-binary, NOTICE-binary updated — N/A

AI Tooling

@joseluisll

Copy link
Copy Markdown
ContributorAuthor

Cross-reference: this pairs with #8659 (HDFS-17957), which sets a default
junit.jupiter.execution.timeout.default on the hdfs modules.
The two cover different halves of the same problem. This PR restores the thread
dump for tests that fail through JUnit, and explicitly cannot cover a fork
killed at forkedProcessTimeoutInSeconds: Surefire sends the fork
Shutdown.KILL, the fork executes Runtime.halt(), and that bypasses
listeners and shutdown hooks alike. #8659 is what moves hung hdfs tests off
that path — with a default @Timeout they fail through JUnit instead, at which
point this listener produces the dump. Neither PR alone yields a usable
diagnostic for a hung HDFS test.
The two touch one file in common, hadoop-hdfs-project/hadoop-hdfs/pom.xml:
#8659 adds inside <systemPropertyVariables>, this one removes the dead
<properties> listener block just after it. I trial-merged the two heads and
the result is clean, with both changes intact, in either order — no rebase
needed whichever lands first.

Comment threadhadoop-common-project/hadoop-common/pom.xml
@pan3793

Copy link
Copy Markdown
Member

The AI-generated PR description is too verbose

GenericTestUtils.waitFor dumps all threads with deadlock analysis to stderr
at the moment the wait expires. Its TimeoutException message drops from
~16KB to one line. Sample failure report: 18,223 -> 1,642 bytes.
TimedOutTestsListener implements TestExecutionListener. The `listener`
provider property the eight poms already carry binds to it once the JUnit
Platform provider accepts platform listeners (SUREFIRE-1639,
apache/maven-surefire#3438). Dormant until then. No pom is touched.
- -Dhadoop.test.timedout.dump=false disables dumps.
- -Dhadoop.test.timedout.dump.limit (default 5) caps them per JVM, with one
elision notice. Both entry points share the budget.
- The exception records "Thread dump printed to stderr." only when a dump
was printed.
- The two checkstyle violations the 2012 test file carried are fixed.
Not covered: Surefire's fork kill at forkedProcessTimeoutInSeconds
(Shutdown.KILL -> Runtime.halt()). HADOOP-19950 captures those reports.
Release note: waitFor's TimeoutException message no longer carries the
thread dump.
Tests: TestTimedOutTestsListener covers timeout detection, the off switch,
the dump limit, dumpForTimeout's return value, and waitFor's exact message
in all three states — dump printed, dumps off, budget spent, driving the
real waitFor. TestGenericTestUtils passes unchanged.
Contains content generated by Claude Code.
@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec18m 28sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 3 new or modified test files.
_ trunk Compile Tests _
+1 💚mvninstall49m 7strunk passed
+1 💚compile18m 35strunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚compile18m 7strunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚checkstyle1m 29strunk passed
+1 💚mvnsite2m 0strunk passed
+1 💚javadoc1m 27strunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚javadoc1m 24strunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚spotbugs3m 12strunk passed
+1 💚shadedclient37m 2sbranch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚mvninstall1m 15sthe patch passed
+1 💚compile16m 51sthe patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚javac16m 51sthe patch passed
+1 💚compile18m 16sthe patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚javac18m 16sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle1m 23shadoop-common-project/hadoop-common: The patch generated 0 new + 10 unchanged - 3 fixed = 10 total (was 13)
+1 💚mvnsite1m 57sthe patch passed
+1 💚javadoc1m 20sthe patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚javadoc1m 18sthe patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚spotbugs3m 22sthe patch passed
+1 💚shadedclient37m 42spatch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚unit23m 26shadoop-common in the patch passed.
+1 💚asflicense1m 15sThe patch does not generate ASF License warnings.
260m 15s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/3/artifact/out/Dockerfile
GITHUB PR#8682
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
unameLinux 0d6d2dafd7bb 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/bin/hadoop.sh
git revisiontrunk / d2d6f0b
Default JavaUbuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Resultshttps://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/3/testReport/
Max. process+thread count1557 (vs. ulimit of 10000)
modulesC: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console outputhttps://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/3/console
versionsgit=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered byApache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@pan3793

Copy link
Copy Markdown
Member

The Maven community has accepted SUREFIRE-1639, and it will be included in the 3.6.0-M2. Let me convert this to a draft first and wait for the release.

@pan3793
pan3793 marked this pull request as draft August 21, 2026 08:38
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@joseluisll@pan3793@hadoop-yetus