Skip to content

TEZ-4719: Move to Junit6 - #502

Merged
abstractdog merged 4 commits into
apache:masterfrom
Aggarwal-Raghav:TEZ-4719
Jul 2, 2026
Merged

TEZ-4719: Move to Junit6#502
abstractdog merged 4 commits into
apache:masterfrom
Aggarwal-Raghav:TEZ-4719

Conversation

@Aggarwal-Raghav

@Aggarwal-RaghavAggarwal-Raghav commented May 31, 2026

Copy link
Copy Markdown
Contributor
  1. Update Annotations:
@Test (Change import from org.junit.Test to org.junit.jupiter.api.Test)
@Before → @BeforeEach
@After → @AfterEach
@BeforeClass → @BeforeAll (Method must be static)
@AfterClass → @AfterAll (Method must be static)
@Ignore → @Disabled
  1. Update Assertions:
    org.junit.Assert.* → org.junit.jupiter.api.Assertions.*

  2. Removed the transitive dependency on Hamcrest matchers.

  3. Migrated all @Test(timeout = ...) parameters to the @Timeout annotation while maintianing the Millisecond timeunit

  4. Replaced @Rule TemporaryFolder with JUnit Jupiter's @TempDir

  5. Note: In JUnit 4, the custom assertion message was the first parameter. In JUnit 6, it is the last parameter.
    In JUnit 4: assertEquals("Values should match", expected, actual);
    In JUnit 6:assertEquals(expected, actual, "Values should match");

@tez-yetus

This comment was marked as outdated.

@Aggarwal-RaghavAggarwal-Raghav changed the title [WIP] TEZ-4719: Move to Junit6TEZ-4719: Move to Junit6Jun 1, 2026

import org.junit.Assert;
import org.junit.Test;
import org.junit.jupiter.api.*;

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which is the preferred way?

  1. Using single line separte import
  2. * imports

@tez-yetus

This comment was marked as outdated.

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

UT has passed, which is good thing, few indendations and java formatting is left that will be handled once final rebase with #505 is done.

@Aggarwal-Raghav
Aggarwal-Raghavforce-pushed the TEZ-4719 branch 3 times, most recently from 5fb400c to 5f401beCompareJune 23, 2026 18:42
@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec1m 58sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 2sNo case conflicting files found.
+0 🆗detsecrets0m 2sdetect-secrets was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 247 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep0m 39sMaven dependency ordering for branch
+1 💚mvninstall2m 51smaster passed
+1 💚compile2m 31smaster passed
+1 💚checkstyle6m 23smaster passed
+1 💚javadoc6m 33smaster passed
+0 🆗spotbugs0m 30shadoop-shim in master has 3 extant spotbugs warnings.
+0 🆗spotbugs0m 46stez-api in master has 92 extant spotbugs warnings.
+0 🆗spotbugs0m 30stez-common in master has 13 extant spotbugs warnings.
+0 🆗spotbugs0m 35stez-runtime-internals in master has 92 extant spotbugs warnings.
+0 🆗spotbugs0m 44stez-runtime-library in master has 143 extant spotbugs warnings.
+0 🆗spotbugs0m 36stez-mapreduce in master has 97 extant spotbugs warnings.
+0 🆗spotbugs0m 27stez-examples in master has 2 extant spotbugs warnings.
+0 🆗spotbugs0m 56stez-dag in master has 537 extant spotbugs warnings.
+0 🆗spotbugs0m 29stez-tests in master has 6 extant spotbugs warnings.
+0 🆗spotbugs0m 30stez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗spotbugs0m 28stez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗spotbugs0m 29stez-plugins/tez-yarn-timeline-history-with-acls in master has 2 extant spotbugs warnings.
+0 🆗spotbugs0m 23stez-plugins/tez-yarn-timeline-cache-plugin in master has 1 extant spotbugs warnings.
+0 🆗spotbugs0m 28stez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗spotbugs0m 30stez-plugins/tez-history-parser in master has 21 extant spotbugs warnings.
+0 🆗spotbugs0m 32stez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗spotbugs3m 41sroot in master has 1054 extant spotbugs warnings.
+0 🆗spotbugs0m 31stez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall2m 15sthe patch passed
+1 💚codespell0m 49sNo new issues.
+1 💚compile2m 34sthe patch passed
-1 ❌javac2m 34s/results-compile-javac-root.txtroot generated 22 new + 684 unchanged - 23 fixed = 706 total (was 707)
+1 💚blanks0m 0sThe patch has no blanks issues.
-0 ⚠️checkstyle0m 21s/results-checkstyle-tez-api.txttez-api: The patch generated 5 new + 149 unchanged - 8 fixed = 154 total (was 157)
-0 ⚠️checkstyle0m 25s/results-checkstyle-tez-runtime-library.txttez-runtime-library: The patch generated 234 new + 606 unchanged - 53 fixed = 840 total (was 659)
-0 ⚠️checkstyle0m 20s/results-checkstyle-tez-mapreduce.txttez-mapreduce: The patch generated 1 new + 119 unchanged - 3 fixed = 120 total (was 122)
-0 ⚠️checkstyle0m 27s/results-checkstyle-tez-dag.txttez-dag: The patch generated 63 new + 936 unchanged - 40 fixed = 999 total (was 976)
-0 ⚠️checkstyle0m 20s/results-checkstyle-tez-tests.txttez-tests: The patch generated 31 new + 182 unchanged - 22 fixed = 213 total (was 204)
-0 ⚠️checkstyle0m 17s/results-checkstyle-tez-plugins_tez-protobuf-history-plugin.txttez-plugins/tez-protobuf-history-plugin: The patch generated 8 new + 17 unchanged - 6 fixed = 25 total (was 23)
-0 ⚠️checkstyle0m 16s/results-checkstyle-tez-plugins_tez-aux-services.txttez-plugins/tez-aux-services: The patch generated 4 new + 49 unchanged - 6 fixed = 53 total (was 55)
-0 ⚠️checkstyle0m 48s/results-checkstyle-root.txtroot: The patch generated 346 new + 2307 unchanged - 138 fixed = 2653 total (was 2445)
+1 💚javadoc6m 32sthe patch passed
+1 💚spotbugs16m 25sthe patch passed
_ Other Tests _
+1 💚unit62m 11sroot in the patch passed.
+1 💚asflicense6m 5sThe patch does not generate ASF License warnings.
145m 1s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/7/artifact/out/Dockerfile
Optional Testsdupname compile unit asflicense codespell detsecrets javac javadoc xmllint spotbugs checkstyle
unameLinux 1c89ba3c7658 5.15.0-179-generic #189-Ubuntu SMP Tue May 5 18:20:56 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitytez-personality.sh
git revisionmaster / 17d1549
Default JavaEclipse Adoptium-21.0.11+10-LTS
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/7/testReport/
Max. process+thread count1546 (vs. ulimit of 5500)
modulesC: hadoop-shim tez-api tez-common tez-runtime-internals tez-runtime-library tez-mapreduce tez-examples tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-cache-plugin tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser tez-plugins/tez-aux-services tez-tools/analyzers/job-analyzer . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/7/console
versionsgit=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

@abstractdog , this is ready for review.

@abstractdogabstractdog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is huge so far, thanks @Aggarwal-Raghav ! left some comments

Comment on lines +128 to +132
Environment.PWD.$()
+ File.pathSeparator
+ "USER_PATH"
+ File.pathSeparator
+ "DEFAULT_PATH",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a matter of taste, but I don't think indenting a string concatenation this way makes the code better, a single line is more compact and still pretty much readable

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was using google java formatter in intelliJ, now I have moved to tez project checkstyle present at tez-build-tools/src/main/resources/checkstyle/checkstyle.xml and updated the whole PR manually file by file. Fixed line length >120 in this PR. I hope it should help in checkstyle/spotbugs as well.

@abstractdogabstractdogJul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for clarifying!
I believe we can start soon a community discussion about which formatter to accept, and then we can enforce it by spotless: that's going to be another huge patch, not a problem :)

Comment threadtez-common/src/test/java/org/apache/tez/common/MockDNSToSwitchMapping.java Outdated
Comment threadpom.xml Outdated
Comment threadpom.xml
Comment threadtez-tests/src/test/java/org/apache/tez/test/TestDriver.java Outdated
Comment threadtez-tests/src/test/java/org/apache/tez/test/TestSecureShuffle.java Outdated
Comment threadtez-tests/src/test/java/org/apache/tez/test/TestSecureShuffle.java Outdated
Comment threadtez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestDAGImpl.java Outdated
@Aggarwal-Raghav
Aggarwal-Raghavforce-pushed the TEZ-4719 branch 2 times, most recently from dd8e0a0 to 6374968CompareJune 30, 2026 19:25
@tez-yetus

This comment was marked as outdated.

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

CI has failed because of some artifactory issue, have re-triggered it

@tez-yetus

This comment was marked as outdated.

@Aggarwal-Raghav

Aggarwal-Raghav commented Jul 1, 2026

Copy link
Copy Markdown
ContributorAuthor

Giving 1 more try, there is some infra issue i guess, puppycrawler is going OOM, and different issue in checkstyle , sporbugs. then will play aroung Jenkinsfile. I know there are few checkstyle issues still because of this PR, will handle once CI start passing.

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

Failing with OOM.

13:05:14 [INFO] Running org.apache.tez.common.TestTezCommonUtils
13:05:24 [ERROR] java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

@abstractdog , is it because of some jenkins node problem?
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/09, UT never ran — Some other failure
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/10, UT passed — Ran on hadoop14 node — Still other error are present
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/11, UT OOM — Ran on hadoop52 node — UT failed

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

rebasing as there are merge conflict. The older run is stuck, neither failing nor proceeding.

@abstractdog

Copy link
Copy Markdown
Contributor

rebasing as there are merge conflict. The older run is stuck, neither failing nor proceeding.

yeah, please be aware of a recent unit test, which might need to be migrated: 8fde353

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

rebasing as there are merge conflict. The older run is stuck, neither failing nor proceeding.

yeah, please be aware of a recent unit test, which might need to be migrated: 8fde353

yes, doing that only :-)

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

For TestTezCommonUtils where OOM happened in last run, testMkDirForAM, i optimized that particular test as MiniDFSCluster was already created in BeforeAll setup

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 12sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 3sNo case conflicting files found.
+0 🆗detsecrets0m 3sdetect-secrets was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 224 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep0m 46sMaven dependency ordering for branch
+1 💚mvninstall4m 31smaster passed
+1 💚compile4m 15smaster passed
+1 💚checkstyle9m 25smaster passed
+1 💚javadoc9m 23smaster passed
+0 🆗spotbugs0m 39shadoop-shim in master has 3 extant spotbugs warnings.
+0 🆗spotbugs1m 18stez-api in master has 92 extant spotbugs warnings.
+0 🆗spotbugs0m 44stez-common in master has 13 extant spotbugs warnings.
+0 🆗spotbugs0m 51stez-runtime-internals in master has 92 extant spotbugs warnings.
+0 🆗spotbugs1m 11stez-runtime-library in master has 143 extant spotbugs warnings.
+0 🆗spotbugs0m 53stez-mapreduce in master has 97 extant spotbugs warnings.
+0 🆗spotbugs0m 43stez-examples in master has 2 extant spotbugs warnings.
+0 🆗spotbugs1m 37stez-dag in master has 537 extant spotbugs warnings.
+0 🆗spotbugs0m 49stez-tests in master has 6 extant spotbugs warnings.
+0 🆗spotbugs0m 47stez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗spotbugs0m 44stez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗spotbugs0m 40stez-plugins/tez-yarn-timeline-history-with-acls in master has 2 extant spotbugs warnings.
+0 🆗spotbugs0m 43stez-plugins/tez-yarn-timeline-cache-plugin in master has 1 extant spotbugs warnings.
+0 🆗spotbugs0m 45stez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗spotbugs0m 46stez-plugins/tez-history-parser in master has 21 extant spotbugs warnings.
+0 🆗spotbugs0m 46stez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗spotbugs6m 26sroot in master has 1054 extant spotbugs warnings.
+0 🆗spotbugs0m 46stez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for patch
+1 💚mvninstall4m 1sthe patch passed
+1 💚codespell1m 42sNo new issues.
+1 💚compile4m 11sthe patch passed
-1 ❌javac4m 11s/results-compile-javac-root.txtroot generated 24 new + 682 unchanged - 25 fixed = 706 total (was 707)
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 22sThe patch passed checkstyle in hadoop-shim
-0 ⚠️checkstyle0m 27s/results-checkstyle-tez-api.txttez-api: The patch generated 5 new + 144 unchanged - 13 fixed = 149 total (was 157)
+1 💚checkstyle0m 22sThe patch passed checkstyle in tez-common
+1 💚checkstyle0m 24sThe patch passed checkstyle in tez-runtime-internals
-0 ⚠️checkstyle0m 35s/results-checkstyle-tez-runtime-library.txttez-runtime-library: The patch generated 165 new + 596 unchanged - 57 fixed = 761 total (was 653)
-0 ⚠️checkstyle0m 26s/results-checkstyle-tez-mapreduce.txttez-mapreduce: The patch generated 1 new + 117 unchanged - 4 fixed = 118 total (was 121)
+1 💚checkstyle0m 24sThe patch passed checkstyle in tez-examples
-0 ⚠️checkstyle0m 43s/results-checkstyle-tez-dag.txttez-dag: The patch generated 42 new + 919 unchanged - 42 fixed = 961 total (was 961)
-0 ⚠️checkstyle0m 28s/results-checkstyle-tez-tests.txttez-tests: The patch generated 9 new + 139 unchanged - 24 fixed = 148 total (was 163)
+1 💚checkstyle0m 25sThe patch passed checkstyle in tez-ext-service-tests
-0 ⚠️checkstyle0m 24s/results-checkstyle-tez-plugins_tez-protobuf-history-plugin.txttez-plugins/tez-protobuf-history-plugin: The patch generated 8 new + 17 unchanged - 6 fixed = 25 total (was 23)
+1 💚checkstyle0m 24sThe patch passed checkstyle in tez-yarn-timeline-history
+1 💚checkstyle0m 24sThe patch passed checkstyle in tez-yarn-timeline-history-with-acls
+1 💚checkstyle0m 23sThe patch passed checkstyle in tez-yarn-timeline-cache-plugin
+1 💚checkstyle0m 25sThe patch passed checkstyle in tez-yarn-timeline-history-with-fs
+1 💚checkstyle0m 26sThe patch passed checkstyle in tez-history-parser
+1 💚checkstyle0m 23stez-plugins/tez-aux-services: The patch generated 0 new + 49 unchanged - 6 fixed = 49 total (was 55)
+1 💚checkstyle0m 25sThe patch passed checkstyle in job-analyzer
-0 ⚠️checkstyle1m 14s/results-checkstyle-root.txtroot: The patch generated 230 new + 2220 unchanged - 152 fixed = 2450 total (was 2372)
+1 💚javadoc9m 16sthe patch passed
+1 💚spotbugs26m 32sthe patch passed
_ Other Tests _
+1 💚unit76m 33sroot in the patch passed.
+1 💚asflicense7m 43sThe patch does not generate ASF License warnings.
200m 19s
SubsystemReport/Notes
DockerClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/12/artifact/out/Dockerfile
Optional Testsdupname compile unit asflicense codespell detsecrets javac javadoc xmllint spotbugs checkstyle
unameLinux c32fbec5b6b3 5.15.0-177-generic #187-Ubuntu SMP Sat Apr 11 22:54:33 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitytez-personality.sh
git revisionmaster / 8fde353
Default JavaEclipse Adoptium-21.0.11+10-LTS
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/12/testReport/
Max. process+thread count2113 (vs. ulimit of 5500)
modulesC: hadoop-shim tez-api tez-common tez-runtime-internals tez-runtime-library tez-mapreduce tez-examples tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-cache-plugin tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser tez-plugins/tez-aux-services tez-tools/analyzers/job-analyzer . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-502/12/console
versionsgit=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@Aggarwal-Raghav

Copy link
Copy Markdown
ContributorAuthor

CI has passed @abstractdog , I think the checkstyle especially in javac can be handled separately?

@abstractdog

Copy link
Copy Markdown
Contributor

CI has passed @abstractdog , I think the checkstyle especially in javac can be handled separately?

agree! those problems were not introduced by this patch, and this is already huge, it's worth moving on before further conflicts :)

@abstractdog
abstractdog self-requested a review July 2, 2026 07:10

@abstractdogabstractdog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@abstractdog
abstractdog merged commit d53e67d into apache:masterJul 2, 2026
6 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Aggarwal-Raghav@tez-yetus@abstractdog