TEZ-4721: Remove legacy Hadoop 2.x support and hadoop-shim-impls module by Aggarwal-Raghav · Pull Request #505 · apache/tez

Aggarwal-Raghav · 2026-06-02T17:03:18Z

Drop Hadoop 2.x Support by eliminate Hadoop Shim Impl module
Removed hadoop27, hadoop28 profiles

tez-yetus · 2026-06-02T21:20:32Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	10m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 32 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 42s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 35s		master passed
+1 💚	compile	4m 1s		master passed
+1 💚	checkstyle	10m 42s		master passed
-1 ❌	mvnsite	0m 43s	/branch-mvnsite-tez-api.txt	tez-api in master failed.
-1 ❌	mvnsite	0m 40s	/branch-mvnsite-tez-runtime-library.txt	tez-runtime-library in master failed.
-1 ❌	mvnsite	0m 37s	/branch-mvnsite-tez-mapreduce.txt	tez-mapreduce in master failed.
-1 ❌	mvnsite	1m 50s	/branch-mvnsite-root.txt	root in master failed.
-1 ❌	mvnsite	0m 27s	/branch-mvnsite-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
-1 ❌	javadoc	0m 23s	/branch-javadoc-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
+0 🆗	spotbugs	0m 41s		hadoop-shim in master has 3 extant spotbugs warnings.
+0 🆗	spotbugs	1m 18s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	0m 48s		tez-common in master has 13 extant spotbugs warnings.
+0 🆗	spotbugs	0m 54s		tez-runtime-internals in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 10s		tez-runtime-library in master has 143 extant spotbugs warnings.
+0 🆗	spotbugs	0m 55s		tez-mapreduce in master has 97 extant spotbugs warnings.
+0 🆗	spotbugs	0m 45s		tez-examples in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	1m 39s		tez-dag in master has 537 extant spotbugs warnings.
+0 🆗	spotbugs	0m 54s		tez-tests in master has 6 extant spotbugs warnings.
+0 🆗	spotbugs	1m 51s		tez-plugins in master has 43 extant spotbugs warnings.
+0 🆗	spotbugs	0m 47s		tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗	spotbugs	0m 42s		tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 41s		tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	0m 50s		tez-tools/analyzers in master has 20 extant spotbugs warnings.
+0 🆗	spotbugs	0m 28s		branch/docs no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	6m 26s		root in master has 1054 extant spotbugs warnings.
-1 ❌	spotbugs	0m 24s	/branch-spotbugs-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
+0 🆗	spotbugs	0m 28s		branch/tez-dist no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	0m 45s		tez-tools/analyzers/job-analyzer in master has 20 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 43s		the patch passed
+1 💚	codespell	1m 34s		The patch generated 0 new + 29 unchanged - 1 fixed = 29 total (was 30)
+1 💚	compile	3m 49s		the patch passed
+1 💚	javac	3m 49s		root generated 0 new + 741 unchanged - 1 fixed = 741 total (was 742)
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	checkstyle	0m 30s		The patch passed checkstyle in tez-api
-0 ⚠️	checkstyle	0m 26s	/results-checkstyle-tez-common.txt	tez-common: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	checkstyle	0m 28s		The patch passed checkstyle in tez-runtime-internals
-0 ⚠️	checkstyle	0m 33s	/results-checkstyle-tez-runtime-library.txt	tez-runtime-library: The patch generated 1 new + 88 unchanged - 0 fixed = 89 total (was 88)
+1 💚	checkstyle	0m 29s		The patch passed checkstyle in tez-mapreduce
+1 💚	checkstyle	0m 26s		The patch passed checkstyle in tez-examples
+1 💚	checkstyle	0m 44s		tez-dag: The patch generated 0 new + 1138 unchanged - 3 fixed = 1138 total (was 1141)
+1 💚	checkstyle	0m 28s		The patch passed checkstyle in tez-tests
+1 💚	checkstyle	0m 26s		The patch passed checkstyle in tez-ext-service-tests
+1 💚	checkstyle	0m 31s		The patch passed checkstyle in tez-plugins
+1 💚	checkstyle	0m 26s		The patch passed checkstyle in tez-protobuf-history-plugin
+1 💚	checkstyle	0m 26s		The patch passed checkstyle in tez-yarn-timeline-history
+1 💚	checkstyle	0m 27s		The patch passed checkstyle in tez-yarn-timeline-history-with-fs
+1 💚	checkstyle	0m 26s		The patch passed checkstyle in analyzers
+1 💚	checkstyle	0m 27s		The patch passed checkstyle in job-analyzer
+1 💚	checkstyle	0m 23s		The patch passed checkstyle in tez-dist
+1 💚	checkstyle	0m 21s		The patch passed checkstyle in docs
-0 ⚠️	checkstyle	1m 9s	/results-checkstyle-root.txt	root: The patch generated 2 new + 1459 unchanged - 17 fixed = 1461 total (was 1476)
-1 ❌	markdownlint	0m 7s	/results-markdownlint.txt	The patch generated 1 new + 117 unchanged - 2 fixed = 118 total (was 119)
-1 ❌	mvnsite	0m 43s	/patch-mvnsite-tez-api.txt	tez-api in the patch failed.
-1 ❌	mvnsite	0m 39s	/patch-mvnsite-tez-runtime-library.txt	tez-runtime-library in the patch failed.
-1 ❌	mvnsite	0m 34s	/patch-mvnsite-tez-mapreduce.txt	tez-mapreduce in the patch failed.
-1 ❌	mvnsite	1m 36s	/patch-mvnsite-root.txt	root in the patch failed.
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	javadoc	9m 13s		the patch passed
-1 ❌	spotbugs	1m 7s	/new-spotbugs-tez-runtime-internals.html	tez-runtime-internals generated 7 new + 82 unchanged - 10 fixed = 89 total (was 92)
-1 ❌	spotbugs	1m 54s	/new-spotbugs-tez-dag.html	tez-dag generated 3 new + 533 unchanged - 4 fixed = 536 total (was 537)
+0 🆗	spotbugs	0m 29s		tez-dist has no data from spotbugs
+0 🆗	spotbugs	0m 27s		docs has no data from spotbugs
-1 ❌	spotbugs	7m 7s	/new-spotbugs-root.html	root generated 10 new + 1037 unchanged - 17 fixed = 1047 total (was 1054)
			_ Other Tests _
+1 💚	unit	72m 48s		root in the patch passed.
+1 💚	asflicense	7m 48s		The patch does not generate ASF License warnings.
		256m 35s

Reason	Tests
SpotBugs	module:tez-runtime-internals
	Exception thrown in class org.apache.tez.runtime.task.TezChild at new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At TezChild.java:at new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At TezChild.java:[line 189]
	new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.credentials At TezChild.java:String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.credentials At TezChild.java:[line 149]
	new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.defaultConf At TezChild.java:String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.defaultConf At TezChild.java:[line 141]
	new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.objectRegistry At TezChild.java:String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.objectRegistry At TezChild.java:[line 173]
	new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.serviceProviderEnvMap At TezChild.java:String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.serviceProviderEnvMap At TezChild.java:[line 145]
	new org.apache.tez.runtime.task.TezTaskRunner2(Configuration, UserGroupInformation, String[], TaskSpec, int, Map, Map, Multimap, TaskReporterInterface, ExecutorService, ObjectRegistry, String, ExecutionContext, long, boolean, TezExecutors) may expose internal representation by storing an externally mutable object into TezTaskRunner2.taskReporter At TezTaskRunner2.java:int, Map, Map, Multimap, TaskReporterInterface, ExecutorService, ObjectRegistry, String, ExecutionContext, long, boolean, TezExecutors) may expose internal representation by storing an externally mutable object into TezTaskRunner2.taskReporter At TezTaskRunner2.java:[line 142]
	new org.apache.tez.runtime.task.TezTaskRunner2(Configuration, UserGroupInformation, String[], TaskSpec, int, Map, Map, Multimap, TaskReporterInterface, ExecutorService, ObjectRegistry, String, ExecutionContext, long, boolean, TezExecutors) may expose internal representation by storing an externally mutable object into TezTaskRunner2.ugi At TezTaskRunner2.java:int, Map, Map, Multimap, TaskReporterInterface, ExecutorService, ObjectRegistry, String, ExecutionContext, long, boolean, TezExecutors) may expose internal representation by storing an externally mutable object into TezTaskRunner2.ugi At TezTaskRunner2.java:[line 141]
SpotBugs	module:tez-dag
	new org.apache.tez.dag.app.rm.TaskSchedulerManager(AppContext, DAGClientServer, EventHandler, ContainerSignatureMatcher, WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.appContext At TaskSchedulerManager.java:WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.appContext At TaskSchedulerManager.java:[line 191]
	new org.apache.tez.dag.app.rm.TaskSchedulerManager(AppContext, DAGClientServer, EventHandler, ContainerSignatureMatcher, WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.clientService At TaskSchedulerManager.java:WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.clientService At TaskSchedulerManager.java:[line 193]
	new org.apache.tez.dag.app.rm.TaskSchedulerManager(AppContext, DAGClientServer, EventHandler, ContainerSignatureMatcher, WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.webUI At TaskSchedulerManager.java:WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.webUI At TaskSchedulerManager.java:[line 195]
SpotBugs	module:root
	new org.apache.tez.dag.app.rm.TaskSchedulerManager(AppContext, DAGClientServer, EventHandler, ContainerSignatureMatcher, WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.appContext At TaskSchedulerManager.java:WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.appContext At TaskSchedulerManager.java:[line 191]
	new org.apache.tez.dag.app.rm.TaskSchedulerManager(AppContext, DAGClientServer, EventHandler, ContainerSignatureMatcher, WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.clientService At TaskSchedulerManager.java:WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.clientService At TaskSchedulerManager.java:[line 193]
	new org.apache.tez.dag.app.rm.TaskSchedulerManager(AppContext, DAGClientServer, EventHandler, ContainerSignatureMatcher, WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.webUI At TaskSchedulerManager.java:WebUIService, List, boolean) may expose internal representation by storing an externally mutable object into TaskSchedulerManager.webUI At TaskSchedulerManager.java:[line 195]
	Exception thrown in class org.apache.tez.runtime.task.TezChild at new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At TezChild.java:at new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At TezChild.java:[line 189]
	new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.credentials At TezChild.java:String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.credentials At TezChild.java:[line 149]
	new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.defaultConf At TezChild.java:String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.defaultConf At TezChild.java:[line 141]
	new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.objectRegistry At TezChild.java:String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.objectRegistry At TezChild.java:[line 173]
	new org.apache.tez.runtime.task.TezChild(Configuration, String, int, String, String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.serviceProviderEnvMap At TezChild.java:String, int, String, String[], Map, ObjectRegistryImpl, String, ExecutionContext, Credentials, long, String, TezTaskUmbilicalProtocol, boolean) may expose internal representation by storing an externally mutable object into TezChild.serviceProviderEnvMap At TezChild.java:[line 145]
	new org.apache.tez.runtime.task.TezTaskRunner2(Configuration, UserGroupInformation, String[], TaskSpec, int, Map, Map, Multimap, TaskReporterInterface, ExecutorService, ObjectRegistry, String, ExecutionContext, long, boolean, TezExecutors) may expose internal representation by storing an externally mutable object into TezTaskRunner2.taskReporter At TezTaskRunner2.java:int, Map, Map, Multimap, TaskReporterInterface, ExecutorService, ObjectRegistry, String, ExecutionContext, long, boolean, TezExecutors) may expose internal representation by storing an externally mutable object into TezTaskRunner2.taskReporter At TezTaskRunner2.java:[line 142]
	new org.apache.tez.runtime.task.TezTaskRunner2(Configuration, UserGroupInformation, String[], TaskSpec, int, Map, Map, Multimap, TaskReporterInterface, ExecutorService, ObjectRegistry, String, ExecutionContext, long, boolean, TezExecutors) may expose internal representation by storing an externally mutable object into TezTaskRunner2.ugi At TezTaskRunner2.java:int, Map, Map, Multimap, TaskReporterInterface, ExecutorService, ObjectRegistry, String, ExecutionContext, long, boolean, TezExecutors) may expose internal representation by storing an externally mutable object into TezTaskRunner2.ugi At TezTaskRunner2.java:[line 141]

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/1/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense codespell detsecrets markdownlint mvnsite xmllint javac javadoc spotbugs checkstyle shellcheck shelldocs
uname	Linux 107462f69a36 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `330fdc8`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/1/testReport/
Max. process+thread count	2125 (vs. ulimit of 5500)
modules	C: tez-api tez-common tez-runtime-internals tez-runtime-library tez-mapreduce tez-examples tez-dag tez-tests tez-ext-service-tests tez-plugins tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-fs tez-tools/analyzers tez-tools/analyzers/job-analyzer tez-dist docs . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/1/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1 markdownlint=0.46.0 shellcheck=0.11.0
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

abstractdog · 2026-06-03T09:08:02Z

Drop Hadoop 2.x Support by eliminate Hadoop Shim Layer
Moved the Hadoop 2.8+ shims logic to TezUtilsInternal.java.
Removed hadoop27, hadoop28 profiles
Removed GuavaShim layer as well which was pre v18.0 of guava.

thanks for the patch @Aggarwal-Raghav so far!
dropping the whole shim looks strange for me, I thought this is only for dropping hadoop 2.x, but shims are more generic way to handle different hadoop versions, which can be useful in the future too, right? please let me know if I miss something

the same applies to GuavaShim: it was related to incompatible Guava versions, and I'm afraid removing it can make extra work in the future if we find ourselves in a similar situation...but regardless, removing it should be a scope of a different ticket

this one can focus on dropping hadoop 2.x support only, and we can make it happen quite easily

Aggarwal-Raghav · 2026-06-03T09:26:26Z

Drop Hadoop 2.x Support by eliminate Hadoop Shim Layer
Moved the Hadoop 2.8+ shims logic to TezUtilsInternal.java.
Removed hadoop27, hadoop28 profiles
Removed GuavaShim layer as well which was pre v18.0 of guava.
thanks for the patch @Aggarwal-Raghav so far! dropping the whole shim looks strange for me, I thought this is only for dropping hadoop 2.x, but shims are more generic way to handle different hadoop versions, which can be useful in the future too, right? please let me know if I miss something
the same applies to GuavaShim: it was related to incompatible Guava versions, and I'm afraid removing it can make extra work in the future if we find ourselves in a similar situation...but regardless, removing it should be a scope of a different ticket
this one can focus on dropping hadoop 2.x support only, and we can make it happen quite easily

I thought about it before dropping the shims framework, I considered the potential for future incompatibility scenarios. These include the incompatibility between Hadoop3.x and Hadoop 4.x (in the future maybe). But these scenario are speculative and its better to avoid the maintenance of two versions of the hadoop or guava. It is preferable to support the latest version at the present time.

We had Hadoop27 profile but if you try to run on master it is not even running becuase of self-referene in 1 of the pom.xml. Another instance is that hadoop2.x is jdk8 but now its jdk21. I hightly doubt even with shims present the JVM 8 and JDK 21 JVM cross-compatiblity will work.

But if you think its better to retain shims framework confirm once. I have expressed my thoughts but I don't have the final say here 😅

Aggarwal-Raghav · 2026-06-03T18:20:22Z

ok, I've updated the PR, let me know if its in-line with the scope, I have updated the DefaultHadoopShims to have methods like setHadoopCallerContext , earlier in Hadoop3.x it was no-op. There are few unused fields and methods , which needs discussion.

abstractdog · 2026-06-04T11:10:34Z

ok, I've updated the PR, let me know if its in-line with the scope, I have updated the DefaultHadoopShims to have methods like setHadoopCallerContext , earlier in Hadoop3.x it was no-op. There are few unused fields and methods , which needs discussion.

spot-checked, looks good, let me get back to this in few days in detail
thanks so far!

Aggarwal-Raghav · 2026-06-04T16:09:06Z

Jenkins stopped the last run, will re-trigger.

01:02:49 ERROR: Caught signal TERM.
01:02:52 01:02:52 01:02:52 Cleaning up docker image used for testing.
01:02:53 Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
01:02:53

tez-yetus · 2026-06-04T19:03:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 40s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 36s		master passed
+1 💚	compile	4m 2s		master passed
+1 💚	checkstyle	5m 17s		master passed
-1 ❌	mvnsite	0m 42s	/branch-mvnsite-tez-api.txt	tez-api in master failed.
-1 ❌	mvnsite	1m 53s	/branch-mvnsite-root.txt	root in master failed.
-1 ❌	mvnsite	0m 28s	/branch-mvnsite-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
-1 ❌	javadoc	0m 24s	/branch-javadoc-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
+0 🆗	spotbugs	0m 42s		hadoop-shim in master has 3 extant spotbugs warnings.
+0 🆗	spotbugs	1m 17s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 52s		tez-plugins in master has 43 extant spotbugs warnings.
+0 🆗	spotbugs	0m 41s		tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 49s		tez-tools/analyzers in master has 20 extant spotbugs warnings.
+0 🆗	spotbugs	0m 26s		branch/docs no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	6m 29s		root in master has 1054 extant spotbugs warnings.
-1 ❌	spotbugs	0m 25s	/branch-spotbugs-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
+0 🆗	spotbugs	0m 28s		branch/tez-dist no spotbugs output file (spotbugsXml.xml)
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 43s		the patch passed
+1 💚	codespell	1m 34s		No new issues.
+1 💚	compile	3m 56s		the patch passed
+1 💚	javac	3m 56s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 25s	/results-checkstyle-hadoop-shim.txt	hadoop-shim: The patch generated 1 new + 8 unchanged - 0 fixed = 9 total (was 8)
-0 ⚠️	checkstyle	1m 1s	/results-checkstyle-root.txt	root: The patch generated 1 new + 8 unchanged - 2 fixed = 9 total (was 10)
+1 💚	markdownlint	0m 5s		The patch generated 0 new + 117 unchanged - 2 fixed = 117 total (was 119)
-1 ❌	mvnsite	0m 42s	/patch-mvnsite-tez-api.txt	tez-api in the patch failed.
-1 ❌	mvnsite	1m 53s	/patch-mvnsite-root.txt	root in the patch failed.
+1 💚	javadoc	4m 49s		the patch passed
+0 🆗	spotbugs	0m 28s		tez-dist has no data from spotbugs
+0 🆗	spotbugs	0m 26s		docs has no data from spotbugs
			_ Other Tests _
+1 💚	unit	71m 58s		root in the patch passed.
+1 💚	asflicense	4m 9s		The patch does not generate ASF License warnings.
		172m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/3/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense codespell detsecrets markdownlint mvnsite xmllint javac javadoc spotbugs checkstyle
uname	Linux dd47cbdd944c 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `330fdc8`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/3/testReport/
Max. process+thread count	1394 (vs. ulimit of 5500)
modules	C: hadoop-shim tez-api tez-plugins tez-plugins/tez-yarn-timeline-history tez-tools/analyzers tez-dist docs . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/3/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1 markdownlint=0.46.0
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

tez-yetus · 2026-06-17T09:13:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	9m 26s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 43s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 37s		master passed
+1 💚	compile	4m 2s		master passed
+1 💚	checkstyle	5m 7s		master passed
-1 ❌	mvnsite	0m 26s	/branch-mvnsite-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
-1 ❌	javadoc	0m 23s	/branch-javadoc-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
+0 🆗	spotbugs	0m 41s		hadoop-shim in master has 3 extant spotbugs warnings.
+0 🆗	spotbugs	1m 17s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 50s		tez-plugins in master has 43 extant spotbugs warnings.
+0 🆗	spotbugs	0m 40s		tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 47s		tez-tools/analyzers in master has 20 extant spotbugs warnings.
+0 🆗	spotbugs	0m 25s		branch/docs no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	6m 35s		root in master has 1054 extant spotbugs warnings.
-1 ❌	spotbugs	0m 25s	/branch-spotbugs-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
+0 🆗	spotbugs	0m 29s		branch/tez-dist no spotbugs output file (spotbugsXml.xml)
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 51s		the patch passed
+1 💚	codespell	1m 34s		No new issues.
+1 💚	compile	3m 57s		the patch passed
+1 💚	javac	3m 57s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 23s	/results-checkstyle-hadoop-shim.txt	hadoop-shim: The patch generated 1 new + 8 unchanged - 0 fixed = 9 total (was 8)
-0 ⚠️	checkstyle	0m 59s	/results-checkstyle-root.txt	root: The patch generated 1 new + 8 unchanged - 2 fixed = 9 total (was 10)
+1 💚	markdownlint	0m 5s		The patch generated 0 new + 117 unchanged - 2 fixed = 117 total (was 119)
+1 💚	mvnsite	21m 47s		the patch passed
+1 💚	javadoc	4m 42s		the patch passed
+0 🆗	spotbugs	0m 27s		tez-dist has no data from spotbugs
+0 🆗	spotbugs	0m 25s		docs has no data from spotbugs
			_ Other Tests _
-1 ❌	unit	71m 32s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	3m 57s		The patch does not generate ASF License warnings.
		202m 5s

Reason	Tests
Failed junit tests	tez.history.TestHistoryParser

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/4/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense codespell detsecrets markdownlint mvnsite xmllint javac javadoc spotbugs checkstyle
uname	Linux 022764deb853 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `5766cda`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/4/testReport/
Max. process+thread count	2121 (vs. ulimit of 5500)
modules	C: hadoop-shim tez-api tez-plugins tez-plugins/tez-yarn-timeline-history tez-tools/analyzers tez-dist docs . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/4/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1 markdownlint=0.46.0
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Aggarwal-Raghav · 2026-06-17T09:18:59Z

[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.tez.history.TestHistoryParser
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.63 s -- in org.apache.tez.history.TestHistoryParser
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:20 min
[INFO] Finished at: 2026-06-17T14:48:33+05:30
[INFO] ------------------------------------------------------------------------

UT seems flaky, will re-run after review

…ntation - Retains the core HadoopShim framework and GuavaShim - Removes hadoop27 and hadoop28 profiles from all pom.xml files - Removes hadoop-shim-impls module (Hadoop 2.7 and 2.8 specific shims)

tez-yetus · 2026-06-23T21:23:03Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	9m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 44s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 32s		master passed
+1 💚	compile	3m 58s		master passed
+1 💚	checkstyle	4m 45s		master passed
-1 ❌	javadoc	0m 23s	/branch-javadoc-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
+0 🆗	spotbugs	0m 41s		hadoop-shim in master has 3 extant spotbugs warnings.
+0 🆗	spotbugs	1m 16s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 49s		tez-plugins in master has 43 extant spotbugs warnings.
+0 🆗	spotbugs	0m 41s		tez-plugins/tez-yarn-timeline-history in master has 2 extant spotbugs warnings.
+0 🆗	spotbugs	0m 46s		tez-tools/analyzers in master has 20 extant spotbugs warnings.
+0 🆗	spotbugs	6m 27s		root in master has 1054 extant spotbugs warnings.
-1 ❌	spotbugs	0m 24s	/branch-spotbugs-hadoop-shim-impls_hadoop-shim-2.7.txt	hadoop-shim-2.7 in master failed.
+0 🆗	spotbugs	0m 29s		branch/tez-dist no spotbugs output file (spotbugsXml.xml)
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 37s		the patch passed
+1 💚	codespell	1m 33s		No new issues.
+1 💚	compile	3m 57s		the patch passed
+1 💚	javac	3m 57s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 22s		The patch passed checkstyle in hadoop-shim
+1 💚	checkstyle	0m 29s		The patch passed checkstyle in tez-api
+1 💚	checkstyle	0m 28s		The patch passed checkstyle in tez-plugins
+1 💚	checkstyle	0m 24s		The patch passed checkstyle in tez-yarn-timeline-history
+1 💚	checkstyle	0m 25s		The patch passed checkstyle in analyzers
+1 💚	checkstyle	0m 22s		The patch passed checkstyle in tez-dist
+1 💚	checkstyle	0m 59s		root: The patch generated 0 new + 8 unchanged - 2 fixed = 8 total (was 10)
+1 💚	javadoc	4m 16s		the patch passed
+0 🆗	spotbugs	0m 29s		tez-dist has no data from spotbugs
			_ Other Tests _
+1 💚	unit	74m 26s		root in the patch passed.
+1 💚	asflicense	3m 31s		The patch does not generate ASF License warnings.
		155m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/5/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense codespell detsecrets xmllint javac javadoc spotbugs checkstyle
uname	Linux 6c35b1835440 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `61cc636`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/5/testReport/
Max. process+thread count	1529 (vs. ulimit of 5500)
modules	C: hadoop-shim tez-api tez-plugins tez-plugins/tez-yarn-timeline-history tez-tools/analyzers tez-dist . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-505/5/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Aggarwal-Raghav · 2026-06-24T08:08:22Z

@abstractdog , can you please take a look at this?

abstractdog

LGTM
tez.tar.gz contents also didn't change only the shim implementation disappeared which was expected

This was referenced Jun 2, 2026
TEZ-4711: [Addendum] Normalize ASF license header in hadoop27 profile #503
Closed
TEZ-4719: Move to Junit6 #502
Merged

Aggarwal-Raghav force-pushed the TEZ-4721 branch from 1988758 to 3d6cf60Compare June 3, 2026 18:15

Aggarwal-Raghav force-pushed the TEZ-4721 branch from 3d6cf60 to 14dcdf4Compare June 4, 2026 16:09

Aggarwal-Raghav changed the title ~~TEZ-4721: Remove legacy Hadoop 2.x support and hadoop-shim modules~~TEZ-4721: Remove legacy Hadoop 2.x support and hadoop-shim-impls moduleJun 5, 2026

Aggarwal-Raghav force-pushed the TEZ-4721 branch from 14dcdf4 to 605fcfbCompare June 17, 2026 05:50

TEZ-4721: Drop Hadoop 2.x support profiles and specific shims impleme…
26cb629
…ntation - Retains the core HadoopShim framework and GuavaShim - Removes hadoop27 and hadoop28 profiles from all pom.xml files - Removes hadoop-shim-impls module (Hadoop 2.7 and 2.8 specific shims)

Aggarwal-Raghav force-pushed the TEZ-4721 branch from 605fcfb to 26cb629Compare June 23, 2026 18:46

abstractdog self-requested a review June 25, 2026 14:15

abstractdog approved these changes Jun 25, 2026
View reviewed changes

abstractdog merged commit 9796ac5 into apache:masterJun 25, 2026
6 checks passed

Aggarwal-Raghav deleted the TEZ-4721 branch June 28, 2026 17:24

Navigation Menu

Uh oh!

TEZ-4721: Remove legacy Hadoop 2.x support and hadoop-shim-impls module - #505

TEZ-4721: Remove legacy Hadoop 2.x support and hadoop-shim-impls module#505
abstractdog merged 1 commit into
apache:masterfrom
Aggarwal-Raghav:TEZ-4721

Aggarwal-Raghav commented Jun 2, 2026•
edited
Loading

Uh oh!

tez-yetus commented Jun 2, 2026

Uh oh!

abstractdog commented Jun 3, 2026

Uh oh!

Aggarwal-Raghav commented Jun 3, 2026

Uh oh!

Aggarwal-Raghav commented Jun 3, 2026•
edited
Loading

Uh oh!

abstractdog commented Jun 4, 2026

Uh oh!

Aggarwal-Raghav commented Jun 4, 2026

Uh oh!

tez-yetus commented Jun 4, 2026

Uh oh!

tez-yetus commented Jun 17, 2026

Uh oh!

Aggarwal-Raghav commented Jun 17, 2026

Uh oh!

tez-yetus commented Jun 23, 2026

Uh oh!

Aggarwal-Raghav commented Jun 24, 2026

Uh oh!

abstractdog left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Aggarwal-Raghav commented Jun 2, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tez-yetus commented Jun 2, 2026

Uh oh!

abstractdog commented Jun 3, 2026

Uh oh!

Aggarwal-Raghav commented Jun 3, 2026

Uh oh!

Aggarwal-Raghav commented Jun 3, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abstractdog commented Jun 4, 2026

Uh oh!

Aggarwal-Raghav commented Jun 4, 2026

Uh oh!

tez-yetus commented Jun 4, 2026

Uh oh!

tez-yetus commented Jun 17, 2026

Uh oh!

Aggarwal-Raghav commented Jun 17, 2026

Uh oh!

tez-yetus commented Jun 23, 2026

Uh oh!

Aggarwal-Raghav commented Jun 24, 2026

Uh oh!

abstractdog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Aggarwal-Raghav commented Jun 2, 2026•
edited
Loading

Aggarwal-Raghav commented Jun 3, 2026•
edited
Loading