TEZ-4715: Migrate Tez PB Protocols from ProtobufRPCEngine to ProtobufRPCEngine2 by Aggarwal-Raghav · Pull Request #493 · apache/tez

Aggarwal-Raghav · 2026-04-28T18:14:07Z

Migrated Protobuf imports from com.google.protobuf to shaded org.apache.hadoop.thirdparty.protobuf
Added hadoop-thirdparty dependency (v1.4.0) to the root POM to ensure Hadoop 3.4.2 compatibility. This will be updated for every hadoop version upgrade. For now I have kept proto version to 3.25.5 as it was in tez and will be in-sync with hadoop-3.5.0
Used maven-replacer-plugin (same as hadoop) to automated package refactoring in generated proto code.

tez-yetus · 2026-04-28T18:15:27Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/1/console in case of problems.

Aggarwal-Raghav · 2026-04-30T05:38:23Z

Jenkins is not updating the final status 😔

[2026-04-29T22:12:13.679Z] [2026-04-29T22:12:13.679Z] ============================================================================
[2026-04-29T22:12:13.679Z] ============================================================================
[2026-04-29T22:12:13.679Z] Adding comment to Github
[2026-04-29T22:12:13.679Z] ============================================================================
[2026-04-29T22:12:13.679Z] ============================================================================
[2026-04-29T22:12:13.679Z] [2026-04-29T22:12:13.679Z] [2026-04-29T22:12:16.867Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:16.867Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:17.431Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:17.431Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:17.997Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:18.561Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:18.561Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:19.124Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:19.689Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:19.689Z] ERROR: Failed to write github status. Token expired or missing repo:status write?
[2026-04-29T22:12:20.253Z] ERROR: Failed to write github status. Token expired or missing repo:status write?

Aggarwal-Raghav · 2026-05-01T07:45:11Z

The UT failure debugging in #483 (comment) requires hadoop thirdparty making this PR dependent.

TEZ-4715 + TEZ-4708 will completely migrate tez from WritableRPCEngine + ProtobufRPCEngine => ProtobufRPCEngine2

ayushtkn · 2026-05-02T19:37:32Z

@abstractdog From the code point of view, this seems right, We had to just use the RPCEngine2 & migrate to the Hadoop Thirdparty. That is rightly done here.

But I think this would be an incompatible change, beyond this point I believe Hive master won't build nor work with Tez master. Hive has to do the same activity when moving to Tez-1.0.0 & that would be incompatible for Hive as well & believe the older Hadoop versions won't work post this activity in Hive or Tez, unless there was something done later to fix it, which I don't think...

There were compat issues that is why I believe HADOOP-17046 was done later to make sure older client can work with them. Means protobuf is indeed a very known heavy Incompat change, a lot of discussion at HADOOP-13363, in case interested.

These are the tradeoffs, I just wanted to highlight. Rest all cool

Aggarwal-Raghav · 2026-05-06T18:28:32Z

@abstractdog @ayushtkn , can you help on the CI for this?

[2026-05-01T08:30:07.897Z] ERROR: Caught signal TERM.
[2026-05-01T08:30:07.897Z] [2026-05-01T08:30:07.897Z] [2026-05-01T08:30:07.897Z] Cleaning up docker image used for testing.
[2026-05-01T08:30:07.897Z] [Fri May 1 08:30:04 AM UTC 2026 DEBUG]: dockercmd: /usr/bin/docker rmi yetus/tez:tp-3788
[2026-05-01T08:30:07.898Z] Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Aggarwal-Raghav · 2026-05-10T11:01:08Z

I verified that the JAVA_HOME exists in the docker image

but it the jenkins the JAVA_HOME is explicitly set:

2026-05-10T10:51:24.988Z] [Sun May 10 10:51:24 AM UTC 2026 DEBUG]: dockercmd: /usr/bin/docker run --rm=true -i -v ... --env=JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64

tez-yetus · 2026-05-10T11:13:49Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/6/console in case of problems.

tez-yetus · 2026-05-10T11:43:29Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/7/console in case of problems.

…erfile (apache#493) - Resolve JAVA_HOME mismatch by creating a symlink from the host's expected path (/usr/lib/jvm/java-XX-openjdk-amd64) to the container's internal path (/opt/java/openjdk). - Parameterize JAVA_VERSION using a build argument to simplify future JDK upgrades. - Add descriptive comments regarding environment variable propagation from Yetus. - Fix ShellCheck SC2086 by adding quotes to the symlink path.

Aggarwal-Raghav · 2026-05-10T12:02:20Z

@abstractdog , can you please remove the JAVA_HOME env varaible setting in docke run command from Jenkins?
I tried creating symlink but its not helping.

Let me know if i need to revert #494.

CI for tez is not a good experiece at all, this issue should have been caught in #494 itself. Moreover, not sure why mvninstall is failing in run 4. This diff patch running is not helping as it is trying to dowload the SNAPSHOT tez modules jars (i believe).

abstractdog · 2026-05-10T16:46:02Z

@abstractdog , can you please remove the JAVA_HOME env varaible setting in docke run command from Jenkins? I tried creating symlink but its not helping.
Let me know if i need to revert #494.
CI for tez is not a good experiece at all, this issue should have been caught in #494 itself. Moreover, not sure why mvninstall is failing in run 4. This diff patch running is not helping as it is trying to dowload the SNAPSHOT tez modules jars (i believe).

this is very weird, I guess your Dockerfile change didn't trigger a precommit test
I'm in favor of running the whole test for every PR, I already created a corresponding ticket for that https://issues.apache.org/jira/browse/TEZ-4355

your improvement in #494 was great, so I would not revert it if we can fix the problem in 1-2 days (considering the project's pace, it's fine)

Aggarwal-Raghav · 2026-05-10T21:04:57Z

OK, pipeline seems to be progressing now. I have updated the JAVA_HOME in Jenkinsfile to match the eclipse temurin JAVA_HOME. Internally we use cloudbees Jenkins and some additional tools that passes the env variables, so I thought that JAVA_HOME was coming from Jenkins UI but it was not the case. That's why I asked:

can you please remove the JAVA_HOME env varaible setting in docke run command from Jenkins? I tried creating symlink but its not helping.

If the run#8 works then I'll raise an addendum PR for TEZ-4717 incorporating Jenkins file change in it. I don't see any other place in project where it need updation for JAVA_HOME

Aggarwal-Raghav · 2026-05-11T14:57:08Z

This is ready for review @ayushtkn @abstractdog , the spotbugs are unrelated and mvninstall is false positive

Aggarwal-Raghav · 2026-05-20T08:28:24Z

Rebased the PR with latest master.

tez-yetus · 2026-05-20T11:22:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 53s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 34 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 41s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 45s		master passed
+1 💚	compile	3m 56s		master passed
+1 💚	checkstyle	6m 17s		master passed
+1 💚	javadoc	6m 35s		master passed
+0 🆗	spotbugs	1m 22s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	0m 46s		tez-common in master has 13 extant spotbugs warnings.
+0 🆗	spotbugs	0m 53s		tez-runtime-internals in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 11s		tez-runtime-library in master has 143 extant spotbugs warnings.
+0 🆗	spotbugs	0m 55s		tez-mapreduce in master has 97 extant spotbugs warnings.
+0 🆗	spotbugs	1m 40s		tez-dag in master has 537 extant spotbugs warnings.
+0 🆗	spotbugs	0m 50s		tez-tests in master has 6 extant spotbugs warnings.
+0 🆗	spotbugs	0m 47s		tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗	spotbugs	0m 42s		tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	0m 42s		tez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	6m 49s		root in master has 1054 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 59s		the patch passed
+1 💚	codespell	1m 45s		No new issues.
+1 💚	compile	3m 58s		the patch passed
+1 💚	javac	3m 58s		root generated 0 new + 738 unchanged - 4 fixed = 738 total (was 742)
+1 💚	blanks	0m 1s		The patch has no blanks issues.
+1 💚	checkstyle	6m 12s		the patch passed
+1 💚	javadoc	6m 41s		the patch passed
-1 ❌	spotbugs	1m 3s	/new-spotbugs-tez-plugins_tez-protobuf-history-plugin.html	tez-plugins/tez-protobuf-history-plugin generated 4 new + 3 unchanged - 4 fixed = 7 total (was 7)
-1 ❌	spotbugs	7m 15s	/new-spotbugs-root.html	root generated 4 new + 1050 unchanged - 4 fixed = 1054 total (was 1054)
			_ Other Tests _
+1 💚	unit	70m 44s		root in the patch passed.
+1 💚	asflicense	5m 41s		The patch does not generate ASF License warnings.
		174m 12s

Reason	Tests
SpotBugs	module:tez-plugins/tez-protobuf-history-plugin
	Exception thrown in class org.apache.tez.dag.history.logging.proto.DatePartitionedLogger at new org.apache.tez.dag.history.logging.proto.DatePartitionedLogger(Parser, Path, Configuration, Clock) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At DatePartitionedLogger.java:at new org.apache.tez.dag.history.logging.proto.DatePartitionedLogger(Parser, Path, Configuration, Clock) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At DatePartitionedLogger.java:[line 68]
	org.apache.tez.dag.history.logging.proto.ProtoMessageWritable.getMessage() may expose internal representation by returning ProtoMessageWritable.message At ProtoMessageWritable.java:by returning ProtoMessageWritable.message At ProtoMessageWritable.java:[line 48]
	org.apache.tez.dag.history.logging.proto.ProtoMessageWritable.setMessage(MessageLite) may expose internal representation by storing an externally mutable object into ProtoMessageWritable.message At ProtoMessageWritable.java:by storing an externally mutable object into ProtoMessageWritable.message At ProtoMessageWritable.java:[line 52]
	Exception thrown in class org.apache.tez.dag.history.logging.proto.ProtoMessageWriter at new org.apache.tez.dag.history.logging.proto.ProtoMessageWriter(Configuration, Path, Parser) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At ProtoMessageWriter.java:at new org.apache.tez.dag.history.logging.proto.ProtoMessageWriter(Configuration, Path, Parser) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At ProtoMessageWriter.java:[line 41]
SpotBugs	module:root
	Exception thrown in class org.apache.tez.dag.history.logging.proto.DatePartitionedLogger at new org.apache.tez.dag.history.logging.proto.DatePartitionedLogger(Parser, Path, Configuration, Clock) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At DatePartitionedLogger.java:at new org.apache.tez.dag.history.logging.proto.DatePartitionedLogger(Parser, Path, Configuration, Clock) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At DatePartitionedLogger.java:[line 68]
	org.apache.tez.dag.history.logging.proto.ProtoMessageWritable.getMessage() may expose internal representation by returning ProtoMessageWritable.message At ProtoMessageWritable.java:by returning ProtoMessageWritable.message At ProtoMessageWritable.java:[line 48]
	org.apache.tez.dag.history.logging.proto.ProtoMessageWritable.setMessage(MessageLite) may expose internal representation by storing an externally mutable object into ProtoMessageWritable.message At ProtoMessageWritable.java:by storing an externally mutable object into ProtoMessageWritable.message At ProtoMessageWritable.java:[line 52]
	Exception thrown in class org.apache.tez.dag.history.logging.proto.ProtoMessageWriter at new org.apache.tez.dag.history.logging.proto.ProtoMessageWriter(Configuration, Path, Parser) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At ProtoMessageWriter.java:at new org.apache.tez.dag.history.logging.proto.ProtoMessageWriter(Configuration, Path, Parser) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At ProtoMessageWriter.java:[line 41]

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/10/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense codespell detsecrets javac javadoc xmllint spotbugs checkstyle
uname	Linux eca32b9d2182 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `b6b5d42`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/10/testReport/
Max. process+thread count	1358 (vs. ulimit of 5500)
modules	C: tez-api tez-common tez-runtime-internals tez-runtime-library tez-mapreduce tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-aux-services . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/10/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Aggarwal-Raghav · 2026-05-20T11:24:41Z

The CI has passed, the spotbugs are not relevant to my change.
In Run9 older Jenkinsfile mvninstall was failing.
in Run10, new Jenkinsfile mvninstall is passing.

Aggarwal-Raghav · 2026-05-20T14:31:04Z

@ayushtkn , This is ready from my end. Could you please verify if it is ready for merging?

Aggarwal-Raghav · 2026-08-13T18:01:25Z

Rebased to resolve merge conflicts

tez-yetus · 2026-08-13T19:34:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	9m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 34 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 43s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 49s		master passed
+1 💚	compile	3m 58s		master passed
+1 💚	checkstyle	6m 11s		master passed
+1 💚	javadoc	6m 29s		master passed
+0 🆗	spotbugs	1m 21s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	0m 46s		tez-common in master has 13 extant spotbugs warnings.
+0 🆗	spotbugs	0m 52s		tez-runtime-internals in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 12s		tez-runtime-library in master has 143 extant spotbugs warnings.
+0 🆗	spotbugs	0m 55s		tez-mapreduce in master has 97 extant spotbugs warnings.
+0 🆗	spotbugs	1m 42s		tez-dag in master has 537 extant spotbugs warnings.
+0 🆗	spotbugs	0m 50s		tez-tests in master has 6 extant spotbugs warnings.
+0 🆗	spotbugs	0m 48s		tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗	spotbugs	0m 43s		tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	0m 44s		tez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	6m 50s		root in master has 1054 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for patch
-1 ❌	mvninstall	2m 3s	/patch-mvninstall-root.txt	root in the patch failed.
+1 💚	codespell	1m 45s		No new issues.
-1 ❌	compile	1m 7s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	javac	1m 7s	/patch-compile-root.txt	root in the patch failed.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 32s	/results-checkstyle-tez-runtime-library.txt	tez-runtime-library: The patch generated 6 new + 535 unchanged - 0 fixed = 541 total (was 535)
-0 ⚠️	checkstyle	0m 22s	/results-checkstyle-tez-plugins_tez-yarn-timeline-history-with-fs.txt	tez-plugins/tez-yarn-timeline-history-with-fs: The patch generated 1 new + 7 unchanged - 0 fixed = 8 total (was 7)
-0 ⚠️	checkstyle	1m 6s	/results-checkstyle-root.txt	root: The patch generated 7 new + 1706 unchanged - 0 fixed = 1713 total (was 1706)
-1 ❌	javadoc	0m 24s	/patch-javadoc-tez-runtime-library.txt	tez-runtime-library in the patch failed.
-1 ❌	javadoc	0m 23s	/patch-javadoc-tez-plugins_tez-yarn-timeline-history-with-fs.txt	tez-yarn-timeline-history-with-fs in the patch failed.
-1 ❌	javadoc	0m 42s	/patch-javadoc-root.txt	root in the patch failed.
-1 ❌	spotbugs	0m 23s	/patch-spotbugs-tez-runtime-library.txt	tez-runtime-library in the patch failed.
-1 ❌	spotbugs	0m 40s	/patch-spotbugs-tez-dag.txt	tez-dag in the patch failed.
-1 ❌	spotbugs	0m 32s	/patch-spotbugs-tez-tests.txt	tez-tests in the patch failed.
-1 ❌	spotbugs	1m 2s	/new-spotbugs-tez-plugins_tez-protobuf-history-plugin.html	tez-plugins/tez-protobuf-history-plugin generated 4 new + 3 unchanged - 4 fixed = 7 total (was 7)
-1 ❌	spotbugs	0m 24s	/patch-spotbugs-tez-plugins_tez-yarn-timeline-history-with-fs.txt	tez-yarn-timeline-history-with-fs in the patch failed.
-1 ❌	spotbugs	2m 1s	/patch-spotbugs-root.txt	root in the patch failed.
			_ Other Tests _
-1 ❌	unit	5m 17s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	4m 42s		The patch does not generate ASF License warnings.
		92m 41s

Reason	Tests
SpotBugs	module:tez-plugins/tez-protobuf-history-plugin
	Exception thrown in class org.apache.tez.dag.history.logging.proto.DatePartitionedLogger at new org.apache.tez.dag.history.logging.proto.DatePartitionedLogger(Parser, Path, Configuration, Clock) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At DatePartitionedLogger.java:at new org.apache.tez.dag.history.logging.proto.DatePartitionedLogger(Parser, Path, Configuration, Clock) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At DatePartitionedLogger.java:[line 68]
	org.apache.tez.dag.history.logging.proto.ProtoMessageWritable.getMessage() may expose internal representation by returning ProtoMessageWritable.message At ProtoMessageWritable.java:by returning ProtoMessageWritable.message At ProtoMessageWritable.java:[line 48]
	org.apache.tez.dag.history.logging.proto.ProtoMessageWritable.setMessage(MessageLite) may expose internal representation by storing an externally mutable object into ProtoMessageWritable.message At ProtoMessageWritable.java:by storing an externally mutable object into ProtoMessageWritable.message At ProtoMessageWritable.java:[line 52]
	Exception thrown in class org.apache.tez.dag.history.logging.proto.ProtoMessageWriter at new org.apache.tez.dag.history.logging.proto.ProtoMessageWriter(Configuration, Path, Parser) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At ProtoMessageWriter.java:at new org.apache.tez.dag.history.logging.proto.ProtoMessageWriter(Configuration, Path, Parser) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At ProtoMessageWriter.java:[line 41]

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/11/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense codespell detsecrets javac javadoc xmllint spotbugs checkstyle
uname	Linux 59deaedc85d2 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `d348bf1`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/11/testReport/
Max. process+thread count	286 (vs. ulimit of 5500)
modules	C: tez-api tez-common tez-runtime-internals tez-runtime-library tez-mapreduce tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-aux-services . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/11/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

…RPCEngine2

tez-yetus · 2026-08-13T21:15:45Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 34 new or modified test files.
			_ master Compile Tests _
+0 🆗	mvndep	0m 43s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 48s		master passed
+1 💚	compile	4m 1s		master passed
+1 💚	checkstyle	6m 12s		master passed
+1 💚	javadoc	6m 31s		master passed
+0 🆗	spotbugs	1m 21s		tez-api in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	0m 47s		tez-common in master has 13 extant spotbugs warnings.
+0 🆗	spotbugs	0m 53s		tez-runtime-internals in master has 92 extant spotbugs warnings.
+0 🆗	spotbugs	1m 13s		tez-runtime-library in master has 143 extant spotbugs warnings.
+0 🆗	spotbugs	0m 55s		tez-mapreduce in master has 97 extant spotbugs warnings.
+0 🆗	spotbugs	1m 41s		tez-dag in master has 537 extant spotbugs warnings.
+0 🆗	spotbugs	0m 50s		tez-tests in master has 6 extant spotbugs warnings.
+0 🆗	spotbugs	0m 49s		tez-plugins/tez-protobuf-history-plugin in master has 7 extant spotbugs warnings.
+0 🆗	spotbugs	0m 43s		tez-plugins/tez-yarn-timeline-history-with-fs in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	0m 44s		tez-plugins/tez-aux-services in master has 5 extant spotbugs warnings.
+0 🆗	spotbugs	6m 51s		root in master has 1054 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s		Maven dependency ordering for patch
-1 ❌	mvninstall	2m 9s	/patch-mvninstall-root.txt	root in the patch failed.
+1 💚	codespell	1m 46s		No new issues.
-1 ❌	compile	1m 24s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	javac	1m 24s	/patch-compile-root.txt	root in the patch failed.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	5m 58s		the patch passed
+1 💚	javadoc	6m 23s		the patch passed
-1 ❌	spotbugs	0m 34s	/patch-spotbugs-tez-runtime-library.txt	tez-runtime-library in the patch failed.
-1 ❌	spotbugs	0m 41s	/patch-spotbugs-tez-dag.txt	tez-dag in the patch failed.
-1 ❌	spotbugs	0m 31s	/patch-spotbugs-tez-tests.txt	tez-tests in the patch failed.
-1 ❌	spotbugs	1m 2s	/new-spotbugs-tez-plugins_tez-protobuf-history-plugin.html	tez-plugins/tez-protobuf-history-plugin generated 4 new + 3 unchanged - 4 fixed = 7 total (was 7)
-1 ❌	spotbugs	0m 27s	/patch-spotbugs-tez-plugins_tez-yarn-timeline-history-with-fs.txt	tez-yarn-timeline-history-with-fs in the patch failed.
-1 ❌	spotbugs	2m 8s	/patch-spotbugs-root.txt	root in the patch failed.
			_ Other Tests _
-1 ❌	unit	5m 22s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	4m 48s		The patch does not generate ASF License warnings.
		85m 56s

Reason	Tests
SpotBugs	module:tez-plugins/tez-protobuf-history-plugin
	Exception thrown in class org.apache.tez.dag.history.logging.proto.DatePartitionedLogger at new org.apache.tez.dag.history.logging.proto.DatePartitionedLogger(Parser, Path, Configuration, Clock) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At DatePartitionedLogger.java:at new org.apache.tez.dag.history.logging.proto.DatePartitionedLogger(Parser, Path, Configuration, Clock) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At DatePartitionedLogger.java:[line 68]
	org.apache.tez.dag.history.logging.proto.ProtoMessageWritable.getMessage() may expose internal representation by returning ProtoMessageWritable.message At ProtoMessageWritable.java:by returning ProtoMessageWritable.message At ProtoMessageWritable.java:[line 48]
	org.apache.tez.dag.history.logging.proto.ProtoMessageWritable.setMessage(MessageLite) may expose internal representation by storing an externally mutable object into ProtoMessageWritable.message At ProtoMessageWritable.java:by storing an externally mutable object into ProtoMessageWritable.message At ProtoMessageWritable.java:[line 52]
	Exception thrown in class org.apache.tez.dag.history.logging.proto.ProtoMessageWriter at new org.apache.tez.dag.history.logging.proto.ProtoMessageWriter(Configuration, Path, Parser) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At ProtoMessageWriter.java:at new org.apache.tez.dag.history.logging.proto.ProtoMessageWriter(Configuration, Path, Parser) will leave the constructor. The object under construction remains partially initialized and may be vulnerable to Finalizer attacks. At ProtoMessageWriter.java:[line 41]

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/12/artifact/out/Dockerfile
Optional Tests	dupname compile unit asflicense codespell detsecrets javac javadoc xmllint spotbugs checkstyle
uname	Linux a13b9698642c 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	tez-personality.sh
git revision	master / `d348bf1`
Default Java	Eclipse Adoptium-21.0.11+10-LTS
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/12/testReport/
Max. process+thread count	264 (vs. ulimit of 5500)
modules	C: tez-api tez-common tez-runtime-internals tez-runtime-library tez-mapreduce tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-aux-services . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-493/12/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

abstractdog · 2026-08-17T10:56:36Z

@Aggarwal-Raghav: besides the reported spotbugs problems (not sure how valid they are), can you please double-check if this upgrade breaks the current hive docker-compose setup? in general, Tez SNAPSHOT is not supposed to be stable all the time, but we're interested in merging this change can break anything
this is the setup I'm referring to: apache/hive#6435 (comment)
I can think of manually override the tez jars somehow
if a DAG is succeeded with new tez jars, we can make sure that the two proto versions are compatible over the wire

Aggarwal-Raghav mentioned this pull request Apr 29, 2026
TEZ-4712: Class-level SpotBugs exclusions for generated Protobuf classes #489
Merged

Aggarwal-Raghav force-pushed the TEZ-4715 branch from 580e597 to b459b5dCompare April 29, 2026 18:06

Aggarwal-Raghav force-pushed the TEZ-4715 branch from b459b5d to 2f2e58fCompare May 1, 2026 05:48

abstractdog requested a review from ayushtkn May 2, 2026 05:51

Aggarwal-Raghav force-pushed the TEZ-4715 branch from 2f2e58f to 6330365Compare May 8, 2026 18:03