TEZ-4552. Upgrade protobuf to 3.24.4 due to CVE.

[slfan1989]

JIRA: TEZ-4552. Upgrade protobuf to 3.24.4 due to CVE.

I found that there are 3 CVE issues that we need to deal with. These CVE issues are related to protobuf. Our protobuf uses 3.21.1, which is an old version. This PR will try to upgrade the protobuf version to solve the CVE issue.

• CVE-2022-3171
• CVE-2022-3509
• CVE-2022-3510

[Aggarwal-Raghav]

@slfan1989, for protobuf version upgrade, you need to update the generated code for the corresponding proto files as well.

[abstractdog]

@slfan1989, for protobuf version upgrade, you need to update the generated code for the corresponding proto files as well.

what kind of generated code upgrade do we expect here? does anything change if we run "mvn clean install -DskipTests"?

[Aggarwal-Raghav]

@slfan1989, for protobuf version upgrade, you need to update the generated code for the corresponding proto files as well.

what kind of generated code upgrade do we expect here? does anything change if we run "mvn clean install -DskipTests"?

My bad, as per my understanding, for .proto file, the generated code for that corresponding proto file, generally have some changes when we change protobuf version but here its not the case(I should have tested this before commenting 🙁)

[abstractdog]

@slfan1989, for protobuf version upgrade, you need to update the generated code for the corresponding proto files as well.

what kind of generated code upgrade do we expect here? does anything change if we run "mvn clean install -DskipTests"?

My bad, as per my understanding, for .proto file, the generated code for that corresponding proto file, generally have some changes when we change protobuf version but here its not the case(I should have tested this before commenting 🙁)

no worries, thanks for the comments and activity here!
I believe we are good to proceed with this once it's moved from draft and we have a precommit test run

[BilwaST]

@slfan1989@abstractdog Can we update protobuf version to 3.24.4? Hive is also using the same version

[slfan1989]

@slfan1989@abstractdog Can we update protobuf version to 3.24.4? Hive is also using the same version

Thanks for your message, I will try to upgrade to 3.24.4.

[BilwaST]

Thanks for your patch @slfan1989. Looks good to me

[slfan1989]

@abstractdog@Aggarwal-Raghav@BilwaST Thank you for paying attention to this pr! The reason I want to upgrade protobuf is because there are some CVE vulnerabilities in lower versions of protobuf, so I try to upgrade protobuf to a higher version to solve related issues.

Some known protobuf vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-3171
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-3509
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-3510

[tez-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	22m 11s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+1 💚	mvninstall	15m 11s	master passed
+1 💚	compile	2m 16s	master passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	compile	2m 7s	master passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~22.04-b06
+1 💚	javadoc	1m 35s	master passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javadoc	1m 12s	master passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~22.04-b06
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 13s	the patch passed
+1 💚	compile	2m 18s	the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javac	2m 18s	the patch passed
+1 💚	compile	2m 13s	the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~22.04-b06
+1 💚	javac	2m 14s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	javadoc	1m 15s	the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1
+1 💚	javadoc	1m 12s	the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~22.04-b06
		_ Other Tests _
-1 ❌	unit	50m 53s	root in the patch failed.
+1 💚	asflicense	0m 38s	The patch does not generate ASF License warnings.
		108m 32s

Reason	Tests
Failed junit tests	tez.test.TestAMRecovery
	tez.test.TestRecovery
	tez.test.TestDAGRecovery

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-345/3/artifact/out/Dockerfile
GITHUB PR	#345
Optional Tests	dupname asflicense javac javadoc unit xml compile
uname	Linux ad02bf218b98 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / 906059a
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~22.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu222.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~22.04-b06
unit	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-345/3/artifact/out/patch-unit-root.txt
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-345/3/testReport/
Max. process+thread count	2100 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-345/3/console
versions	git=2.34.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[abstractdog]

failing unit tests are because of TEZ-4559, this looks good to me