TEZ-4521: Partition stats should be always uncompressed size

[okumin]

Currently, we need to configure the compressed size for ordered outputs and the decompressed size for unordered output. It makes sense if we can have consistent semantics.
https://issues.apache.org/jira/browse/TEZ-4521

[tez-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	13m 19s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	15m 15s	master passed
+1 💚	compile	0m 22s	master passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04
+1 💚	compile	0m 19s	master passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
+1 💚	checkstyle	1m 12s	master passed
+1 💚	javadoc	0m 32s	master passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04
+1 💚	javadoc	0m 19s	master passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
+0 🆗	spotbugs	1m 7s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	1m 6s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 13s	the patch passed
+1 💚	compile	0m 13s	the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04
+1 💚	javac	0m 13s	the patch passed
+1 💚	compile	0m 12s	the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
+1 💚	javac	0m 12s	the patch passed
+1 💚	checkstyle	0m 11s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 12s	the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04
+1 💚	javadoc	0m 11s	the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
+1 💚	findbugs	0m 35s	the patch passed
		_ Other Tests _
+1 💚	unit	4m 23s	tez-runtime-library in the patch passed.
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		39m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-317/1/artifact/out/Dockerfile
GITHUB PR	#317
JIRA Issue	TEZ-4521
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux 45f7623bdf4c 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / 51d6f53
Default Java	Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-317/1/testReport/
Max. process+thread count	2100 (vs. ulimit of 5500)
modules	C: tez-runtime-library U: tez-runtime-library
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-317/1/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[abstractdog]

does totalSize change with this patch? if it doesn't, why? if it does, can we validate it with this unit test or in anyway that makes sense to you @okumin ?

[okumin]

This patch doesn't change the total size. That's because the total size is stored in a different field of Protbuf from partitions stats. The design is valid since users have an option not to take partition stats at all(tez.runtime.report.partition.stats=none).
TEZ-4521 would remove the possibility where partition stats contain the compressed size. That's why I revised this file to prevent future users from being confused.

[abstractdog]

thanks @okumin for this patch, I've put a minor comment

[okumin]

@abstractdog Thanks! This patch is related to #306 and I'd be glad if you could take a look at it.

[abstractdog]

merged, thanks @okumin
I'll check the rest of this fair scheduling work as soon as I can, I promise :)