TEZ-4518: Added capability to limit number of spill files being generated

[mudit1289]

ASF Jira: https://issues.apache.org/jira/browse/TEZ-4518

We have been facing some issues where many of our cluster node disks go full because of some rogue applications creating a lot of spill data

We wanted to fail the app if more than a threshold amount of spill files are written

In this PR, we are trying to introduce a config to limit the count of sort spill files being written by a container

[mudit1289]

@abstractdog@ayushtkn can you please review this

[tez-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 22s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	18m 7s	master passed
+1 💚	compile	0m 39s	master passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04
+1 💚	compile	0m 38s	master passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
+1 💚	checkstyle	1m 29s	master passed
+1 💚	javadoc	0m 49s	master passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04
+1 💚	javadoc	0m 33s	master passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
+0 🆗	spotbugs	1m 42s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	1m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 22s	the patch passed
+1 💚	compile	0m 24s	the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04
+1 💚	javac	0m 24s	the patch passed
+1 💚	compile	0m 22s	the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
+1 💚	javac	0m 22s	the patch passed
+1 💚	checkstyle	0m 23s	tez-runtime-library: The patch generated 0 new + 215 unchanged - 1 fixed = 215 total (was 216)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 21s	the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04
+1 💚	javadoc	0m 19s	the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
+1 💚	findbugs	1m 2s	the patch passed
		_ Other Tests _
+1 💚	unit	6m 8s	tez-runtime-library in the patch passed.
+1 💚	asflicense	0m 18s	The patch does not generate ASF License warnings.
		35m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-312/6/artifact/out/Dockerfile
GITHUB PR	#312
JIRA Issue	TEZ-4518
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux 876b9f8da1fc 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / 5bba1ff
Default Java	Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~22.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-312/6/testReport/
Max. process+thread count	1100 (vs. ulimit of 5500)
modules	C: tez-runtime-library U: tez-runtime-library
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-312/6/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[abstractdog]

shouldn't this duplicated logic go to ExternalSorter instead of all children (PipelinedSorter, DefaultSorter)?