Support stage level schedule for final write stage

[ulysses-you]

Why are the changes needed?

This pr adds a new rule FinalStageResourceManager to inject custom reousrce profile for the final write stage.

It nowprovide two features:

• Kill redundant executors
  We first get the final stage partition which is the actually required cores, then kill the redundant executors. The priority of kill executors follow:

  1. kill executor who is younger than other (The older the JIT works better)
  2. kill executor who produces less shuffle data first

• Custom resource profile
  We can specify the custom executor resource for final write stage. It now supports:

  executor core
  executor memory
  executor memory overhead
  executor off heap memory
  

The reason why add this feature is that, if the previous stage contains lots executors but final stage has less, then the tasks of final stage would be scheduled randomly in all exists executors which may cause resource waste. e.g., each executor only run 1 or 2 tasks but holds 4 or 5 cores.

How was this patch tested?

test manually

• test for the kill executor

• test for custom resource profile

[codecov-commenter]

Codecov Report

Merging #4574 (5ce2c5d) into master (465e23a) will increase coverage by 0.03%.
The diff coverage is 96.87%.

❗ Current head 5ce2c5d differs from pull request most recent head 54afa36. Consider uploading reports for the commit 54afa36 to get more accurate results

@@ Coverage Diff @@## master #4574 +/- ##
============================================
+ Coverage 53.32% 53.36% +0.03% 
Complexity 13 13 ============================================
Files 573 577 +4 Lines 31501 31589 +88 Branches 4239 4245 +6 ============================================
+ Hits 16797 16856 +59 - Misses 13128 13147 +19 - Partials 1576 1586 +10 

Impacted Files	Coverage Δ
...in/scala/org/apache/kyuubi/sql/KyuubiSQLConf.scala	98.61% <96.87%> (-0.50%)	⬇️

... and 22 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[ulysses-you]

It is really hard to add some tests, so I test all of this manully. cc @yaooqinn@pan3793@bowenliang123@cfmcgrady if you have some other idea about this.

[bowenliang123]

Is there any description available for the manual test results other than the screenshots? I don't quite understand the screenshots showing consequence related to the purposed features.

[ulysses-you]

@bowenliang123 I have added some tags, ideally, the screenshots show that:

1. kill executor
2. inject custom resource profile

[bowenliang123]

LGTM. Thx for the supplement.

[bowenliang123]

LGTM. Thx for the supplement.

[bowenliang123]

LGTM. Thx for the supplement.

[yaooqinn]

please separate this PR to small ones

[ulysses-you]

I seperate thie pr into two parts: 1. kill executor, 2. inject custom resource profile.

here is the pr #4592 for kill executors