[SPARK-27157][DOCS] Add Executor level metrics to monitoring docs

[LantaoJin]

What changes were proposed in this pull request?

A sub-task of SPARK-23206
Add Executor level metrics to monitoring docs

How was this patch tested?

jekyll

[wangyum]

two </tr>?

[LantaoJin]

removed

[srowen]

This is probably OK, but are these metrics things that Spark generates or that are generated automatically by Ganglia et al? that is, do we need to document them or point at existing external docs?

[SparkQA]

Test build #4620 has finished for PR 24090 at commit 4049689.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[LucaCanali]

This looks to me a useful addition to the monitoring documentation.
An additional comment is that ProcessTree executor metrics are collected only under certain conditions. I was thinking that maybe this fact can be documented in this PR?
As far as I can see the conditions are the logical AND of the folloving: /proc filesystem exists, spark.eventLog.logStageExecutorProcessTreeMetrics.enabled=true, spark.eventLog.logStageExecutorMetrics.enabled=true

[LantaoJin]

This is probably OK, but are these metrics things that Spark generates or that are generated automatically by Ganglia et al? that is, do we need to document them or point at existing external docs?

@srowen They are generated by Spark, see ExecutorMetricType

[LantaoJin]

Thank you @LucaCanali, I've added this metrics collection condition.

[SparkQA]

Test build #4630 has finished for PR 24090 at commit 4779f7f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.