[SPARK-26196][SPARK-26281][WebUI] Total tasks title in the stage page is incorrect when there are failed or killed tasks and update duration metrics by shahidki31 · Pull Request #23160 · apache/spark

shahidki31 · 2018-11-28T01:58:35Z

What changes were proposed in this pull request?

This PR fixes 3 issues

Total tasks message in the tasks table is incorrect, when there are failed or killed tasks
Sorting of the "Duration" column is not correct
Duration in the aggregated tasks summary table and the tasks table and not matching.

Total tasks = numCompleteTasks + numActiveTasks + numKilledTasks + numFailedTasks;

Corrected the duration metrics in the tasks table as executorRunTime based on the PR #23081

How was this patch tested?

test step:
1)

bin/spark-shell scala > sc.parallelize(1 to 100, 10).map{ x => throw new RuntimeException("Bad executor")}.collect()

After patch:

Duration metrics:
Before patch:

After patch:

SparkQA · 2018-11-28T06:17:57Z

Test build #99346 has finished for PR 23160 at commit bbd745a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-11-28T06:21:09Z

Test build #99345 has finished for PR 23160 at commit 37bcd62.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

gatorsmile · 2018-11-28T23:01:48Z

cc @gengliangwang

gengliangwang

LGTM. Please add tag [WebUI] in the PR title

gengliangwang · 2018-11-30T12:17:24Z

cc @tgravescs @pgandhi999

pgandhi999 · 2018-11-30T12:28:07Z

What I meant was like in the screenshot posted above, whether the task table has 7 entries and they all show accurate information.

tgravescs · 2018-11-30T14:07:22Z

reviewing, to clarify @pgandhi999 question, I assume the screenshots above are cut off and the task table is really showing 7 total failed tasks?

shahidki31 · 2018-11-30T14:14:24Z

@tgravescs Yes. Task table is showing all the tasks. Sorry, I didn't put the whole screen shot.

pgandhi999 · 2018-11-30T12:25:52Z


 // building tasks table that uses server side functionality
- var totalTasksToShow = responseBody.numCompleteTasks + responseBody.numActiveTasks;
+ var totalTasksToShow = responseBody.numCompleteTasks + responseBody.numActiveTasks +


Ah that makes sense. Looks good to me, however could you also please verify whether the data in tasks table reflects information about the failed / killed tasks as well, as I am passing the value of that variable in the ajax request while fetching data for task tables. Thank you for catching that.

Yes @pgandhi999 . All the failed tasks are displaying in the tasks table. Thanks for your great work.

tgravescs

+1, thanks @shahidki31 !

shahidki31 · 2018-12-01T06:12:48Z

Hi @pgandhi999 , There is a small nit there. Recently we fixed a bug related to the "duration" metrics in the tasks table (see #23081), but that hasn't reflected here. I have updated the code

pgandhi999

My guess is that this might break the sorting on the Duration column in the tasks table as we are using the value of the name field in the column data to match with the appropriate name of the data field in the server data below in StagesResource:

withUI(_.store.taskList(stageId, stageAttemptId, pageStartIndex, pageLength, indexName(columnNameToSort), isAscendingStr.equalsIgnoreCase("asc"))) }
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/api/v1/StagesResource.scala#L178.
So as far as I can think, I expect that if you sort the Duration column, it will perform sorting on row.duration instead of row.taskMetrics.executorRunTime, thus, not getting the desired results.

Can you please verify whether sorting on Duration column works as expected and fix if necessary? Thank you.

pgandhi999 · 2018-12-01T07:23:14Z

- if (row.duration) {
- return type === 'display' ? formatDuration(row.duration) : row.duration;
+ if (row.taskMetrics && row.taskMetrics.executorRunTime) {
+ return type === 'display' ? formatDuration(row.taskMetrics.executorRunTime) : row.taskMetrics.executorRunTime;


Ok, that makes sense. However, you also need to update the search filter method to not search on duration in StagesResource.scala here:
|| containsValue(f.duration.getOrElse(defaultOptionString))
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/api/v1/StagesResource.scala#L213.

Thanks removed.

SparkQA · 2018-12-01T10:05:12Z

Test build #99542 has finished for PR 23160 at commit 71aff34.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-12-01T10:17:29Z

Test build #99543 has finished for PR 23160 at commit 661566b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

shahidki31 · 2018-12-01T11:14:43Z

So as far as I can think, I expect that if you sort the Duration column, it will perform sorting on row.duration instead of row.taskMetrics.executorRunTime, thus, not getting the desired results.

@pgandhi999 Actually we have mapped the "Duration" to "executorRunTime" for sorting, in the PR which I have mentioned above. So, after this PR the "Duration" is sorting correctly

spark/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala

Lines 813 to 815 in 327ac83

// SPARK-26109: Duration of task as executorRunTime to make it consistent with the
// aggregated tasks summary metrics table and the previous versions of Spark.
HEADER_DURATION->TaskIndexNames.EXEC_RUN_TIME, 

SparkQA · 2018-12-01T15:21:44Z

Test build #99549 has finished for PR 23160 at commit 5369a86.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

pgandhi999 · 2018-12-03T06:32:52Z

LGTM

shahidki31 · 2018-12-06T15:38:55Z

cc @tgravescs @srowen

srowen · 2018-12-07T20:34:16Z

Merged to master. @shahidki31 does this need to go in branch 2.4, 2.3? I tried back porting it, but looks like a lot of the affected code didn't exist in 2.4. If the fix can or should also be back-ported and you're willing, you're welcome to open another PR against 2.4.

shahidki31 · 2018-12-08T00:41:43Z

Thanks @srowen. This issue happens only in the master branch.

Thank you all for the review and comments

… is incorrect when there are failed or killed tasks and update duration metrics ## What changes were proposed in this pull request? This PR fixes 3 issues 1) Total tasks message in the tasks table is incorrect, when there are failed or killed tasks 2) Sorting of the "Duration" column is not correct 3) Duration in the aggregated tasks summary table and the tasks table and not matching. Total tasks = numCompleteTasks + numActiveTasks + numKilledTasks + numFailedTasks; Corrected the duration metrics in the tasks table as executorRunTime based on the PR apache#23081 ## How was this patch tested? test step: 1) ``` bin/spark-shell scala > sc.parallelize(1 to 100, 10).map{ x => throw new RuntimeException("Bad executor")}.collect() ``` ![screenshot from 2018-11-28 07-26-00](https://user-images.githubusercontent.com/23054875/49123523-e2691880-f2de-11e8-9c16-60d1865e6e77.png) After patch: ![screenshot from 2018-11-28 07-24-31](https://user-images.githubusercontent.com/23054875/49123525-e432dc00-f2de-11e8-89ca-4a53e19c9c18.png) 2) Duration metrics: Before patch: ![screenshot from 2018-12-06 03-25-14](https://user-images.githubusercontent.com/23054875/49546591-9e8d9900-f906-11e8-8a0b-157742c47655.png) After patch: ![screenshot from 2018-12-06 03-23-14](https://user-images.githubusercontent.com/23054875/49546589-9cc3d580-f906-11e8-827f-52ef8ffdeaec.png) Closesapache#23160 from shahidki31/totalTasks. Authored-by: Shahid <shahidki31@gmail.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>

Total tasks title in the stage page is incorrect
bbd745a

shahidki31 force-pushed the totalTasks branch from 37bcd62 to bbd745aCompare November 28, 2018 02:06

gengliangwang approved these changes Nov 30, 2018
View reviewed changes

shahidki31 changed the title ~~[SPARK-26196]Total tasks title in the stage page is incorrect when there are failed or killed tasks~~[SPARK-26196][WebUI] Total tasks title in the stage page is incorrect when there are failed or killed tasksNov 30, 2018

pgandhi999 approved these changes Nov 30, 2018
View reviewed changes

tgravescs approved these changes Nov 30, 2018
View reviewed changes

change duration as executorRunTime
661566b

shahidki31 force-pushed the totalTasks branch from 71aff34 to 661566bCompare December 1, 2018 06:14

pgandhi999 reviewed Dec 1, 2018
View reviewed changes

address comment
5369a86

shahidki31 mentioned this pull request Dec 5, 2018
[SPARK-26281][WebUI] Duration column of task table should be executor run time instead of real duration #23240
Closed

shahidki31 changed the title ~~[SPARK-26196][WebUI] Total tasks title in the stage page is incorrect when there are failed or killed tasks~~[SPARK-26196][SPARK-26281][WebUI] Total tasks title in the stage page is incorrect when there are failed or killed tasksDec 5, 2018

srowen approved these changes Dec 6, 2018
View reviewed changes

asfgit closed this in 3b8ae23Dec 7, 2018

Uh oh!

Conversation

shahidki31 commented Nov 28, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

SparkQA commented Nov 28, 2018

Uh oh!

SparkQA commented Nov 28, 2018

Uh oh!

gatorsmile commented Nov 28, 2018

Uh oh!

gengliangwang left a comment

Choose a reason for hiding this comment

Uh oh!

gengliangwang commented Nov 30, 2018

Uh oh!

pgandhi999 commented Nov 30, 2018

Uh oh!

tgravescs commented Nov 30, 2018

Uh oh!

shahidki31 commented Nov 30, 2018

Uh oh!

pgandhi999Nov 30, 2018

Choose a reason for hiding this comment

Uh oh!

shahidki31Nov 30, 2018

Choose a reason for hiding this comment

Uh oh!

tgravescs left a comment

Choose a reason for hiding this comment

Uh oh!

shahidki31 commented Dec 1, 2018

Uh oh!

pgandhi999 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pgandhi999Dec 1, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shahidki31Dec 1, 2018

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Dec 1, 2018

Uh oh!

SparkQA commented Dec 1, 2018

Uh oh!

shahidki31 commented Dec 1, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Dec 1, 2018

Uh oh!

pgandhi999 commented Dec 3, 2018

Uh oh!

shahidki31 commented Dec 6, 2018

Uh oh!

srowen commented Dec 7, 2018

Uh oh!

shahidki31 commented Dec 8, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

shahidki31 commented Nov 28, 2018•
edited
Loading

pgandhi999 left a comment •
edited
Loading

pgandhi999Dec 1, 2018•
edited
Loading

shahidki31 commented Dec 1, 2018•
edited
Loading

shahidki31 commented Dec 8, 2018•
edited
Loading