Uh oh!
There was an error while loading. Please reload this page.
[SPARK-29596][Web UI] Task duration not updating for running tasks - #27026
[SPARK-29596][Web UI] Task duration not updating for running tasks#27026Le-Dai wants to merge 2 commits into
Conversation
Ngone51
left a comment
There was a problem hiding this comment.
Would you please explain more about why task duration doesn't update in UI?
Besides, does this problem only exists in branch-2.4? Because I also see the same code in master branch.
| </div> | ||
| </td> | ||
| <td>{UIUtils.formatDate(task.launchTime)}</td> | ||
| <td>{formatDuration(task.taskMetrics.map(_.executorRunTime))}</td> |
There was a problem hiding this comment.
Hi @MrDLontheway , please note that this change is intentionally introduced in #23081.
Le-Dai
commented
Dec 28, 2019
Thank you for your reminder, during the task running process, because the executive run time is not updated, the duration will not be updated. I will seek other solutions |
Ngone51
commented
Dec 28, 2019
I know that executor run time doesn't update according to your description, but I also want to know the root cause why it doesn't change. |
gengliangwang
commented
Jan 3, 2020
@MrDLontheway could you provide more details and screenshot for the issue? |
Le-Dai
commented
Jan 3, 2020
|
planga82
left a comment
There was a problem hiding this comment.
I'm not sure about this change. There are a lot of more information that is used from metrics to create TaskDataWrapper, so, the other metrics are not updated? If this is the case the problem could be we are not updating metrics in the right way and we have to change it and not this specific metric only.
| var duration: Long = 0 | ||
| var executorRunTime: Long = 0 | ||
| if (info.finished) { |
There was a problem hiding this comment.
vars are not very cool in scala... how about something like this?
val (duration, executorRunTime) = if (info.finished) {
(info.duration, metrics.executorRunTime)
} else {
val timeRunning = info.timeRunning(lastUpdateTime.getOrElse(System.currentTimeMillis()))
(timeRunning, timeRunning)
}
holdenk
commented
Apr 27, 2020
Jenkins ok to test |
SparkQA
commented
Apr 28, 2020
Test build #121933 has finished for PR 27026 at commit
|
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
What changes were proposed in this pull request?
Changed the source of web ui duration information
Why are the changes needed?
Task duration not updating for running tasks
Does this PR introduce any user-facing change?
no
How was this patch tested?
cat web ui