Uh oh!
There was an error while loading. Please reload this page.
[SPARK-14704][CORE] create accumulators in TaskMetrics - #12472
Conversation
cloud-fan
commented
Apr 18, 2016
SparkQA
commented
Apr 18, 2016
Test build #56065 has finished for PR 12472 at commit
|
cloud-fan
commented
Apr 19, 2016
ready for review :) |
SparkQA
commented
Apr 19, 2016
Test build #56193 has finished for PR 12472 at commit
|
| @@ -65,7 +65,7 @@ object TaskContext { | |||
| * An empty task context that does not represent an actual task. | |||
There was a problem hiding this comment.
while you are at this, can you document this is only used for testing?
rxin
commented
Apr 19, 2016
We should be able to remove almost all the methods on InternalAccumulators.scala, shouldn't we? All that includes create, createAll, createShuffleReadAccums, ... |
SparkQA
commented
Apr 19, 2016
Test build #56204 has finished for PR 12472 at commit
|
SparkQA
commented
Apr 19, 2016
Test build #56207 has finished for PR 12472 at commit
|
rxin
commented
Apr 19, 2016
looks pretty good now |
SparkQA
commented
Apr 19, 2016
Test build #56211 has finished for PR 12472 at commit
|
SparkQA
commented
Apr 19, 2016
Test build #56227 has finished for PR 12472 at commit
|
cloud-fan
commented
Apr 20, 2016
retest this please |
SparkQA
commented
Apr 20, 2016
Test build #56300 has finished for PR 12472 at commit
|
SparkQA
commented
Apr 20, 2016
Test build #56306 has finished for PR 12472 at commit
|
SparkQA
commented
Apr 20, 2016
Test build #56310 has finished for PR 12472 at commit
|
What changes were proposed in this pull request?
Before this PR, we create accumulators at driver side(and register them) and send them to executor side, then we create
TaskMetricswith these accumulators at executor side.After this PR, we will create
TaskMetricsat driver side and send it to executor side, so that we can create accumulators insideTaskMetricsdirectly, which is cleaner.How was this patch tested?
existing tests.