Uh oh!
There was an error while loading. Please reload this page.
[SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch - #14273
Closed
MechCoder wants to merge 7 commits into
Closed
[SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch#14273MechCoder wants to merge 7 commits into
MechCoder wants to merge 7 commits into
Conversation
MechCoder
commented
Jul 20, 2016
ContributorAuthor
ping @jkbradley@mengxr |
SparkQA
commented
Jul 20, 2016
Test build #62566 has finished for PR 14273 at commit
|
SparkQA
commented
Jul 20, 2016
Test build #62620 has finished for PR 14273 at commit
|
MechCoder
commented
Jul 28, 2016
ContributorAuthor
@jkbradley Would you be able to have a look? |
MechCoder
commented
Aug 12, 2016
ContributorAuthor
bump? |
jkbradley
commented
Feb 27, 2017
Member
Sorry about the delay here. Do you still have time to work on this? |
sethah
commented
Feb 28, 2017
Contributor
@jkbradley I do not think @MechCoder is working on Spark for the time being. |
jkbradley
commented
Feb 28, 2017
Member
OK apologies @MechCoder for the delay. I guess we can close this issue, and someone else can open up a PR based on yours. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Builds upon the work done by @hhbyyh in #7871 . This replaces all occurrences of TimeTracker with the more useful MultiStopWatch. More useful because it is possible to bench the total time across the worker nodes as well, for instance in the method
binsToBestSplitusing theDistributedStopwatch. It is also very useful to measure the optimizations in terms of time done in #13959 and should be merged before that gets reviewed. It also removes theTimeTrackersince it is not being used elsewhere except the tree module.How was this patch tested?
It was run using
setLogLevel("INFO")and the following timings are printed out.16/07/19 16:45:18 INFO RandomForest: {
binsToBestSplit: 26ms,
chooseSplits: 301ms,
findBestSplits: 307ms,
findSplitsBins: 553ms,
init: 1229ms,
total: 1572ms
}