Uh oh!
There was an error while loading. Please reload this page.
[SPARK-6685][ML]Use DSYRK to compute AtA in ALS - #19536
Conversation
set stack size > 128 and comments added
mpjlu
commented
Oct 19, 2017
Hi @yanboliang , I reopen this PR per your suggestion, thanks. |
hqzizania
commented
Oct 19, 2017
Wow, thank you for reopening. LOL @mpjlu |
| def setColdStartStrategy(value: String): this.type = set(coldStartStrategy, value) | ||
| @Since("2.3.0") | ||
| def setThreshold(value: Int): this.type = set(threshold, value) |
There was a problem hiding this comment.
I'm not sure callers can meaningfully understand and set this. Can't we pick a threshold programmatically?
There was a problem hiding this comment.
Yes, my test result is better than the previous result, especially for native BLAS. I will update my test results here soon, and I will change this set.
| ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.linalg.Vector.toSparseWithSize") | ||
| ) ++ Seq( | ||
| // [SPARK-6685][ML]Use DSYRK to compute AtA in ALS | ||
| ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.recommendation.ALS.train") |
There was a problem hiding this comment.
This breaks an API unnecessarily, even though it's a dev API. I think we instead need to remove this as a user-facing param and avoid it altogether.
There was a problem hiding this comment.
I am ok to remove this, and use a loose threshold (e.g. 100), which is helpful for most cases. How about it?
SparkQA
commented
Oct 19, 2017
Test build #82905 has finished for PR 19536 at commit
|
mpjlu
commented
Oct 24, 2017
This is the performance data of this PR. |
mpjlu
commented
Oct 24, 2017
cc @srowen@mengxr@yanboliang |
mpjlu
commented
Jan 15, 2018
Because I don't have the environment to continue this work, I will close it. Thanks. |
What changes were proposed in this pull request?
This is a reopen of PR: #13891 with mima fix and code optimization.
There is about 30-40% performance improvement comparing with #13891
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.