Uh oh!
There was an error while loading. Please reload this page.
[SPARK-14516][FOLLOWUP] Adding ClusteringEvaluator to examples - #19676
[SPARK-14516][FOLLOWUP] Adding ClusteringEvaluator to examples#19676mgaido91 wants to merge 2 commits into
Conversation
SparkQA
commented
Nov 6, 2017
Test build #83500 has finished for PR 19676 at commit
|
mgaido91
commented
Nov 10, 2017
sorry for pinging you, what do you think about adding |
yanboliang
commented
Dec 7, 2017
It's good to have this, sorry for late response, I will make a pass tomorrow. Thanks. |
yanboliang
left a comment
There was a problem hiding this comment.
LGTM except one minor comment. Thanks.
| // Evaluate clustering by computing Silhouette score | ||
| ClusteringEvaluator evaluator = new ClusteringEvaluator() | ||
| .setFeaturesCol("features") | ||
| .setPredictionCol("prediction") |
There was a problem hiding this comment.
We use default values here, so it's not necessary to set them explicitly. We should keep examples as simple as possible. Thanks.
SparkQA
commented
Dec 9, 2017
Test build #84681 has finished for PR 19676 at commit
|
| ClusteringEvaluator evaluator = new ClusteringEvaluator(); | ||
| double silhouette = evaluator.evaluate(predictions); | ||
| System.out.println("Silhouette with squared euclidean distance = " + silhouette); |
There was a problem hiding this comment.
euclidean -> Euclidean, but not important to change unless you're touching the code again anyway
There was a problem hiding this comment.
Thanks, I don't think I am changing the code again, but I can fix this grammatical error if you want.
srowen
commented
Dec 11, 2017
Merged to master |
What changes were proposed in this pull request?
In SPARK-14516 we have introduced ClusteringEvaluator, but we didn't put any reference in the documentation and the examples were still relying on the sum of squared errors to show a way to evaluate the clustering model.
The PR adds the ClusteringEvaluator in the examples.
How was this patch tested?
Manual runs of the examples.