[SPARK-14434][ML]:User guide doc and examples for GaussianMixture in spark.ml by wangmiao1981 · Pull Request #12788 · apache/spark

wangmiao1981 · 2016-04-29T21:23:27Z

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

Add guide doc and examples for GaussianMixture in Spark.ml in Java, Scala and Python.

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

Manual compile and test all examples

SparkQA · 2016-04-29T21:36:15Z

Test build #57363 has finished for PR 12788 at commit 03ac9b4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wangmiao1981 · 2016-04-30T22:18:27Z

cc @yanboliang @jkbradley @MLnick @holdenk

sethah · 2016-05-04T20:03:06Z

"The MLlib" -> "The spark.ml"

sethah · 2016-05-04T20:50:40Z

I made a pass. @zhengruifeng can you please take a look as well?

sethah · 2016-05-04T20:52:07Z

outer println is unnecessary

zhengruifeng · 2016-05-05T03:02:08Z

@sethah It seems that these examples follow ml.kmeans's examples.
I agreed that it's better to keep in line with mllib's, especially the output information.

SparkQA · 2016-05-06T17:40:45Z

Test build #58009 has finished for PR 12788 at commit c844a6e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wangmiao1981 · 2016-05-06T19:18:57Z

retest it please.

SparkQA · 2016-05-06T19:30:51Z

Test build #58020 has finished for PR 12788 at commit 4b5152a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wangmiao1981 · 2016-05-06T19:48:24Z

@sethah @zhengruifeng @yanboliang I made changes to address comments.

wangmiao1981 · 2016-05-06T21:26:39Z

retest this please

SparkQA · 2016-05-06T21:36:07Z

Test build #58033 has finished for PR 12788 at commit 4b5152a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

sethah · 2016-05-09T14:48:08Z

typo: "Creates"

SparkQA · 2016-05-12T18:46:59Z

Test build #58512 has finished for PR 12788 at commit aaea542.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-05-12T21:36:08Z

Test build #58514 has finished for PR 12788 at commit 8f6f78f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wangmiao1981 · 2016-05-12T21:37:21Z

@yanboliang @MLnick @zhengruifeng @sethah simplified the example using kmeans data and addressed review comments.

SparkQA · 2016-05-12T23:40:56Z

Test build #58522 has finished for PR 12788 at commit 7175a1d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-05-12T23:45:45Z

Test build #58523 has finished for PR 12788 at commit 5259d4d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

zhengruifeng · 2016-05-13T02:14:26Z

Python API docs

zhengruifeng · 2016-05-13T02:31:12Z

nit, j -> 'i' to keep in line with scala example

SparkQA · 2016-05-13T06:15:50Z

Test build #58546 has finished for PR 12788 at commit eb9469e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wangmiao1981 · 2016-05-15T22:32:40Z

ping @MLnick @sethah @zhengruifeng @yanboliang

MLnick · 2016-05-16T06:25:19Z

@wangmiao1981 could you resolve merge conflicts by rebasing to master?

MLnick · 2016-05-16T06:27:56Z

we do setSeed here in the Python example but not in the Scala/Java examples. We should make it consistent (perhaps remove the setSeed here)

MLnick · 2016-05-16T06:31:39Z

A few other minor comments, otherwise LGTM

SparkQA · 2016-05-16T16:46:03Z

Test build #58645 has finished for PR 12788 at commit f56344b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wangmiao1981 · 2016-05-17T04:35:37Z

@MLnick I made changes accordingly. Thanks!

…spark.ml ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) Add guide doc and examples for GaussianMixture in Spark.ml in Java, Scala and Python. ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) Manual compile and test all examples Author: wm624@hotmail.com <wm624@hotmail.com> Closes#12788 from wangmiao1981/example. (cherry picked from commit 4134ff0) Signed-off-by: Nick Pentreath <nickp@za.ibm.com>

MLnick · 2016-05-17T13:22:18Z

Merged to master/branch-2.0. Thanks @sethah @zhengruifeng for the review!

sethah reviewed May 4, 2016
View reviewed changes

Comment threaddocs/ml-clustering.md Outdated
sethahMay 4, 2016
Copy link
Copy Markdown
Contributor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"The MLlib" -> "The spark.ml"

sethah reviewed May 4, 2016
View reviewed changes

wangmiao1981 force-pushed the example branch from c844a6e to 6154218Compare May 6, 2016 19:15

sethah reviewed May 9, 2016
View reviewed changes

Comment threadexamples/src/main/scala/org/apache/spark/examples/ml/GaussianMixtureExample.scala Outdated
sethahMay 9, 2016
Copy link
Copy Markdown
Contributor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: "Creates"

wangmiao1981 force-pushed the example branch from 4b5152a to aaea542Compare May 12, 2016 18:35

zhengruifeng reviewed May 13, 2016
View reviewed changes

Comment threaddocs/ml-clustering.md Outdated
zhengruifengMay 13, 2016
Copy link
Copy Markdown
Contributor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Python API docs

zhengruifeng reviewed May 13, 2016
View reviewed changes

MLnick reviewed May 16, 2016
View reviewed changes

wangmiao1981 added 12 commits May 16, 2016 09:22

add GMM scala example and document
9d157bd

add Java example and revise the document
02cace3

add python example and add python source in doc
323efeb

commit for updating upstream
179167a

address review comments
e8b7835

fix typo
91b17c4

change data and simplify the example
c8b6d23

hard code k and make copy & paste example work in pyspark shell
0625380

hard code k in java code
9392430

remove extra blank line
09407df

address review comments
f2d35dd

address review comments
f56344b

wangmiao1981 force-pushed the example branch from eb9469e to f56344bCompare May 16, 2016 16:35

asfgit closed this in 4134ff0May 17, 2016

Uh oh!

Conversation

wangmiao1981 commented Apr 29, 2016

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

SparkQA commented Apr 29, 2016

Uh oh!

wangmiao1981 commented Apr 30, 2016

Uh oh!

sethahMay 4, 2016

Choose a reason for hiding this comment

Uh oh!

sethah commented May 4, 2016

Uh oh!

sethahMay 4, 2016

Choose a reason for hiding this comment

Uh oh!

zhengruifeng commented May 5, 2016

Uh oh!

SparkQA commented May 6, 2016

Uh oh!

wangmiao1981 commented May 6, 2016

Uh oh!

SparkQA commented May 6, 2016

Uh oh!

wangmiao1981 commented May 6, 2016

Uh oh!

wangmiao1981 commented May 6, 2016

Uh oh!

SparkQA commented May 6, 2016

Uh oh!

sethahMay 9, 2016

Choose a reason for hiding this comment

Uh oh!

SparkQA commented May 12, 2016

Uh oh!

SparkQA commented May 12, 2016

Uh oh!

wangmiao1981 commented May 12, 2016

Uh oh!

SparkQA commented May 12, 2016

Uh oh!

SparkQA commented May 12, 2016

Uh oh!

zhengruifengMay 13, 2016

Choose a reason for hiding this comment

Uh oh!

zhengruifengMay 13, 2016

Choose a reason for hiding this comment

Uh oh!

SparkQA commented May 13, 2016

Uh oh!

wangmiao1981 commented May 15, 2016

Uh oh!

MLnick commented May 16, 2016

Uh oh!

MLnickMay 16, 2016

Choose a reason for hiding this comment

Uh oh!

MLnick commented May 16, 2016

Uh oh!

SparkQA commented May 16, 2016

Uh oh!

wangmiao1981 commented May 17, 2016

Uh oh!

MLnick commented May 17, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants