[SPARK-5012][MLLib][PySpark]Python API for Gaussian Mixture Model by FlytxtRnD · Pull Request #4059 · apache/spark

FlytxtRnD · 2015-01-15T12:12:16Z

Python API for the Gaussian Mixture Model clustering algorithm in MLLib.

AmplabJenkins · 2015-01-15T12:17:09Z

Can one of the admins verify this patch?

mengxr · 2015-01-15T19:05:13Z

add to whitelist

SparkQA · 2015-01-15T19:07:43Z

Test build #25609 has started for PR 4059 at commit 5c83825.

This patch merges cleanly.

SparkQA · 2015-01-15T19:08:37Z

Test build #25609 has finished for PR 4059 at commit 5c83825.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class GaussianMixtureModel(object):
- class GaussianMixtureEM(object):

AmplabJenkins · 2015-01-15T19:08:38Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25609/
Test FAILed.

jkbradley · 2015-01-15T20:36:14Z

@FlytxtRnD Are you still running into the py4j serialization issue you mentioned on the JIRA?

FlytxtRnD · 2015-01-16T04:05:36Z

@jkbradley py4j serialization issue has been solved by the commit 8ead999

SparkQA · 2015-01-16T04:07:34Z

Test build #25634 has started for PR 4059 at commit f82750b.

This patch merges cleanly.

SparkQA · 2015-01-16T05:05:36Z

Test build #25634 has finished for PR 4059 at commit f82750b.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class GaussianMixtureModel(object):
- class GaussianMixtureEM(object):

AmplabJenkins · 2015-01-16T05:05:40Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25634/
Test FAILed.

…ter' into PythonGmmWrapper

…nGmmWrapper' into PythonGmmWrapper Conflicts: examples/src/main/python/mllib/gaussian_mixture_model.py

SparkQA · 2015-01-16T08:42:41Z

Test build #25648 has started for PR 4059 at commit c1d4c71.

This patch merges cleanly.

SparkQA · 2015-01-16T09:51:36Z

Test build #25648 has finished for PR 4059 at commit c1d4c71.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class GaussianMixtureModel(object):
- class GaussianMixtureEM(object):

AmplabJenkins · 2015-01-16T09:51:40Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25648/
Test PASSed.

…ter' into PythonGmmWrapper

mengxr · 2015-01-22T18:57:48Z

separate spark imports from python imports

FlytxtRnD · 2015-01-23T04:31:49Z

@mengxr Thank you for the review and comments. I am changing the code according to #3923 (tgaloppo).

SparkQA · 2015-02-02T10:22:47Z

Test build #26509 has started for PR 4059 at commit d5b36ab.

This patch merges cleanly.

SparkQA · 2015-02-02T11:26:23Z

Test build #26509 has finished for PR 4059 at commit d5b36ab.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class GaussianMixtureModel(object):
- class GaussianMixture(object):
- class MultivariateGaussian(object):

AmplabJenkins · 2015-02-02T11:26:27Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26509/
Test FAILed.

FlytxtRnD · 2015-02-02T11:59:18Z

Is it possible to start a test build in Jenkins without updating the PR? TestResults shows "no failures" but the console output shows errors. All tests were passed when ran locally.

SparkQA · 2015-02-02T12:22:36Z

Test build #26515 has started for PR 4059 at commit fa0a142.

This patch merges cleanly.

SparkQA · 2015-02-02T13:33:35Z

Test build #26515 has finished for PR 4059 at commit fa0a142.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-02-02T13:33:39Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26515/
Test PASSed.

FlytxtRnD · 2015-02-02T13:38:59Z

Please review and merge..

mengxr · 2015-02-02T17:56:58Z

The default seed is not generated. If users don't specify seed on the Python side, it will pass in as null on the JVM side.
if (seed != null) gmmAlg.setSeed(seed)

In python side I have added a default value None ..So is it required to add this statement

…nMixture

…ter' into PythonGmmWrapper

SparkQA · 2015-02-03T05:37:29Z

Test build #26607 has started for PR 4059 at commit c973ab3.

This patch merges cleanly.

SparkQA · 2015-02-03T06:43:24Z

Test build #26607 has finished for PR 4059 at commit c973ab3.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class GaussianMixtureModel(object):
- class GaussianMixture(object):
- class MultivariateGaussian(namedtuple('MultivariateGaussian', ['mu', 'sigma'])):

AmplabJenkins · 2015-02-03T06:43:27Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26607/
Test PASSed.

mengxr · 2015-02-03T07:05:19Z

LGTM. Merged into master. Thanks for adding GMM Python API!

FlytxtRnD · 2015-02-03T07:06:17Z

Thanks @mengxr for your help.

FlytxtRnD added 2 commits January 15, 2015 17:30

Python API for Gaussian Mixture Model
fda60f3

Split input file with space delimiter
5c83825

Fixed style issues
f82750b

FlytxtRnD added 3 commits January 16, 2015 12:00

Merge branch 'PythonGmmWrapper', remote-tracking branch 'upstream/mas…
332bad1
…ter' into PythonGmmWrapper

Added random seed parameter
426d130

Merge branch 'PythonGmmWrapper', remote-tracking branch 'origin/Pytho…
c1d4c71
…nGmmWrapper' into PythonGmmWrapper Conflicts: examples/src/main/python/mllib/gaussian_mixture_model.py

Merge branch 'PythonGmmWrapper', remote-tracking branch 'upstream/mas…
3464d19
…ter' into PythonGmmWrapper

mengxr reviewed Jan 22, 2015
View reviewed changes

Comment threadexamples/src/main/python/mllib/gaussian_mixture_model.py Outdated
mengxrJan 22, 2015
Copy link
Copy Markdown
Contributor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
separate spark imports from python imports

Changed argument names to lowercase
d5b36ab

New line added
fa0a142

mengxr reviewed Feb 2, 2015
View reviewed changes

FlytxtRnD added 2 commits February 3, 2015 10:46

Added MultivariateGaussian namedtuple and Arraybuffer in trainGaussia…
339b09c
…nMixture

Merge branch 'PythonGmmWrapper', remote-tracking branch 'upstream/mas…
c973ab3
…ter' into PythonGmmWrapper

asfgit closed this in 50a1a87Feb 3, 2015

Uh oh!

Conversation

FlytxtRnD commented Jan 15, 2015

Uh oh!

AmplabJenkins commented Jan 15, 2015

Uh oh!

mengxr commented Jan 15, 2015

Uh oh!

SparkQA commented Jan 15, 2015

Uh oh!

SparkQA commented Jan 15, 2015

Uh oh!

AmplabJenkins commented Jan 15, 2015

Uh oh!

jkbradley commented Jan 15, 2015

Uh oh!

FlytxtRnD commented Jan 16, 2015

Uh oh!

SparkQA commented Jan 16, 2015

Uh oh!

SparkQA commented Jan 16, 2015

Uh oh!

AmplabJenkins commented Jan 16, 2015

Uh oh!

SparkQA commented Jan 16, 2015

Uh oh!

SparkQA commented Jan 16, 2015

Uh oh!

AmplabJenkins commented Jan 16, 2015

Uh oh!

mengxrJan 22, 2015

Choose a reason for hiding this comment

Uh oh!

FlytxtRnD commented Jan 23, 2015

Uh oh!

SparkQA commented Feb 2, 2015

Uh oh!

SparkQA commented Feb 2, 2015

Uh oh!

AmplabJenkins commented Feb 2, 2015

Uh oh!

FlytxtRnD commented Feb 2, 2015

Uh oh!

SparkQA commented Feb 2, 2015

Uh oh!

SparkQA commented Feb 2, 2015

Uh oh!

AmplabJenkins commented Feb 2, 2015

Uh oh!

FlytxtRnD commented Feb 2, 2015

Uh oh!

mengxrFeb 2, 2015

Choose a reason for hiding this comment

Uh oh!

FlytxtRnDFeb 3, 2015

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Feb 3, 2015

Uh oh!

SparkQA commented Feb 3, 2015

Uh oh!

AmplabJenkins commented Feb 3, 2015

Uh oh!

mengxr commented Feb 3, 2015

Uh oh!

FlytxtRnD commented Feb 3, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants