Skip to content

[SPARK-24102][ML][MLLIB] ML Evaluators should use weight column - added weight column for regression evaluator - #17085

Closed
imatiach-msft wants to merge 3 commits into
apache:masterfrom
imatiach-msft:ilmat/regression-evaluate
Closed

[SPARK-24102][ML][MLLIB] ML Evaluators should use weight column - added weight column for regression evaluator#17085
imatiach-msft wants to merge 3 commits into
apache:masterfrom
imatiach-msft:ilmat/regression-evaluate

Conversation

@imatiach-msft

@imatiach-msftimatiach-msft commented Feb 27, 2017

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

The evaluators BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator and the corresponding metrics classes BinaryClassificationMetrics, RegressionMetrics and MulticlassMetrics should use sample weight data.

I've closed the PR: #16557
as recommended in favor of creating three pull requests, one for each of the evaluators (binary/regression/multiclass) to make it easier to review/update.

The updates to the regression metrics were based on (and updated with new changes based on comments):
https://issues.apache.org/jira/browse/SPARK-11520
("RegressionMetrics should support instance weights")
but the pull request was closed as the changes were never checked in.

How was this patch tested?

I added tests to the metrics class.

@imatiach-msftimatiach-msft mentioned this pull request Feb 27, 2017
Closed
@SparkQA

SparkQA commented Feb 27, 2017

Copy link
Copy Markdown

Test build #73528 has finished for PR 17085 at commit 48800eb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@imatiach-msft

imatiach-msft commented Feb 27, 2017

Copy link
Copy Markdown
ContributorAuthor

@sethah@Lewuathe@thunterdb@WeichenXu123@jkbradley@actuaryzhang@srowen would you be able to take a look? I've split the larger pull request into three parts as suggested.

@imatiach-msft

imatiach-msft commented Mar 16, 2017

Copy link
Copy Markdown
ContributorAuthor

ping @sethah@Lewuathe@thunterdb@WeichenXu123@jkbradley@actuaryzhang@srowen could you please take a look? thank you!

@imatiach-msft
imatiach-msftforce-pushed the ilmat/regression-evaluate branch from 48800eb to d5acd46CompareApril 16, 2018 16:52
@SparkQA

SparkQA commented Apr 16, 2018

Copy link
Copy Markdown

Test build #89406 has finished for PR 17085 at commit d5acd46.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@imatiach-msft
imatiach-msftforce-pushed the ilmat/regression-evaluate branch from d5acd46 to 17c1626CompareApril 16, 2018 19:03
@imatiach-msft

imatiach-msft commented Apr 16, 2018

Copy link
Copy Markdown
ContributorAuthor

Jenkins retest this please

@SparkQA

SparkQA commented Apr 16, 2018

Copy link
Copy Markdown

Test build #89414 has finished for PR 17085 at commit 17c1626.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@imatiach-msftimatiach-msft changed the title [SPARK-18693][ML][MLLIB] ML Evaluators should use weight column - added weight column for regression evaluator[SPARK-24102][ML][MLLIB] ML Evaluators should use weight column - added weight column for regression evaluatorMay 14, 2018
@imatiach-msft
imatiach-msftforce-pushed the ilmat/regression-evaluate branch from 17c1626 to aca6255CompareDecember 11, 2018 05:12
@imatiach-msft

imatiach-msft commented Dec 11, 2018

Copy link
Copy Markdown
ContributorAuthor

ping @sethah@WeichenXu123@jkbradley@actuaryzhang@srowen could you please take a look? I've updated the PR to latest and made it similar to the multiclass PR that was merged: #17086

@SparkQA

SparkQA commented Dec 11, 2018

Copy link
Copy Markdown

Test build #99947 has finished for PR 17085 at commit 0de3209.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Dec 11, 2018

Copy link
Copy Markdown

Test build #99948 has finished for PR 17085 at commit 0480721.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Dec 11, 2018

Copy link
Copy Markdown

Test build #99952 has finished for PR 17085 at commit 0cb2daf.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Dec 11, 2018

Copy link
Copy Markdown

Test build #99946 has finished for PR 17085 at commit aca6255.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

srowen
srowen reviewed Dec 11, 2018
View reviewed changes

@srowensrowen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks OK to me. What about the classification evaluators? is there the same meaningful notion of weights? I'd imagine it's possible but not sure I've seen weighted accuracy, etc.

Comment threadmllib/src/main/scala/org/apache/spark/mllib/stat/MultivariateOnlineSummarizer.scala Outdated

@srowensrowenDec 11, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I don't think the rename was necessary, but it is OK

@imatiach-msftimatiach-msftDec 11, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

@imatiach-msftimatiach-msftDec 11, 2018•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind, it looks like the build failed because the private variable conflicts with the public variable that was defined:

/**

  • Sum of weights.
    */
    override def weightSum: Double = totalWeightSum

I think this may be the best name for the public variable so I would prefer to keep it. The private variable now follows the naming convention of the other private array variables so I think this makes sense.

@imatiach-msft

imatiach-msft commented Dec 11, 2018•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

@srowen yes, exactly, there is a third PR here for classification: #17084
But I need to update it in a similar way to how I just updated this PR (eg 2.2.0 -> 3.0.0).

The original PR had all three but it was recommended that I break it up into 3 parts so I closed it and opened three separate PRs:
#16557

@imatiach-msft
imatiach-msftforce-pushed the ilmat/regression-evaluate branch from 0cb2daf to f708edbCompareDecember 11, 2018 16:11
@SparkQA

SparkQA commented Dec 11, 2018

Copy link
Copy Markdown

Test build #99982 has finished for PR 17085 at commit f708edb.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Dec 11, 2018

Copy link
Copy Markdown

Test build #99984 has finished for PR 17085 at commit 24b66da.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@srowen

srowen commented Dec 12, 2018

Copy link
Copy Markdown
Member

Merged to master

holdenk pushed a commit to holdenk/spark that referenced this pull request Jan 5, 2019
@imatiach-msft@holdenk
[SPARK-24102][ML][MLLIB] ML Evaluators should use weight column - add…
c76d70a
…ed weight column for regression evaluator
## What changes were proposed in this pull request?
The evaluators BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator and the corresponding metrics classes BinaryClassificationMetrics, RegressionMetrics and MulticlassMetrics should use sample weight data.
I've closed the PR: apache#16557
as recommended in favor of creating three pull requests, one for each of the evaluators (binary/regression/multiclass) to make it easier to review/update.
The updates to the regression metrics were based on (and updated with new changes based on comments):
https://issues.apache.org/jira/browse/SPARK-11520
("RegressionMetrics should support instance weights")
but the pull request was closed as the changes were never checked in.
## How was this patch tested?
I added tests to the metrics class.
Closesapache#17085 from imatiach-msft/ilmat/regression-evaluate.
Authored-by: Ilya Matiach <ilmat@microsoft.com>
Signed-off-by: Sean Owen <sean.owen@databricks.com>
jackylee-ch pushed a commit to jackylee-ch/spark that referenced this pull request Feb 18, 2019
@imatiach-msftJackey Lee
[SPARK-24102][ML][MLLIB] ML Evaluators should use weight column - add…
277f42a
…ed weight column for regression evaluator
## What changes were proposed in this pull request?
The evaluators BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator and the corresponding metrics classes BinaryClassificationMetrics, RegressionMetrics and MulticlassMetrics should use sample weight data.
I've closed the PR: apache#16557
as recommended in favor of creating three pull requests, one for each of the evaluators (binary/regression/multiclass) to make it easier to review/update.
The updates to the regression metrics were based on (and updated with new changes based on comments):
https://issues.apache.org/jira/browse/SPARK-11520
("RegressionMetrics should support instance weights")
but the pull request was closed as the changes were never checked in.
## How was this patch tested?
I added tests to the metrics class.
Closesapache#17085 from imatiach-msft/ilmat/regression-evaluate.
Authored-by: Ilya Matiach <ilmat@microsoft.com>
Signed-off-by: Sean Owen <sean.owen@databricks.com>
@imatiach-msftimatiach-msft mentioned this pull request Mar 25, 2019
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@srowensrowensrowen approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@imatiach-msft@SparkQA@srowen