[SPARK-9312] [ML] Added max confidence factor to OneVsRestModel

[badriub]

1. I've declared a new method predictPointWithProbability in GeneralizedLinearModel and implemented the method only for LogisticRegressionModel.
2. I calculate the probability using the odds ratio.
3. For the other implementations of GeneralizedLinearModel, I have added an "not implemented exception". I can look into how to calculate probability if needed.

WRT OVR,
I've extracted the rawProbability as the maxProbability of all classes and added a new column in the returned dataframe.
I've fixed the test to include the new column "rawProbability"

[srowen]

Without commenting on the suitability of the PR -- I can at least say that throw new Exception is never a great idea. Use UnsupportedOperationException here, for example.

[jkbradley]

@badriub I just added some thoughts on the JIRA. Could you please take a look and respond there? Thanks!

[badriub]

I have replied on the JIRA, please take a look. Thanks.

[badriub]

Used the Intellj importer plugin as recommended on the style guide.

[jkbradley]

You may need to adjust the import organizer settings, though. The order should be Java, Scala, other, Spark.

[badriub]

@jkbradley : I've reverted the changes for LogisticRegression (for entire class hierarchy) since they were not needed. I've updated JIRA and PR as discussed.
@srowen : I've used the importer plugin and made sure I follow the style guide.

[jkbradley]

It will be easier for you if you have OneVsRestParams extend ClassifierParams. That will include the rawPredictionCol, plus the updated validateAndTransformSchema method.

[jkbradley]

That should do it for a first pass. I'll check back for updates. Thanks!
Also, can you please update:

• the PR description (the first comment in this PR) to match the more recent changes, and
• the title to use the tag "ml" instead of "mllib"?

[viktortnk]

@jkbradley@badriub Any updates on this issue?

From diff I see that rawPrediction has Double type. Does it make sense to return an Array of values from each binary classifier?

I'm happy to help if required to get it through

[AxenGitHub]

Is there any news on this branch? we would benefit a lot from this feature.

[jkbradley]

Hi, sorry I let this PR get stale. This should be resolved now by #21044 so would you mind closing this issue @badriub ? Thanks though!

[AmplabJenkins]

Can one of the admins verify this patch?