SPARK-7137 [ML] : Update SchemaUtils checkInputColumn to print more info if needed

[rekhajoshm]

No description provided.

[mengxr]

@rekhajoshm This was moved to ml.util.SchemaUtils. If you want to output more information, please modify it there.

[rekhajoshm]

hi @mengxr hmm., moving getParam out of Params or to modify its visibility for ${getParam(colName)? Please can you also check with @josephbradley of SPARK-7137 on the PR requirement?

[jkbradley]

@mengxr My motivation for creating SPARK-7137 was to print more help info when schema checks fail. Currently, we say something like:

Column features must be of type StringType but was actually DoubleType.

It would be nice to print more info, particularly which PipelineStage caused the problem and which parameter that column is specified by:

... This column is specified via parameter "inputCol" from "lr_32983" with description: ...

[jkbradley]

After discussing with @mengxr it sounds like the best thing to do will be to add an extra argument to checkInputColumn msg: String, with default value empty string. Then we can pass the extra info as needed. Could you please modify this PR so it does not move the method, but instead adds the extra argument? Part of our motivation is that we eventually want these SchemaUtils to be part of DataFrames, rather than MLlib.

Also, let's not yet modify the uses of checkInputColumn. The reason is that I want to create abstractions for input/output columns, and then those abstractions can handle the schema check internally. I'll create JIRAs for that, to be done after this PR.

Thanks!

[rekhajoshm]

thanks @jkbradley@mengxr for discussion and quick inputs.done.please review.thanks.

[jkbradley]

style: Please make sure you follow the style guide linked from [https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark]

If this fits on 1 line (<= 100 chars), then it should. If not, then there should be 1 param per line.

[rekhajoshm]

thanks @jkbradley for your review.done.please have a look.thanks.

[mengxr]

ok to test

[SparkQA]

Test build #36414 has finished for PR 5992 at commit 33ddd2e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[mengxr]

Please follow the Spark code style guide and use 4-space indentation for arguments: https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide

[rekhajoshm]

thanks @mengxr for your comment.done.please have a look.Also raised SPARK-8806, please check if that makes sense?thanks

[SparkQA]

Test build #36453 has finished for PR 5992 at commit 8c42b57.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

LGTM merging with master
Thanks!

[rekhajoshm]

thanks @jkbradley@mengxr for the reviews and merge.