[SPARK-25785][SQL] Add prettyNames for from_json, to_json, from_csv, and schema_of_json

[HyukjinKwon]

What changes were proposed in this pull request?

This PR adds prettyNames for from_json, to_json, from_csv, and schema_of_json so that appropriate names are used.

How was this patch tested?

Unit tests

[SparkQA]

Test build #97591 has finished for PR 22773 at commit 3447e73.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

retest this please

[SparkQA]

Test build #97599 has finished for PR 22773 at commit 3447e73.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

retest this please.

[viirya]

LGTM

[SparkQA]

Test build #97604 has finished for PR 22773 at commit 3447e73.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

LG, but could you make this as a follow-up for the existing issues or create a new one?

[HyukjinKwon]

Other JIRAs have different fixed versions. Let me create a new JIRA then.

[HyukjinKwon]

Merged to master.

[HyukjinKwon]

Thank you @viirya and @dongjoon-hyun.

[gatorsmile]

This is an external change. This needs a migration guide update. Column names in schema are changed by this PR

[HyukjinKwon]

That's the exact issue I raised before and we ended up with not keeping the compatibility in column names. @cloud-fan and @hvanhovell.

[HyukjinKwon]

BTW, it's closer to bug rather then improvement tho. from_json should have default name from_json rather then jsontostructs - end users would have no idea why it's called jsontostructs.

[cloud-fan]

I think the new names are better and expected, though it's safer to mention it in the migration guide in case some users care about it.

[HyukjinKwon]

My impression so far was that we note things at migration notes when they are improvements (not bugs), and non-trivial and related to backward compatibility.

Shall we clarify what to document at migration guide? Otherwise we should document everything related with, let's say, all external changes, deprecation removal, trivial changes, and many other changes related with column names.

[gatorsmile]

All the things that could break the existing user applications should be documented in the migration guide. This will simplify the system upgrade of our end users.

[HyukjinKwon]

Sure, so for clarification, we will document everything that could break external users applications in any way, right?

[gatorsmile]

Yes. The goal of migration guide is for helping end users upgrade their Spark.

[HyukjinKwon]

Yup, will encourage to update the migration guide in that way.