[SPARK-6784] [SQL] Clean up all the inbound/outbound conversions for DateType

[scwf]

/cc @yhuai and @davies

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

Test build #32301 has started for PR 6027 at commit ae30c22.

[SparkQA]

Test build #32301 has finished for PR 6027 at commit ae30c22.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Merged build finished. Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32301/
Test PASSed.

[marmbrus]

Sorry for the confusion, but this JIRA was not about adding methods for mutating data columns, but fixing places in the execution engine where you could get errors because the types are not handled correctly.

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

Test build #32319 has started for PR 6027 at commit 274b39e.

[scwf]

@marmbrus, thanks your explain. But after i go through the DateType related code i think we have converted date type to int when execution.
I added a unit test for date type support.

[marmbrus]

You are correct that most of the places have been converted but there are still some mistakes. For example, I believe date columns in partitioning columns. JDBC was also doing the conversion wrong when the ticket was opened, but I believe that has been fixed now.

[SparkQA]

Test build #32319 has finished for PR 6027 at commit 274b39e.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Merged build finished. Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32319/
Test FAILed.

[scwf]

For date columns in partitioning columns, should i start this work after #5526 is in, right?

[marmbrus]

Check with @adrian-wang who might be working on the partitioning column issue already.

[adrian-wang]

For the idea of getDate and setDate interface I think it is already rejected in #2284 . And I'm working on partitioning column already...

[scwf]

i am closing this