Skip to content

[SPARK-29840][SQL] PostgreSQL dialect: cast to integer - #26481

Closed
amanomer wants to merge 6 commits into
apache:masterfrom
amanomer:29840
Closed

[SPARK-29840][SQL] PostgreSQL dialect: cast to integer#26481
amanomer wants to merge 6 commits into
apache:masterfrom
amanomer:29840

Conversation

@amanomer

@amanomeramanomer commented Nov 12, 2019

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

To make SparkSQL's cast as int behavior consistent with PostgreSQL when spark.sql.dialect is configured as PostgreSQL.

Why are the changes needed?

SparkSQL and PostgreSQL have a lot different cast behavior between types by default. We should make SparkSQL's cast behavior be consistent with PostgreSQL when spark.sql.dialect is configured as PostgreSQL.

Does this PR introduce any user-facing change?

Yes. If user switches to PostgreSQL dialect now, they will

  • get an AnalysisException when they try to cast ByteType, TimestampType or DateType to IntegerType

How was this patch tested?

Added test cases.

@amanomeramanomer changed the title [WIP][SPARK-29838][SQL] PostgreSQL dialect: cast to integer[WIP][SPARK-29840][SQL] PostgreSQL dialect: cast to integerNov 12, 2019
@amanomeramanomer changed the title [WIP][SPARK-29840][SQL] PostgreSQL dialect: cast to integer[SPARK-29840][SQL] PostgreSQL dialect: cast to integerNov 14, 2019
@amanomer

Copy link
Copy Markdown
ContributorAuthor

cc @cloud-fan

@amanomer

Copy link
Copy Markdown
ContributorAuthor

Retest this please

@maropu

Copy link
Copy Markdown
Member

memo: pending until #26472 merged.

@dongjoon-hyun

Copy link
Copy Markdown
Member

Thank you for contribution, @amanomer . As you know, unfortunately, we decided to remove PostgreSQL dialect via SPARK-30125 (#26763). Sorry about that. I'll close this PR, too.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@amanomer@maropu@dongjoon-hyun