Skip to content

[SPARK-28129][SQL][TEST] Port float8.sql - #24931

Closed
wangyum wants to merge 7 commits into
apache:masterfrom
wangyum:SPARK-28129
Closed

[SPARK-28129][SQL][TEST] Port float8.sql#24931
wangyum wants to merge 7 commits into
apache:masterfrom
wangyum:SPARK-28129

Conversation

@wangyum

@wangyumwangyum commented Jun 21, 2019•
edited by dongjoon-hyun
Loading

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR is to port float8.sql from PostgreSQL regression tests. https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/float8.sql

The expected results can be found in the link: https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/expected/float8.out

When porting the test cases, found six PostgreSQL specific features that do not exist in Spark SQL:
SPARK-28060: Double type can not accept some special inputs
SPARK-28027: Spark SQL does not support prefix operator @ and |/
SPARK-28061: Support for converting float to binary format
SPARK-23906: Support Truncate number
SPARK-28134: Missing Trigonometric Functions

Also, found two bug:
SPARK-28024: Incorrect value when out of range
SPARK-28135: ceil/ceiling/floor/power returns incorrect values

Also, found four inconsistent behavior:
SPARK-27923: Spark SQL insert bad inputs to NULL
SPARK-28028: Cast numeric to integral type need round
SPARK-27923: Spark SQL returns NULL when dividing by zero
SPARK-28007: Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres

How was this patch tested?

N/A

@SparkQA

SparkQA commented Jun 21, 2019

Copy link
Copy Markdown

Test build #106761 has finished for PR 24931 at commit 46a300e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Jul 10, 2019

Copy link
Copy Markdown

Test build #107441 has finished for PR 24931 at commit 4b4551f.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@wangyum

wangyum commented Jul 10, 2019

Copy link
Copy Markdown
MemberAuthor

retest this please

@SparkQA

SparkQA commented Jul 10, 2019

Copy link
Copy Markdown

Test build #107444 has finished for PR 24931 at commit 4b4551f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

dongjoon-hyun
dongjoon-hyun reviewed Jul 10, 2019
View reviewed changes
Comment threadsql/core/src/test/resources/sql-tests/inputs/pgSQL/float8.sql
dongjoon-hyun
dongjoon-hyun reviewed Jul 10, 2019
View reviewed changes
Comment threadsql/core/src/test/resources/sql-tests/inputs/pgSQL/float8.sql
dongjoon-hyun
dongjoon-hyun reviewed Jul 10, 2019
View reviewed changes
Comment threadsql/core/src/test/resources/sql-tests/inputs/pgSQL/float8.sql
dongjoon-hyun
dongjoon-hyun reviewed Jul 10, 2019
View reviewed changes
Comment threadsql/core/src/test/resources/sql-tests/inputs/pgSQL/float8.sql
dongjoon-hyun
dongjoon-hyun reviewed Jul 10, 2019
View reviewed changes
Comment threadsql/core/src/test/resources/sql-tests/inputs/pgSQL/float8.sql
dongjoon-hyun
dongjoon-hyun reviewed Jul 10, 2019
View reviewed changes
Comment threadsql/core/src/test/resources/sql-tests/inputs/pgSQL/float8.sql Outdated
@dongjoon-hyundongjoon-hyun mentioned this pull request Jul 10, 2019
Closed
@dongjoon-hyun

dongjoon-hyun commented Jul 15, 2019

Copy link
Copy Markdown
Member

Hi, @wangyum .
Since #25041 is merged, could you update this PR once more? Thanks!

@wangyum

wangyum commented Jul 15, 2019

Copy link
Copy Markdown
MemberAuthor

Thank you @dongjoon-hyun . I will update it later.

wangyum
wangyum commented Jul 15, 2019
View reviewed changes
Comment threadsql/core/src/test/resources/sql-tests/inputs/pgSQL/float8.sql Outdated
@SparkQA

SparkQA commented Jul 15, 2019

Copy link
Copy Markdown

Test build #107679 has finished for PR 24931 at commit cd5d695.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

dongjoon-hyun
dongjoon-hyun reviewed Jul 16, 2019
View reviewed changes
Comment threadsql/core/src/test/resources/sql-tests/inputs/pgSQL/float8.sql
SELECT double('Infinity') + 100.0;
-- [SPARK-27768] Infinity, -Infinity, NaN should be recognized in a case insensitive manner
SELECT double('Infinity') / double('Infinity');
SELECT double('NaN') / double('NaN');

@dongjoon-hyundongjoon-hyunJul 16, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a comment SPARK-28315 before line 55?

dongjoon-hyun
dongjoon-hyun approved these changes Jul 16, 2019
View reviewed changes

@dongjoon-hyundongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM (except one minor comment request).
Most differences are filed correctly. Thank you so much, @wangyum .

@SparkQA

SparkQA commented Jul 16, 2019

Copy link
Copy Markdown

Test build #107713 has finished for PR 24931 at commit 532486a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Jul 16, 2019

Copy link
Copy Markdown

Test build #107721 has finished for PR 24931 at commit f49cf43.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@wangyum

wangyum commented Jul 16, 2019

Copy link
Copy Markdown
MemberAuthor

retest this please

@SparkQA

SparkQA commented Jul 16, 2019

Copy link
Copy Markdown

Test build #107725 has finished for PR 24931 at commit f49cf43.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon

HyukjinKwon commented Jul 16, 2019

Copy link
Copy Markdown
Member

Merged to master.

@wangyum
wangyum deleted the SPARK-28129 branch July 16, 2019 10:36
vinodkc pushed a commit to vinodkc/spark that referenced this pull request Jul 18, 2019
@wangyum@vinodkc
[SPARK-28129][SQL][TEST] Port float8.sql
ec5b825
## What changes were proposed in this pull request?
This PR is to port float8.sql from PostgreSQL regression tests. https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/float8.sql
The expected results can be found in the link: https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/expected/float8.out
When porting the test cases, found six PostgreSQL specific features that do not exist in Spark SQL:
[SPARK-28060](https://issues.apache.org/jira/browse/SPARK-28060): Double type can not accept some special inputs
[SPARK-28027](https://issues.apache.org/jira/browse/SPARK-28027): Spark SQL does not support prefix operator `` and `|/`
[SPARK-28061](https://issues.apache.org/jira/browse/SPARK-28061): Support for converting float to binary format
[SPARK-23906](https://issues.apache.org/jira/browse/SPARK-23906): Support Truncate number
[SPARK-28134](https://issues.apache.org/jira/browse/SPARK-28134): Missing Trigonometric Functions
Also, found two bug:
[SPARK-28024](https://issues.apache.org/jira/browse/SPARK-28024): Incorrect value when out of range
[SPARK-28135](https://issues.apache.org/jira/browse/SPARK-28135): ceil/ceiling/floor/power returns incorrect values
Also, found four inconsistent behavior:
[SPARK-27923](https://issues.apache.org/jira/browse/SPARK-27923): Spark SQL insert bad inputs to NULL
[SPARK-28028](https://issues.apache.org/jira/browse/SPARK-28028): Cast numeric to integral type need round
[SPARK-27923](https://issues.apache.org/jira/browse/SPARK-27923): Spark SQL returns NULL when dividing by zero
[SPARK-28007](https://issues.apache.org/jira/browse/SPARK-28007): Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres
## How was this patch tested?
N/A
Closesapache#24931 from wangyum/SPARK-28129.
Authored-by: Yuming Wang <yumwang@ebay.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dongjoon-hyundongjoon-hyundongjoon-hyun approved these changes

Assignees

No one assigned

Labels

SQLTESTS

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@wangyum@SparkQA@dongjoon-hyun@HyukjinKwon