Uh oh!
There was an error while loading. Please reload this page.
[SPARK-24803][SQL] add support for numeric - #21766
Conversation
AmplabJenkins
commented
Jul 14, 2018
Can one of the admins verify this patch? |
maropu
commented
Jul 14, 2018
@gatorsmile Is it worth adding a numeric type as an alias of decimal? Both types seems to be in the SQL standard and some dbms (e.g., postgresql and sql server) can parse both. |
Although MS SQL Server looks like use them interchangeably, https://docs.microsoft.com/en-us/sql/t-sql/data-types/decimal-and-numeric-transact-sql?view=sql-server-2017, they are not exactly the same in ANSI SQL.
|
maropu
commented
Jul 15, 2018
aha, I didn't know that and postgresql also uses them interchangeably, too. |
dmateusp
commented
Jul 15, 2018
Just checked out the PR, scala> spark.sql("SELECT CAST(1 as NUMERIC)")
res0: org.apache.spark.sql.DataFrame= [CAST(1ASDECIMAL(10,0)): decimal(10,0)]
scala> spark.sql("SELECT NUMERIC(1)")
org.apache.spark.sql.AnalysisException:Undefinedfunction: 'NUMERIC'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.; line 1 pos 7I imagine some tests could be added here:
Do you think it's worth having a separate DataType or just have it as an alias? |
rxin
commented
Jul 16, 2018
Why did you need this change? Given it's very difficult to revert the change (or introduce a proper numeric type if ever needed in the future), I would not merge this pull request unless there are sufficient justification. |
wangtao605
commented
Jul 17, 2018
@dmateusp Actually i think "Numeric" has no essential difference with "Decimal". May be just have it as an alias is better,i will add some tests if you agree. |
wangtao605
commented
Jul 17, 2018
@rxin In order to support sql syntax better and align SQL standards. I think it is worth to add a numeric type as an alias of decimal. |
gatorsmile
commented
Nov 11, 2018
@wangtao605 Do you mind documenting our behavior in our Spark SQL doc? |
wangtao605
commented
Nov 12, 2018
Yes, it's ok. |
Closesapache#21766Closesapache#21679Closesapache#21161Closesapache#20846Closesapache#19434Closesapache#18080Closesapache#17648Closesapache#17169 Add: Closesapache#22813Closesapache#21994Closesapache#22005Closesapache#22463 Add: Closesapache#15899 Add: Closesapache#22539Closesapache#21868Closesapache#21514Closesapache#21402Closesapache#21322Closesapache#21257Closesapache#20163Closesapache#19691Closesapache#18697Closesapache#18636Closesapache#17176Closesapache#23001 from wangyum/CloseStalePRs. Authored-by: Yuming Wang <yumwang@ebay.com> Signed-off-by: hyukjinkwon <gurwls223@apache.org>
What changes were proposed in this pull request?
numerical is as same with decimal. Spark has already supported decimal,so i think we should add support for numeric to align SQL standards.
(Please fill in changes proposed in this fix)
How was this patch tested?
already exist tests
Please review http://spark.apache.org/contributing.html before opening a pull request.