[SPARK-12600][SQL] Remove deprecated methods in Spark SQL by rxin · Pull Request #10559 · apache/spark

rxin · 2016-01-03T06:45:26Z

No description provided.

rxin · 2016-01-03T06:46:01Z

This is moved from DataType.scala. It is kept here so Spark 2.0 can still read parquet files written by Spark 1.1.

SparkQA · 2016-01-03T08:06:22Z

Test build #48598 has finished for PR 10559 at commit 4d7105b.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-01-04T01:09:14Z

Test build #48622 has finished for PR 10559 at commit c4e37ae.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-01-04T03:00:55Z

Test build #48623 has finished for PR 10559 at commit 4df345e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

…ed-sql

rxin · 2016-01-04T05:31:10Z

cc @davies this fixed a bug before with read.json.

SparkQA · 2016-01-04T07:38:24Z

Test build #48638 has finished for PR 10559 at commit aeaaeb4.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-01-04T07:57:31Z

Test build #48639 has finished for PR 10559 at commit ecdd18c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-01-04T08:58:37Z

Test build #48642 has finished for PR 10559 at commit f3f2821.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

marmbrus · 2016-01-04T20:11:29Z

LGTM

rxin · 2016-01-04T20:15:15Z

this one is actually useful for both R / Python, and required for data sources that don't support paths. So I'm just removing the todo to remove it.

davies · 2016-01-04T20:25:55Z

We should keep this checking somewhere

davies · 2016-01-04T21:54:37Z

LGTM except one comment.

SparkQA · 2016-01-05T00:28:51Z

Test build #48684 has finished for PR 10559 at commit 5d5092d.

This patch fails from timeout after a configured wait of 250m.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-01-05T01:49:04Z

Test build #2315 has finished for PR 10559 at commit 5d5092d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

marmbrus · 2016-01-05T02:02:27Z

To avoid conflicts I'm going to go ahead and merge this. @rxin can address comments in a followup.

Author: Reynold Xin <rxin@databricks.com> Closesapache#10559 from rxin/remove-deprecated-sql.

SparkQA · 2016-01-05T03:30:47Z

Test build #2318 has finished for PR 10559 at commit 5d5092d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

This addresses davies' code review feedback in #10559 Author: Reynold Xin <rxin@databricks.com> Closes#10586 from rxin/remove-deprecated-sql-followup.

rxin davies shivaram Took save mode from my PR #10480, and move everything to writer methods. This is related to PR #10559 - [x] it seems jsonRDD() is broken, need to investigate - this is not a public API though; will look into some more tonight. (fixed) Author: felixcheung <felixcheung_m@hotmail.com> Closes#10584 from felixcheung/rremovedeprecated.

…to branch-1.6 SPARK-13082 actually fixed by #10559. However, it's a big PR and not backported to 1.6. This PR just backported the fix of 'read.json(rdd)' to branch-1.6. Author: Shixiong Zhu <shixiong@databricks.com> Closes#10988 from zsxwing/json-rdd.

rxin reviewed Jan 3, 2016
View reviewed changes

rxin force-pushed the remove-deprecated-sql branch from 4d7105b to 5a83d42Compare January 4, 2016 00:59

rxin added 8 commits January 3, 2016 21:12

Remove deprecated methods in SQL/DataFrame.
00b08c7

Fix Java test suites.
aecbced

Fix Hive test suites.
bfc59c6

mima
31ffc1c

remove extra blank line in mima
69bbd0d

fix mima
101354b

Merge branch 'master' of github.com:apache/spark into remove-deprecat…
649934e
…ed-sql

Python.
aeaaeb4

rxin force-pushed the remove-deprecated-sql branch from 4df345e to aeaaeb4Compare January 4, 2016 05:30

rxin reviewed Jan 4, 2016
View reviewed changes

Comment threadpython/pyspark/sql/readwriter.py
rxinJan 4, 2016
Copy link
Copy Markdown
ContributorAuthor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cc @davies this fixed a bug before with read.json.

rxin added 2 commits January 3, 2016 21:39

Bring back load(path).
ecdd18c

Fix backward compatibility for schema string.
f3f2821

rxin reviewed Jan 4, 2016
View reviewed changes

Skip R tests.
5d5092d

davies reviewed Jan 4, 2016
View reviewed changes

Comment threadsql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala
daviesJan 4, 2016
Copy link
Copy Markdown
Contributor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should keep this checking somewhere

rxin changed the title ~~[SPARK-12600][SQL] Remove deprecated methods in Spark SQL - WIP~~[SPARK-12600][SQL] Remove deprecated methods in Spark SQLJan 4, 2016

felixcheung mentioned this pull request Jan 5, 2016
[SPARK-12625][SPARKR][SQL] replace R usage of Spark SQL deprecated API #10584
Closed
1 task

jongyoul pushed a commit to jongyoul/spark that referenced this pull request Jan 5, 2016
[SPARK-12600][SQL] Remove deprecated methods in Spark SQL
77ab49b
Author: Reynold Xin <rxin@databricks.com> Closesapache#10559 from rxin/remove-deprecated-sql.

rxin closed this Jan 5, 2016

rxin mentioned this pull request Jan 5, 2016
[SPARK-12600][SQL] follow up: add range check for DecimalType #10586
Closed

asfgit pushed a commit that referenced this pull request Jan 5, 2016
[SPARK-12600][SQL] follow up: add range check for DecimalType
b634901
This addresses davies' code review feedback in #10559 Author: Reynold Xin <rxin@databricks.com> Closes#10586 from rxin/remove-deprecated-sql-followup.

zsxwing mentioned this pull request Jan 29, 2016
[SPARK-13082][PySpark]Backport the fix of 'read.json(rdd)' in #10559 to branch-1.6 #10988
Closed

HyukjinKwon mentioned this pull request Mar 30, 2020
[SPARK-31290][R] Add back the deprecated R APIs #28058
Closed

Uh oh!

Conversation

rxin commented Jan 3, 2016

Uh oh!

rxinJan 3, 2016

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Jan 3, 2016

Uh oh!

SparkQA commented Jan 4, 2016

Uh oh!

SparkQA commented Jan 4, 2016

Uh oh!

rxinJan 4, 2016

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Jan 4, 2016

Uh oh!

SparkQA commented Jan 4, 2016

Uh oh!

SparkQA commented Jan 4, 2016

Uh oh!

marmbrus commented Jan 4, 2016

Uh oh!

rxinJan 4, 2016

Choose a reason for hiding this comment

Uh oh!

daviesJan 4, 2016

Choose a reason for hiding this comment

Uh oh!

davies commented Jan 4, 2016

Uh oh!

SparkQA commented Jan 5, 2016

Uh oh!

SparkQA commented Jan 5, 2016

Uh oh!

marmbrus commented Jan 5, 2016

Uh oh!

SparkQA commented Jan 5, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants