Skip to content

[SPARK-5274][SQL] Reconcile Java and Scala UDFRegistration. - #4056

Closed
rxin wants to merge 5 commits into
apache:masterfrom
rxin:udf-registration
Closed

[SPARK-5274][SQL] Reconcile Java and Scala UDFRegistration.#4056
rxin wants to merge 5 commits into
apache:masterfrom
rxin:udf-registration

Conversation

@rxin

@rxinrxin commented Jan 15, 2015

Copy link
Copy Markdown
Contributor

As part of SPARK-5193:

  1. Removed UDFRegistration as a mixin in SQLContext and made it a field ("udf").
  2. For Java UDFs, renamed dataType to returnType.
  3. For Scala UDFs, added type tags.
  4. Added all Java UDF registration methods to Scala's UDFRegistration.
  5. Documentation

@rxin
[SPARK-5193][SQL] Reconcile Java and Scala UDFRegistration.
032f006
1. Removed UDFRegistration as a mixin in SQLContext and made it a field ("udf").
2. For Java UDFs, renamed dataType to returnType, and made that the 2nd argument since the UDF itself can be long.
3. Added all Java UDF registration methods to Scala's UDFRegistration.
4. Documentation
@SparkQA

SparkQA commented Jan 15, 2015

Copy link
Copy Markdown

Test build #25594 has started for PR 4056 at commit 032f006.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Jan 15, 2015

Copy link
Copy Markdown

Test build #25594 has finished for PR 4056 at commit 032f006.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class UDFRegistration (sqlContext: SQLContext) extends org.apache.spark.Logging

@AmplabJenkins

AmplabJenkins commented Jan 15, 2015

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25594/
Test FAILed.

@SparkQA

SparkQA commented Jan 15, 2015

Copy link
Copy Markdown

Test build #25615 has started for PR 4056 at commit 5f00c45.

  • This patch merges cleanly.

@rxinrxin changed the title [SPARK-5193][SQL] Reconcile Java and Scala UDFRegistration.[SPARK-5274][SQL] Reconcile Java and Scala UDFRegistration.Jan 15, 2015
@SparkQA

SparkQA commented Jan 15, 2015

Copy link
Copy Markdown

Test build #25615 has finished for PR 4056 at commit 5f00c45.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class UDFRegistration (sqlContext: SQLContext) extends org.apache.spark.Logging

@AmplabJenkins

AmplabJenkins commented Jan 15, 2015

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25615/
Test FAILed.

@SparkQA

SparkQA commented Jan 15, 2015

Copy link
Copy Markdown

Test build #25616 has started for PR 4056 at commit 675a3c9.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Jan 15, 2015

Copy link
Copy Markdown

Test build #25618 has started for PR 4056 at commit ae9c556.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Jan 15, 2015

Copy link
Copy Markdown

Test build #25616 has finished for PR 4056 at commit 675a3c9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class UDFRegistration (sqlContext: SQLContext) extends org.apache.spark.Logging

@AmplabJenkins

AmplabJenkins commented Jan 15, 2015

Copy link
Copy Markdown

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25616/
Test PASSed.

@rxin

rxin commented Jan 16, 2015

Copy link
Copy Markdown
ContributorAuthor

Merging in master.

@SparkQA

SparkQA commented Jan 16, 2015

Copy link
Copy Markdown

Test build #25618 has finished for PR 4056 at commit ae9c556.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class UDFRegistration (sqlContext: SQLContext) extends org.apache.spark.Logging

@AmplabJenkins

AmplabJenkins commented Jan 16, 2015

Copy link
Copy Markdown

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25618/
Test PASSed.

@rxinrxin mentioned this pull request Jan 16, 2015
Closed
asfgit pushed a commit that referenced this pull request Jan 17, 2015
@rxin
[SPARK-5193][SQL] Remove Spark SQL Java-specific API.
61b427d
After the following patches, the main (Scala) API is now usable for Java users directly.
#4056#4054#4049#4030#3965#3958
Author: Reynold Xin <rxin@databricks.com>
Closes#4065 from rxin/sql-java-api and squashes the following commits:
b1fd860 [Reynold Xin] Fix Mima
6d86578 [Reynold Xin] Ok one more attempt in fixing Python...
e8f1455 [Reynold Xin] Fix Python again...
3e53f91 [Reynold Xin] Fixed Python.
83735da [Reynold Xin] Fix BigDecimal test.
e9f1de3 [Reynold Xin] Use scala BigDecimal.
500d2c4 [Reynold Xin] Fix Decimal.
ba3bfa2 [Reynold Xin] Updated javadoc for RowFactory.
c4ae1c5 [Reynold Xin] [SPARK-5193][SQL] Remove Spark SQL Java-specific API.
andmarios pushed a commit to andmarios/spark-examples that referenced this pull request Mar 20, 2015
@rxin
[SPARK-5193][SQL] Remove Spark SQL Java-specific API.
3364cc8
After the following patches, the main (Scala) API is now usable for Java users directly.
apache/spark#4056apache/spark#4054apache/spark#4049apache/spark#4030apache/spark#3965apache/spark#3958
Author: Reynold Xin <rxin@databricks.com>
Closes #4065 from rxin/sql-java-api and squashes the following commits:
b1fd860 [Reynold Xin] Fix Mima
6d86578 [Reynold Xin] Ok one more attempt in fixing Python...
e8f1455 [Reynold Xin] Fix Python again...
3e53f91 [Reynold Xin] Fixed Python.
83735da [Reynold Xin] Fix BigDecimal test.
e9f1de3 [Reynold Xin] Use scala BigDecimal.
500d2c4 [Reynold Xin] Fix Decimal.
ba3bfa2 [Reynold Xin] Updated javadoc for RowFactory.
c4ae1c5 [Reynold Xin] [SPARK-5193][SQL] Remove Spark SQL Java-specific API.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rxin@SparkQA@AmplabJenkins