Skip to content

SPARK-11716 - UDFRegistration just drops the input type when re-creat… - #9739

Closed
jbonofre wants to merge 2 commits into
apache:masterfrom
jbonofre:SPARK-11716
Closed

SPARK-11716 - UDFRegistration just drops the input type when re-creat…#9739
jbonofre wants to merge 2 commits into
apache:masterfrom
jbonofre:SPARK-11716

Conversation

@jbonofre

@jbonofrejbonofre commented Nov 16, 2015

Copy link
Copy Markdown
Member

…ing the UserDefinedFunction

@const-ae

const-ae commented Nov 17, 2015

Copy link
Copy Markdown

Hi, I guess you should also update the script in the comments that generates the file.

@jbonofre

jbonofre commented Nov 17, 2015

Copy link
Copy Markdown
MemberAuthor

Good point, let me update the PR accordingly. Thanks.

@jbonofre

jbonofre commented Nov 17, 2015

Copy link
Copy Markdown
MemberAuthor

PR rebased and updated according to @Artjom-Metro comment.

@jbonofre

jbonofre commented Nov 18, 2015

Copy link
Copy Markdown
MemberAuthor

Rebase

@yhuai

yhuai commented Nov 18, 2015

Copy link
Copy Markdown
Contributor

Can you change the title format to [SPARK-11716] [SQL] ...?

@yhuai

yhuai commented Nov 18, 2015

Copy link
Copy Markdown
Contributor

ok to test.

@yhuai

yhuai commented Nov 18, 2015

Copy link
Copy Markdown
Contributor

Can you add a test?

@yhuai

yhuai commented Nov 18, 2015

Copy link
Copy Markdown
Contributor

So, the problem is that when we use apply method on this UDF, we will not be able to pass in the type and thus, we fail to do type casting, right? It is not the case for using expr because the udf we get from the function registry has the input type. It will be great to update the description to describe the problem.

@SparkQA

SparkQA commented Nov 19, 2015

Copy link
Copy Markdown

Test build #46257 has finished for PR 9739 at commit 3ebf487.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@jbonofre

jbonofre commented Nov 19, 2015

Copy link
Copy Markdown
MemberAuthor

@yhuai Thanks for your feedbacks. Let me update the PR accordingly.

@yhuaiyhuai mentioned this pull request Nov 20, 2015
Closed
asfgit pushed a commit that referenced this pull request Nov 20, 2015
@jbonofre@marmbrus
[SPARK-11716][SQL] UDFRegistration just drops the input type when re-…
b9b0e17
…creating the UserDefinedFunction
https://issues.apache.org/jira/browse/SPARK-11716
This is one is #9739 and a regression test. When commit it, please make sure the author is jbonofre.
You can find the original PR at #9739closes#9739
Author: Jean-Baptiste Onofré <jbonofre@apache.org>
Author: Yin Huai <yhuai@databricks.com>
Closes#9868 from yhuai/SPARK-11716.
(cherry picked from commit 03ba56d)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@jbonofre

jbonofre commented Nov 21, 2015

Copy link
Copy Markdown
MemberAuthor

Awesome, thanks a lot @yhuai !

@yhuai

yhuai commented Nov 21, 2015

Copy link
Copy Markdown
Contributor

no problem! Thank you for your fix!

kiszk pushed a commit to kiszk/spark-gpu that referenced this pull request Dec 26, 2015
@jbonofre@kiszk
[SPARK-11716][SQL] UDFRegistration just drops the input type when re-…
d4760d5
…creating the UserDefinedFunction
https://issues.apache.org/jira/browse/SPARK-11716
This is one is #9739 and a regression test. When commit it, please make sure the author is jbonofre.
You can find the original PR at apache/spark#9739
closes #9739
Author: Jean-Baptiste Onofré <jbonofre@apache.org>
Author: Yin Huai <yhuai@databricks.com>
Closes #9868 from yhuai/SPARK-11716.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jbonofre@const-ae@yhuai@SparkQA