Skip to content

[Vectorized][UDF] support java-udaf - #9930

Merged
HappenLee merged 2 commits into
apache:masterfrom
zhangstar333:test_udaf
Jun 15, 2022
Merged

[Vectorized][UDF] support java-udaf#9930
HappenLee merged 2 commits into
apache:masterfrom
zhangstar333:test_udaf

Conversation

@zhangstar333

@zhangstar333zhangstar333 commented Jun 2, 2022

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close#8389

Problem Summary:

This feature is used to support Java udaf. Users can create aggregate functions by self,
For example, we create Java udaf function by code below.

CREATE AGGREGATE FUNCTION simple_add (int) RETURNS int PROPERTIES (
"file"="file:///pathTo/java-udaf.jar",
"symbol"="org.apache.doris.udf.SimpleAdd",
"type"="JAVA_UDF"
);

file: indicate where user file is.
symbol: for java udaf means udaf class in this jar.
type: indicate this function is a java udf.

Users may have some necessary conditions
when implementing aggregate functions by self,
for example:

class SimpleAdd {
public static class State { some variables: example count agg: maybe a long type counter;
}
public State create() {
....... if need do some.......
return new State();
}
public void destroy(State s) {
....... if need do some destroy work.......
}
public void add(State s, Type val, [Type val2, .....]) {
do update work here;
like count agg function: you need plus 1
}
public void serialize(State s, DataOutputStream out) {
put you add result into buffer to serialize
}
public void deserialize(State state, DataInputStream in) {
deserialize get data from buffer before you put
}
public void merge(State s, State other) {
merge result from other state
}
public Type getValue(State s) {
return finally result
}
}

Checklist(Required)

  1. Does it affect the original behavior: (Yes/No/I Don't know)
  2. Has unit tests been added: (Yes/No/No Need)
  3. Has document been added or modified: (Yes/No/No Need)
  4. Does it need to update dependencies: (Yes/No)
  5. Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actionsgithub-actionsBot added area/planner Issues or PRs related to the query planner area/vectorization labels Jun 2, 2022
Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
@morningman

Copy link
Copy Markdown
Contributor

Hi @zhangstar333 , could you explain more about what does this PR do?
Because we already supported part of JAVA udf feature, so you need to explain what's new in your PR.

@zhangstar333
zhangstar333force-pushed the test_udaf branch 2 times, most recently from fe08c70 to 47a1392CompareJune 2, 2022 16:25
@HappenLee

Copy link
Copy Markdown
Contributor

need also modify the doc,please

Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
Comment threadfe/java-udf/src/main/java/org/apache/doris/udf/UdafExecutor.java Outdated
Comment threadfe/java-udf/src/main/java/org/apache/doris/udf/UdafExecutor.java Outdated
Comment threadfe/java-udf/src/main/java/org/apache/doris/udf/UdafExecutor.java Outdated
@github-actionsgithub-actionsBot added the kind/docs Categorizes issue or PR as related to documentation. label Jun 7, 2022
Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
@github-actionsgithub-actionsBot added the area/sql/function Issues or PRs related to the SQL functions label Jun 8, 2022
@zhangstar333
zhangstar333force-pushed the test_udaf branch 3 times, most recently from 5ad7c4b to 86cfb01CompareJune 10, 2022 03:24
Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated
rebase to master
fix formatter error
refactor add function
change some code according review
add doc file about java-udaf
refactor serialize and deserialize function
remove test log
change init_udaf to create function
be formatter
change to trigger be test
remove first_init variable

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 15, 2022
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@HappenLee
HappenLee merged commit 4c24586 into apache:masterJun 15, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.area/plannerIssues or PRs related to the query plannerarea/sql/functionIssues or PRs related to the SQL functionsarea/vectorizationkind/docsCategorizes issue or PR as related to documentation.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support Java UDF

5 participants

@zhangstar333@morningman@HappenLee@BiteTheDDDDt@Gabriel39