Uh oh!
There was an error while loading. Please reload this page.
[improve](udf) support java-udf static load - #34980
Conversation
doris-robot
commented
May 16, 2024
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
clang-tidy review says "All clean, LGTM! 👍" |
zhangstar333
commented
May 16, 2024
run buildall |
clang-tidy review says "All clean, LGTM! 👍" |
doris-robot
commented
May 16, 2024
TeamCity be ut coverage result: |
zhangstar333
commented
May 17, 2024
run buildall |
doris-robot
commented
May 17, 2024
TeamCity be ut coverage result: |
zhangstar333
commented
May 17, 2024
run buildall |
clang-tidy review says "All clean, LGTM! 👍" |
zhangstar333
commented
May 17, 2024
run buildall |
doris-robot
commented
May 17, 2024
TeamCity be ut coverage result: |
clang-tidy review says "All clean, LGTM! 👍" |
1 similar comment
clang-tidy review says "All clean, LGTM! 👍" |
zhangstar333
commented
May 21, 2024
run buildall |
doris-robot
commented
May 21, 2024
TeamCity be ut coverage result: |
zhangstar333
commented
Jun 3, 2024
run buildall |
zhangstar333
commented
Jun 3, 2024
run buildall |
clang-tidy review says "All clean, LGTM! 👍" |
doris-robot
commented
Jun 3, 2024
TPC-H: Total hot run time: 40799 ms |
doris-robot
commented
Jun 3, 2024
TeamCity be ut coverage result: |
doris-robot
commented
Jun 3, 2024
TPC-DS: Total hot run time: 170598 ms |
doris-robot
commented
Jun 3, 2024
ClickBench: Total hot run time: 30.78 s |
0ee3d8f to
a2f9d16Compareclang-tidy review says "All clean, LGTM! 👍" |
zhangstar333
commented
Jun 4, 2024
run buildall |
clang-tidy review says "All clean, LGTM! 👍" |
doris-robot
commented
Jun 4, 2024
TPC-H: Total hot run time: 40684 ms |
update update add expiration_time update update
clang-tidy review says "All clean, LGTM! 👍" |
clang-tidy review says "All clean, LGTM! 👍" |
zhangstar333
commented
Jun 25, 2024
run buildall |
clang-tidy review says "All clean, LGTM! 👍" |
doris-robot
commented
Jun 25, 2024
TPC-H: Total hot run time: 40036 ms |
doris-robot
commented
Jun 25, 2024
TPC-DS: Total hot run time: 173500 ms |
doris-robot
commented
Jun 25, 2024
ClickBench: Total hot run time: 30.56 s |
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
## Proposed changes ``` CREATE FUNCTION print_12() RETURNS int PROPERTIES ( "symbol" = "org.apache.doris.udf.AddOne", "type" = "JAVA_UDF", "file" = "file:///mnt/ava-udf-demo-jar-with-dependencies.jar", "static_load" = "true", // default value is false "expiration_time" = "60" // default value is 360 minutes ); ``` if set static_load=true, the udf ClassLoader will be cache, and will be clear when expiration timeout or use drop function stmt.
In this PR #34980, we cache the class load of java udf. But when calling init() method, it is still very time consuming when calling `Class.forName()`. This PR add a new class `UdfClassCache` to cache more things so that the `init()` method will not cost too much time.
In this PR #34980, we cache the class load of java udf. But when calling init() method, it is still very time consuming when calling `Class.forName()`. This PR add a new class `UdfClassCache` to cache more things so that the `init()` method will not cost too much time.
``` CREATE FUNCTION print_12() RETURNS int PROPERTIES ( "symbol" = "org.apache.doris.udf.AddOne", "type" = "JAVA_UDF", "file" = "file:///mnt/ava-udf-demo-jar-with-dependencies.jar", "static_load" = "true", // default value is false "expiration_time" = "60" // default value is 360 minutes ); ``` if set static_load=true, the udf ClassLoader will be cache, and will be clear when expiration timeout or use drop function stmt.
``` CREATE FUNCTION print_12() RETURNS int PROPERTIES ( "symbol" = "org.apache.doris.udf.AddOne", "type" = "JAVA_UDF", "file" = "file:///mnt/ava-udf-demo-jar-with-dependencies.jar", "static_load" = "true", // default value is false "expiration_time" = "60" // default value is 360 minutes ); ``` if set static_load=true, the udf ClassLoader will be cache, and will be clear when expiration timeout or use drop function stmt.
``` CREATE FUNCTION print_12() RETURNS int PROPERTIES ( "symbol" = "org.apache.doris.udf.AddOne", "type" = "JAVA_UDF", "file" = "file:///mnt/ava-udf-demo-jar-with-dependencies.jar", "static_load" = "true", // default value is false "expiration_time" = "60" // default value is 360 minutes ); ``` if set static_load=true, the udf ClassLoader will be cache, and will be clear when expiration timeout or use drop function stmt.
## Proposed changes ``` CREATE FUNCTION print_12() RETURNS int PROPERTIES ( "symbol" = "org.apache.doris.udf.AddOne", "type" = "JAVA_UDF", "file" = "file:///mnt/ava-udf-demo-jar-with-dependencies.jar", "static_load" = "true", // default value is false "expiration_time" = "60" // default value is 360 minutes ); ``` if set static_load=true, the udf ClassLoader will be cache, and will be clear when expiration timeout or use drop function stmt.
In this PR apache#34980, we cache the class load of java udf. But when calling init() method, it is still very time consuming when calling `Class.forName()`. This PR add a new class `UdfClassCache` to cache more things so that the `init()` method will not cost too much time.
Proposed changes
if set static_load=true, the udf ClassLoader will be cache, and will be clear when expiration timeout
or use drop function stmt.
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...