Uh oh!
There was an error while loading. Please reload this page.
[fix](udf) Reject variadic user-defined functions - #67373
Merged
Conversation
linrrzqqq
requested review from
924060929, englefly, morrySnow and starocean999
as code ownersSeptember 1, 2026 08:22
hello-stephen
commented
Sep 1, 2026
Contributor
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
linrrzqqqforce-pushed
the
reject-var-args-udf
branch
2 times, most recently
from
September 1, 2026 15:03
1df9265 to
8ce33b9Comparelinrrzqqq
commented
Sep 1, 2026
CollaboratorAuthor
/review |
Contributor
There was a problem hiding this comment.
Automated code review: request changes
The fixed-arity implementation is otherwise coherent, but the exact reviewed head cannot pass the mandatory FE style gate because the newly added arity test uses forbidden static Mockito imports. I left one inline comment with the concrete fix.
Critical checkpoint conclusions:
- Goal and proof: CREATE now rejects variadic declarations for scalar, aggregate, table, and alias functions; all new catalog objects persist fixed arity; historical Java/Python metadata is exact-gated before unsafe builder work. The parser and arity tests target those boundaries.
- Scope and clarity: the production change is small and consistently covers the two CREATE visitors, four catalog-construction branches, and all six Java/Python scalar/aggregate/table builders.
- Concurrency: no new thread, shared mutable state, lock, blocking operation, or lock-order change is introduced.
- Lifecycle: no resource ownership, static initialization, or special release lifecycle is changed.
- Configuration: no configuration item is added or changed.
- Compatibility: catalog/Thrift formats are unchanged; DROP and SHOW retain variadic signature parsing so historical entries remain addressable, while replay preserves their stored metadata.
- Parallel paths: scalar/UDAF/UDTF/alias creation, Java/Python/RPC metadata construction, database/global scopes, alias's pre-existing exact gate, and table-function normal/outer registration were traced. No parallel variable-arity creation path remains.
- Conditional checks: the exact-size check is the direct fixed-arity invariant and runs during candidate filtering before signature indexing, cast loops, child reconstruction, or FE-to-BE translation.
- Test coverage and results: the added tests cover the main CREATE/DROP/SHOW parser boundary and under/exact/over calls for all six changed builders. Per the review bundle, no local builds or tests were run. The exact-head GitHub CheckStyle job did run and failed on the added test imports.
- Observability: parser/analysis rejection does not require new logs or metrics; failures remain explicit analysis errors.
- Persistence and transactions: new function metadata and table companions journal the fixed bit; historical replay remains unchanged. No user-data transaction, visible-version, or data-write path is involved.
- FE/BE transport: no new transmitted variable or symbol/storage-format change is introduced; existing metadata fields remain intact.
- Performance: the O(1) arity check rejects invalid calls earlier and adds no meaningful allocation or hot-path cost.
- Other issues: no additional correctness, lifecycle, compatibility, performance, or coverage bug remained after the final changed-file and unresolved-risk sweep.
User focus: no additional user-provided focus was supplied.
Review completion: complete after one converged round (two full reviewers plus a separate historical-variadic risk review all returned NO_NEW_VALUABLE_FINDINGS). Reviewed head: 8ce33b9.
| import org.junit.jupiter.api.Assertions; | ||
| import org.junit.jupiter.api.Test; | ||
| import static org.mockito.Mockito.mock; |
Contributor
There was a problem hiding this comment.
[P2] Use the repository's non-static Mockito import
This added test currently makes the mandatory CheckStyle job fail: this line is in the wrong custom import group and violates AvoidStaticImport, while the following when import also violates AvoidStaticImport. Import org.mockito.Mockito normally and call Mockito.mock(...) / Mockito.when(...) so this head passes the FE style gate.
### What problem does this PR solve? Issue Number: None Problem Summary: CREATE FUNCTION accepted variadic signatures even though user-defined function implementations are fixed-arity. Remove variadic syntax from CREATE FUNCTION and CREATE ALIAS FUNCTION, and always create fixed-arity UDF metadata. Java and Python UDF, UDAF, and UDTF builders now also require the actual argument count to exactly match the declared arity. This prevents historical variadic metadata from accepting calls that the fixed-length builder and runtime paths cannot safely construct. Variadic signature parsing remains available for DROP FUNCTION and SHOW CREATE FUNCTION so historical catalog entries remain addressable. ### Release note CREATE FUNCTION no longer accepts variadic argument declarations, and user-defined function calls require an exact argument count. ### Check List (For Author) - Test - [x] Unit Test - ./run-fe-ut.sh --run org.apache.doris.nereids.parser.NereidsParserTest,org.apache.doris.catalog.CreateFunctionTest,org.apache.doris.nereids.trees.expressions.functions.udf.UdfBuilderArityTest,org.apache.doris.nereids.trees.expressions.functions.udf.UdfVolatilityTest - Behavior changed: - [x] Yes. CREATE FUNCTION rejects variadic declarations and UDF builders enforce fixed arity. - Does this need documentation? - [x] No.
linrrzqqqforce-pushed
the
reject-var-args-udf
branch
from
September 1, 2026 17:01
8ce33b9 to
360056bComparelinrrzqqq
commented
Sep 1, 2026
CollaboratorAuthor
run buildall |
hello-stephen
commented
Sep 1, 2026
Contributor
FE UT Coverage ReportIncrement line coverage |
hello-stephen
commented
Sep 1, 2026
Contributor
TPC-H: Total hot run time: 17071 ms |
hello-stephen
commented
Sep 1, 2026
Contributor
TPC-DS: Total hot run time: 82324 ms |
hello-stephen
commented
Sep 1, 2026
Contributor
ClickBench: Total hot run time: 14.71 s |
hello-stephen
commented
Sep 1, 2026
Contributor
FE Regression Coverage ReportIncrement line coverage |
hello-stephen
commented
Sep 2, 2026
Contributor
FE Regression Coverage ReportIncrement line coverage |
hello-stephen
commented
Sep 2, 2026
Contributor
FE Regression Coverage ReportIncrement line coverage |
zhangstar333
approved these changes
Sep 2, 2026
starocean999
approved these changes
Sep 2, 2026
hello-stephen
commented
Sep 2, 2026
Contributor
FE Regression Coverage ReportIncrement line coverage |
hello-stephen
commented
Sep 2, 2026
Contributor
FE Regression Coverage ReportIncrement line coverage |
Uh oh!
There was an error while loading. Please reload this page.
github-actionsBot
pushed a commit
that referenced
this pull request
Sep 4, 2026
User-defined functions exposed variadic DDL metadata without reliable
end-to-end support. Reject variadic declarations during `CREATE
FUNCTION` analysis for scalar, aggregate, table, and alias functions
while retaining variadic signature parsing for `DROP`, `SHOW`, and
historical metadata compatibility.
before:
```sql
CREATE FUNCTION py_add(INT, INT, ...)
RETURNS INT
PROPERTIES (
"type" = "PYTHON_UDF",
"symbol" = "evaluate",
"runtime_version" = "3.12.11",
"volatility" = "immutable"
)
AS $$
def evaluate(a, b, c):
return a + b + c
$$;
SELECT py_add(1, 2, 3);
-- ERROR 1105 (HY000): errCode = 2, detailMessage = Index 2 out of bounds for length 2
```
In `PythonUdfBuilder.java:82`:
```java
public Pair<PythonUdf, PythonUdf> build(String name, List<?> arguments) {
// exprs = (1, 2, 3), size = 3
// argTypes = [INT, INT], size = 2
List<Expression> exprs = arguments.stream().map(Expression.class::cast).collect(Collectors.toList());
List<DataType> argTypes = udf.getSignatures().get(0).argumentsTypes;
List<Expression> processedExprs = Lists.newArrayList();
for (int i = 0; i < exprs.size(); ++i) {
// when i = 2, argTypes.get(2), err occur!
processedExprs.add(TypeCoercionUtils.castIfNotSameType(exprs.get(i), argTypes.get(i)));
}
return Pair.ofSame(udf.withFreshVolatileIdentity().withChildren(processedExprs));
}
```
now:
```sql
CREATE FUNCTION py_add(INT, INT, ...)
RETURNS INT
PROPERTIES (
"type" = "PYTHON_UDF",
"symbol" = "evaluate",
"runtime_version" = "3.12.11",
"volatility" = "immutable"
)
AS $$
def evaluate(a, b, c):
return a + b + c
$$;
-- ERROR 1105 (HY000): errCode = 2, detailMessage = mismatched input ',' expecting ')'(line 1, pos 31)
```
### Release note
Reject variadic declarations for user-defined functions.
### Check List (For Author)
- Test: Unit Test
- ./run-fe-ut.sh --run org.apache.doris.catalog.CreateFunctionTest
- Behavior changed: Yes. Variadic user-defined function declarations are
rejected during analysis.
- Does this need documentation:
apache/doris-website#4103yiguolei pushed a commit
that referenced
this pull request
Sep 5, 2026
User-defined functions exposed variadic DDL metadata without reliable
end-to-end support. Reject variadic declarations during `CREATE
FUNCTION` analysis for scalar, aggregate, table, and alias functions
while retaining variadic signature parsing for `DROP`, `SHOW`, and
historical metadata compatibility.
before:
```sql
CREATE FUNCTION py_add(INT, INT, ...)
RETURNS INT
PROPERTIES (
"type" = "PYTHON_UDF",
"symbol" = "evaluate",
"runtime_version" = "3.12.11",
"volatility" = "immutable"
)
AS $$
def evaluate(a, b, c):
return a + b + c
$$;
SELECT py_add(1, 2, 3);
-- ERROR 1105 (HY000): errCode = 2, detailMessage = Index 2 out of bounds for length 2
```
In `PythonUdfBuilder.java:82`:
```java
public Pair<PythonUdf, PythonUdf> build(String name, List<?> arguments) {
// exprs = (1, 2, 3), size = 3
// argTypes = [INT, INT], size = 2
List<Expression> exprs = arguments.stream().map(Expression.class::cast).collect(Collectors.toList());
List<DataType> argTypes = udf.getSignatures().get(0).argumentsTypes;
List<Expression> processedExprs = Lists.newArrayList();
for (int i = 0; i < exprs.size(); ++i) {
// when i = 2, argTypes.get(2), err occur!
processedExprs.add(TypeCoercionUtils.castIfNotSameType(exprs.get(i), argTypes.get(i)));
}
return Pair.ofSame(udf.withFreshVolatileIdentity().withChildren(processedExprs));
}
```
now:
```sql
CREATE FUNCTION py_add(INT, INT, ...)
RETURNS INT
PROPERTIES (
"type" = "PYTHON_UDF",
"symbol" = "evaluate",
"runtime_version" = "3.12.11",
"volatility" = "immutable"
)
AS $$
def evaluate(a, b, c):
return a + b + c
$$;
-- ERROR 1105 (HY000): errCode = 2, detailMessage = mismatched input ',' expecting ')'(line 1, pos 31)
```
### Release note
Reject variadic declarations for user-defined functions.
### Check List (For Author)
- Test: Unit Test
- ./run-fe-ut.sh --run org.apache.doris.catalog.CreateFunctionTest
- Behavior changed: Yes. Variadic user-defined function declarations are
rejected during analysis.
- Does this need documentation:
apache/doris-website#4103
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Problem Summary:
User-defined functions exposed variadic DDL metadata without reliable end-to-end support. Reject variadic declarations during
CREATE FUNCTIONanalysis for scalar, aggregate, table, and alias functions while retaining variadic signature parsing forDROP,SHOW, and historical metadata compatibility.before:
In
PythonUdfBuilder.java:82:now:
Release note
Reject variadic declarations for user-defined functions.
Check List (For Author)