Uh oh!
There was an error while loading. Please reload this page.
feat: attach Diagnostic to "invalid function argument types" error - #23063
feat: attach Diagnostic to "invalid function argument types" error#23063choplin wants to merge 6 commits into
Conversation
kosiew
commented
Jun 29, 2026
@choplin |
Add span-aware Diagnostic to the error produced by `verify_function_arguments` when a scalar or aggregate function is called with argument types that do not match any supported signature. - Add `spans: Spans` field to `ScalarFunction` and `AggregateFunction` so function-call source locations survive planning and optimization. - Populate spans from sqlparser in the SQL planner when `collect_spans` is enabled. - Pass the function-call span to `Diagnostic::new_error` so downstream consumers can highlight the offending call site. - Update all pattern matches, tree-node transforms, and serialization code for the new field. Closesapache#14431
Collapse a double blank line flagged by rustfmt (blank_lines_upper_bound) in the diagnostic test file.
0eeb855 to
8fc9954Comparechoplin
commented
Jun 29, 2026
@kosiew I rebased onto the latest main HEAD and resolved the conflict. |
Thank you for opening this pull request! Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch). Details |
The `spans` field added to `expr::AggregateFunction` was not included in the substrait producer's exhaustive pattern match, breaking the build for the (non-default) substrait feature with E0027.
| WindowFunctionDefinition::AggregateUDF(udaf) => { | ||
| let new_fields = | ||
| verify_function_arguments(udaf.as_ref(), &fields)?; | ||
| verify_function_arguments(udaf.as_ref(), &fields, None)?; |
There was a problem hiding this comment.
Window aggregate calls still seem to produce diagnostics without a source span.
The regular aggregate path passes spans.first() into verify_function_arguments, but the Expr::WindowFunction path now passes None for both aggregate and window UDFs. For example, SELECT sum(first_name) OVER () FROM person reaches this branch through WindowFunctionDefinition::AggregateUDF, so the new invalid argument type(s) for 'sum' diagnostic cannot highlight the SQL call site.
Could you carry the function-call span on WindowFunction too, set it in SqlToRel when building the window expression, and pass it through here?
There was a problem hiding this comment.
Thanks! Fixed in c8a4431.
WindowFunction now carries a spans field (mirroring ScalarFunction / AggregateFunction). SqlToRel populates it from the function-call token when collect_spans is enabled, and both the AggregateUDF and WindowUDF window branches now pass spans.first() into verify_function_arguments instead of None, so sum(first_name) OVER () and window-UDF calls point at the SQL call site.
Added test_invalid_window_function_argument_types based on your example (sum(first_name) OVER ()), asserting the diagnostic points at the sum call site.
| Ok(()) | ||
| } | ||
| #[test] |
There was a problem hiding this comment.
It would be helpful to add a scalar-function regression test as well.
The PR covers scalar and aggregate calls, but the new test only exercises the aggregate path. A small invalid scalar call with collect_spans enabled would protect the other changed branch too.
There was a problem hiding this comment.
Done in 371b705 — added test_invalid_scalar_function_argument_types.
Window aggregate / window UDF calls (e.g. `sum(first_name) OVER ()`) previously passed `None` as the span into `verify_function_arguments`, so the resulting diagnostic could not point at the SQL call site. Add a `spans` field to `WindowFunction` (mirroring `ScalarFunction` / `AggregateFunction`), populate it in `SqlToRel`, and thread it through `ExprSchemable` so window function argument diagnostics carry the span. Add a regression test for the window function path. Addresses PR review feedback.
The "invalid function argument types" work covered scalar and aggregate calls, but the diagnostic test suite only exercised the aggregate path. Add a test for an invalid scalar call (`power(first_name, first_name)`) with `collect_spans` enabled to cover the scalar branch. Addresses PR review feedback.
choplin
commented
Jul 9, 2026
@kosiew Thanks for the review! I've addressed the two points from your review. Please take a look! |
kosiew
left a comment
There was a problem hiding this comment.
Thanks for the updates. The window aggregate and window UDF diagnostics now propagate WindowFunction.spans from SqlToRel and pass spans.first() into verify_function_arguments, so that concern is addressed. The scalar-function regression test has also been added.
There is still one high-severity semver issue that needs to be resolved before merge. The new spans fields change the public layouts of externally constructible expression structs.
| /// List of expressions to feed to the functions as arguments | ||
| pub args: Vec<Expr>, | ||
| /// Original source code location, if known | ||
| pub spans: Spans, |
There was a problem hiding this comment.
Could we avoid adding spans as a required public field on these expression structs?
ScalarFunction, AggregateFunction, and WindowFunction are publicly constructible with struct literals. Adding a new required field means existing downstream code using those literals will no longer compile. cargo-semver-checks reports constructible_struct_adds_field for ScalarFunction.spans, AggregateFunction.spans, and WindowFunction.spans.
Please keep the diagnostic span data outside these public struct layouts, or use another semver-compatible design. The new required public fields should be removed before this is merged.
There was a problem hiding this comment.
Sorry for the slow reply here!
You're right about the semver break — adding spans to these public structs tripsconstructible_struct_adds_field.
My thinking so far: I noticed Column already carries the same pub spans: Spans
field, added the same way to that (publicly constructible) struct in the earlier
diagnostics work (#13664) — so one direction would be to stay consistent with that
precedent. Another would be to keep the span off these struct layouts entirely and
validate arguments eagerly in SqlToRel, where the call-site span is already in scope,
so the diagnostic is attached at planning time.
I'm not sure which fits best here, though, and there may well be a cleaner
semver-compatible approach I'm not seeing. Do you have a direction you'd prefer — or a
better idea than either of these?
This comment was marked as duplicate.
This comment was marked as duplicate.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
This comment was marked as duplicate.
This comment was marked as duplicate.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
| /// List of expressions to feed to the functions as arguments | ||
| pub args: Vec<Expr>, | ||
| /// Original source code location, if known | ||
| pub spans: Spans, |
There was a problem hiding this comment.
Between the two options you mentioned, I'd lean toward keeping the parser-only span information out of these public struct layouts. Validating in SqlToRel, where the call-site span is already available, seems like a good fit and avoids introducing a semver break while still preserving the improved diagnostics. If there's another semver-compatible way to associate the span with these expressions, that would also work, but I don't think adding the new public fields is something we should ship in a patch release.
There was a problem hiding this comment.
Sorry this took so long. I removed the parser-only spans from the public expression structs and now validate scalar, aggregate, and window functions in SqlToRel, attaching the call-site span only to matching diagnostics. Fixed in 879faea.
There was a problem hiding this comment.
@kosiew Sorry, I've noticed that fix I reported in 879faea doesn't work.
Validating in SqlToRel happens before the TypeCoercion analyzer rule, so it only sees the argument types as written. This rejects valid queries. The existing test select_where_nullif_division fails:
SELECT c3/(c4+c5) FROM aggregate_test_100 WHERE c3/nullif(c4+c5, 0) >0.1invalid argument type(s) for 'nullif'
called with argument type(s): Int32, Int64
candidate function(s): nullif(Int32, Int32)
nullif(Int32, Int64) is coercible to nullif(Int32, Int32), but the check runs too early to see the coerced types.
The difficulty is that the error is only known where coercion runs (verify_function_arguments, during analysis), while the span is only known during planning. Something has to bring the two together. These are the four options I can think of:
- Make the check in
SqlToRelcoercion-aware, so it accepts arguments that coercion would accept. This keeps the current design and needs no struct change, but it reimplements whatTypeCoercionalready does, in the planner, just to produce a diagnostic. The two would have to stay in sync, and if they drift the planner rejects queries the engine can run. - Point the diagnostic at the arguments. Arguments are
Expr::Columns and already carry spans, soverify_function_argumentscan take the span from them instead of from the call. Validation stays where it is, so valid queries are no longer rejected. The location becomes approximate:sum(first_name)is reported atfirst_name, and for a call likesum(a + b)it is reported at whichever column comes first inside the argument. - Attach the span after analysis. The diagnostic is produced without a span, and a later step matches it against the parsed statement by function name to fill in the location. This changes no struct, but the match is a heuristic. For
SELECT sum(a), sum(b)there are two calls to the same function and no reliable way to tell which one failed. - Add
spansto the expression structs, as the original approach did, and land it in a future major release.
Option 4 is the one I would prefer. If the expression itself holds the span, the span is still available when the error is raised, so the check can stay in verify_function_arguments and run after coercion. The diagnostic then points at the call itself. It also follows Column, which has carried pub spans: Spans since #13664.
If there is a better way to carry the span into analysis that I have missed, I would be glad to hear it.
Your concern was about shipping the new public fields in a patch release. I read the API health policy, but I could not tell whether a contribution that has to wait for a major version bump is acceptable, or how such a change is usually handled here. Could you advise? Depending on that, I can hold this PR until the next major release, or land the diagnostic without a span now and add the span later.
There was a problem hiding this comment.
I would advise:
Separate the work. Keep this PR patch-compatible: no
spansfields onScalarFunction,AggregateFunction, orWindowFunction. The major-version PR should contain only the API/layout change and the propagation required for it, rather than bundling a diagnostic behavior change with a release break.Coordinate the release boundary first. Ask maintainers/release owners to confirm the target major release and its versioning plan. The version in the relevant public crate manifest(s) must be advanced as part of that coordinated major-release work. Version 55 is in progress already, so this may have to wait till version 56.
Make the break explicit. State in the PR description and release notes that public struct literals are source-incompatible. Show the downstream migration:
// BeforeScalarFunction{ func, args }// After (if a literal is still required)ScalarFunction{ func, args,spans:Spans::new(),}// Prefer the stable constructor where it expresses the use caseScalarFunction::new_udf(func, args)
Equivalent literal users of
AggregateFunctionandWindowFunctionneed the same update. Add or retain constructors that initialize empty spans so normal programmatic expression construction does not need parser metadata.
Function-call spans were stored on public expression structs, which made externally constructible layouts source-incompatible. Validate scalar, aggregate, and window functions while the SQL parser span is still in scope, attach it only to matching diagnostics, and restore the original public expression layouts.
There was a problem hiding this comment.
Thanks for iterating on this. The scalar regression test is covered, the window aggregate/window-UDF path now has call-site span coverage for canonical names, and the public spans fields have been removed, so the earlier SemVer issue is resolved.
I still see two issues that should be fixed before merging.
The main blocker is that validate_function_expr now calls Expr::to_field during SqlToRel. That means function arguments are being validated before the TypeCoercion analyzer has had a chance to insert casts. I reproduced this with the existing select_where_nullif_division test. nullif(c4 + c5, 0) is rejected as nullif(Int32, Int64) during planning even though the analyzer normally coerces the literal and the query should succeed.
I think validation should stay in the existing analysis/type-coercion path. For this PR, I would prefer attaching the best available argument or subexpression span when verify_function_arguments creates the diagnostic. That location may be less precise than the exact function-call span, but it preserves existing query acceptance and avoids duplicating coercion or validation logic in SqlToRel. If no useful span is available, keeping the diagnostic span as None is still better than changing validation timing.
There is also an alias issue in the current matching logic. The diagnostic uses the resolved function name, while validate_function_expr compares against the SQL spelling. For example, pow(...) resolves to power, so the diagnostic says invalid argument type(s) for 'power', while the helper looks for ... 'pow'. That means the call-site span is not attached for aliases.
For the longer-term exact-span design, I agree that carrying span state on the expression structs is the most direct way to preserve the exact function-call location through analysis and rewrites. The problem is that adding required fields to ScalarFunction, AggregateFunction, and WindowFunction is a source-breaking public API change because downstream struct literals stop compiling. I would keep that out of this PR and handle it separately as an explicitly coordinated major-release API change if maintainers decide the precision is worth the compatibility cost.
So for this PR, my recommendation is: remove the SqlToRel validation, keep the diagnostic generation in verify_function_arguments, and use the best SemVer-compatible span available there.
| } else { | ||
| None | ||
| }; | ||
| validate_function_expr(&expr, schema, &name, span)?; |
There was a problem hiding this comment.
I think this validation is happening too early. validate_function_expr calls Expr::to_field while we are still in SqlToRel, before the TypeCoercion analyzer has inserted any necessary casts.
That changes behavior for valid coercible calls. I reproduced it with the existing select_where_nullif_division test: nullif(c4 + c5, 0) reaches this point as nullif(Int32, Int64) and is rejected, even though the analyzer normally coerces the literal and the query should plan successfully.
Could we avoid validating here and keep the existing post-coercion validation boundary? I would prefer attaching the best available argument or subexpression span when verify_function_arguments emits the diagnostic, even if that span is approximate, rather than changing when validation happens.
| function_span: Option<Span>, | ||
| ) -> Result<()> { | ||
| expr.to_field(schema).map(|_| ()).map_err(|err| { | ||
| let expected_message = format!("invalid argument type(s) for '{function_name}'"); |
There was a problem hiding this comment.
I think aliases will miss the call-site span here. function_name is the spelling from the SQL query, but verify_function_arguments builds the diagnostic using the resolved UDF's canonical name.
For example, pow(first_name, first_name) resolves to power, so the diagnostic says invalid argument type(s) for 'power', while this helper looks for invalid argument type(s) for 'pow'. The messages do not match, so the original diagnostic is returned without the span.
If this matching approach remains, could we compare against the canonical resolved function name while still attaching the span from the SQL call? It would also be good to add alias regression coverage for the applicable scalar, aggregate, and window paths.
Which issue does this PR close?
Diagnosticto "invalid function argument types" error #14431Rationale for this change
When a scalar or aggregate function is called with argument types that don't match any supported signature (e.g.
SELECT sum('a')), the error message is a raw text dump with no source location. This PR attaches aDiagnosticso downstream consumers can highlight the offending call site in the original SQL query.What changes are included in this PR?
spans: Spansfield toScalarFunctionandAggregateFunctionso function-call source locations survive planning and optimization.collect_spansis enabled, following the same pattern used forColumn.verify_function_arguments, attach aDiagnosticwith:invalid argument type(s) for 'sum')called with argument type(s): Utf8)candidate function(s): sum(Decimal), sum(UInt64), ...)spansfield.Are there any user-facing changes?
No breaking API changes.
ScalarFunctionandAggregateFunctiongain aspans: Spansfield, butSpansis ignored inPartialEq/Hashand defaults to empty, so existing code usingScalarFunction::new_udf/AggregateFunction::new_udfis unaffected.When an error occurs, a diagnositc is attached in the error message.
Before (no
Diagnosticattached):After (same error message, with
Diagnosticnow attached):