Uh oh!
There was an error while loading. Please reload this page.
GH-50869: [C++][Compute] Tighten coalesce exact dispatch for decimal varargs - #50870
GH-50869: [C++][Compute] Tighten coalesce exact dispatch for decimal varargs#50870zanmato1984 wants to merge 3 commits into
Conversation
zanmato1984
commented
Aug 15, 2026
@pitrou, could you please review this when you have a chance? Thanks! |
There was a problem hiding this comment.
Pull request overview
This PR prevents coalesce expression binding from selecting a too-broad “exact” decimal varargs kernel when decimal arguments differ in precision/scale, ensuring the binder falls back to DispatchBest so decimal normalization and implicit casts are applied before execution.
Changes:
- Introduces a decimal-only
MatchConstraintforcoalesceexact dispatch that requires all arguments to have identical concrete decimalDataType(precision/scale and width). - Attaches that constraint to the
decimal128anddecimal256varargs kernel registrations forcoalesce. - Adds regression tests covering both dispatch behavior (
DispatchExact/DispatchBest) and expression bind/execute for mixed decimal types.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| cpp/src/arrow/compute/kernels/scalar_if_else.cc | Adds and wires a decimal-only exact-dispatch constraint for coalesce decimal varargs kernels. |
| cpp/src/arrow/compute/kernels/scalar_if_else_test.cc | Adds dispatch regressions ensuring mixed concrete decimal types do not exact-dispatch and instead normalize via DispatchBest. |
| cpp/src/arrow/compute/expression_test.cc | Adds expression-binding and end-to-end execution regressions for mixed decimal inputs to coalesce. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…cimal varargs Signed-off-by: Rossi Sun <zanmato1984@gmail.com>
23e9ff6 to
662a932CompareThere was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (1)
cpp/src/arrow/compute/kernels/scalar_if_else.cc:2814
AddCoalesceKernelnow supportsMatchConstraint, but it’s only applied to decimal kernels. TheDictionaryTypecoalesce kernel signature is still broad (InputType(Type::DICTIONARY)), so expression binding can exact-dispatch mixed dictionary types (different value/index types) and then fail at execution time inCheckIdenticalTypes. Consider adding an exact-dispatch constraint for dictionary coalesce kernels requiring all args to have identical fullDataType, so mixed dictionaries fall back toDispatchBest(which already decodes dictionaries before selecting a kernel).
void AddCoalesceKernel(const std::shared_ptr<ScalarFunction>& scalar_function,
detail::GetTypeId get_id, ArrayKernelExec exec,
std::shared_ptr<MatchConstraint> constraint = nullptr) {
ScalarKernel kernel(KernelSignature::Make({InputType(get_id.id)}, FirstType,
/*is_varargs=*/true, std::move(constraint)),
exec);
zanmato1984
commented
Sep 1, 2026
Kindly ping @pitrou . Thanks. |
| {decimal128(4, 3), decimal128(4, 3)}); | ||
| CheckDispatchBest("coalesce", {decimal128(3, 2), decimal256(3, 2)}, | ||
| {decimal256(3, 2), decimal256(3, 2)}); | ||
| CheckDispatchBest("coalesce", {decimal256(3, 2), decimal128(3, 2)}, |
There was a problem hiding this comment.
What about e.g. {decimal256(4, 1), decimal128(3, 2)}?
There was a problem hiding this comment.
Good catch. I added this case in both argument orders. The common type is decimal256(5, 2), and the binding tests verify that both inputs are cast to it.
| return arrow::compute::detail::NoMatchingKernel(this, *types); | ||
| } | ||
| static std::shared_ptr<MatchConstraint> DecimalMatchConstraint() { |
There was a problem hiding this comment.
It would be surprising if if_else was the only kernel to use this constraint, should it be exposed in kernel.h? I see there's already DecimalsHaveSameScale there.
There was a problem hiding this comment.
Agreed. I moved the constraint to kernel.h / kernel.cc and generalized it as AllTypesAreIdentical() and AllTypesAreIdenticalFrom(first_type_index). coalesce uses the former, while case_when uses AllTypesAreIdenticalFrom(1).
Signed-off-by: Rossi Sun <zanmato1984@gmail.com>
There was a problem hiding this comment.
🟡 Changes recommended
The newly added AllTypesAreIdenticalFrom constraint can invoke undefined behavior in release builds when called with too few input types due to reliance on DCHECK only.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 1
- Review effort level: Lite
| return MatchConstraint::Make( | ||
| [first_type_index](const std::vector<TypeHolder>& types) -> bool { | ||
| DCHECK_LT(first_type_index, types.size()); | ||
| return std::all_of(types.begin() + first_type_index + 1, types.end(), | ||
| [&types, first_type_index](const TypeHolder& type) { | ||
| return type == types[first_type_index]; | ||
| }); | ||
| }); |
There was a problem hiding this comment.
Thanks. This follows the existing MatchConstraint precondition style: DecimalsHaveSameScale() likewise uses DCHECK for both the minimum input count and the input kinds. In the normal dispatch path, Function::CheckArity() validates the argument count and the per-input signature checks run before the constraint; the current callers guarantee at least one input for AllTypesAreIdentical() and at least two for AllTypesAreIdenticalFrom(1).
Therefore, a shorter types vector would be a caller contract violation rather than a runtime input condition. I would prefer to keep the DCHECK here for consistency. Making all public constraints defensive could be considered separately, including DecimalsHaveSameScale().
There was a problem hiding this comment.
🟢 Approval recommended
The constraint change is narrowly scoped to decimal exact dispatch and is supported by targeted dispatch, binding, and execution regressions covering the reported failure modes.
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 0 new
- Review effort level: Lite
zanmato1984
commented
Sep 4, 2026
Hi @pitrou , I've addressed the comments. Mind to take another look? Thanks. |
Rationale for this change
Expression binding tries
DispatchExactbeforeDispatchBest. Thecoalescedecimal varargs kernels used broad decimal signatures, so mixed concrete decimal types could exact-match and bypass the existing decimal normalization and cast insertion inDispatchBest. Executing the resulting bound expression then failed with a type compatibility error.What changes are included in this PR?
MatchConstraintrequiring allcoalescearguments to have the same full decimalDataTypefor exact dispatch.Are these changes tested?
Yes. I ran:
arrow-compute-expression-test --gtest_filter='Expression.BindWithImplicitCastsForCoalesceOnDecimal:Expression.ExecuteCoalesceOnMixedDecimalTypes'arrow-compute-scalar-if-else-test --gtest_filter='TestCoalesce.*:TestCoalesceNumeric.*:TestCoalesceBinary.*:TestCoalesceList.*'The expression tests (2 tests) and complete
TestCoalesceselection (13 tests) passed locally.AI assistance
I used an AI coding assistant to help inspect the existing
MatchConstraintpatterns, draft the implementation and regression tests, and prepare the issue and pull request text. I reviewed and revised the generated changes, reproduced the bug on currentmain, verified the dispatch and expression-binding behavior before and after the fix, and ran the tests listed above. I understand and take responsibility for the submitted changes. No external copyrighted material was incorporated.Are there any user-facing changes?
Yes.
coalesceexpressions with compatible mixed decimal types now bind with casts to a common decimal type and execute successfully instead of failing with a type compatibility error.