Uh oh!
There was an error while loading. Please reload this page.
perf: specialize CASE WHEN for divide-by-zero protection - #24402
Open
shinzoxD wants to merge 1 commit into
Open
Conversation
Detect `CASE WHEN y {>, !=, <} 0 THEN x / y [ELSE NULL]` and evaluate
it by applying the WHEN mask to the divisor (null on excluded rows)
instead of the general CASE filter/scatter path.
Unlike apache#19994, this keeps the original predicate (so `>`, `!=`, and
`<` stay correct) and does not dummy-divide by 1.
Closesapache#11570
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
A common TPC-DS pattern protects against divide-by-zero with:
The general
CaseExprpath (filter + scatter) is expensive for this. #19994 tried replacing it with a fully vectorized always-divide (eq/zip/div/nullif, dummy-dividing zeros by 1). That wins when few denominators are zero, but loses when many are.After #20097 (
ExpressionOrExpressionforCASE WHEN x THEN y [ELSE NULL]) and #20498 (type-specific scatter), the general CASE path is in better shape. This PR specializes the remaining divide-by-zero protection pattern without the #19994 always-divide tradeoff.What changes are included in this PR?
New
EvalMethod::DivideByZeroProtectionthat detects:CASE WHEN y {>, !=, <} 0 THEN x / y [ELSE NULL]including swapped comparisons (
0 < y,0 != y,0 > y) andCast/TryCastwrappers around the checked operand / divisor.Evaluation keeps the original WHEN predicate so
>,!=, and<stay correct, then:nullif)Only applied when:
ELSE NULLCast/TryCast)Unlike #19994, this does not dummy-divide by 1 and does not collapse
>/</!=into a single "is zero" mask.Are these changes tested?
Yes.
Unit tests in
datafusion/physical-expr/src/expressions/case.rs:test_divide_by_zero_protection_specialization: pattern detection + results withCasttest_divide_by_zero_protection_predicates:!=,>,<, and swapped0 != dtest_divide_by_zero_protection_all_true_all_false_and_nullstest_divide_by_zero_protection_specialization_not_applied: WHENa/ divisorbmust not specializetest_divide_by_zero_protection_not_applied_with_else:ELSE 0must not specializeSLTs in
datafusion/sqllogictest/test_files/case.slt(existing>,!=,<cases from the #19994 review, plus):WHEN 0 < dBench comment in
datafusion/physical-expr/benches/case_when.rsupdated to match the specialization trigger (!= 0).Are there any user-facing changes?
No. Internal optimization; results are unchanged.
Testing
case.slt>,!=,<divide-by-zero rows still correctcargo fmt --all/cargo clippy --all-targets --all-features -- -D warnings(CI)