Uh oh!
There was an error while loading. Please reload this page.
branch-4.1: [fix](nereids) Guard UniqueFunction in multiple filter/topn pushdown rules #62742 - #65322
Merged
Merged
Conversation
yujun777
commented
Jul 7, 2026
ContributorAuthor
run buildall |
hello-stephen
commented
Jul 7, 2026
Contributor
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
yujun777force-pushed
the
cherry-pick-pr-62742
branch
from
July 7, 2026 09:15
3731847 to
80a014aCompareyujun777
commented
Jul 7, 2026
ContributorAuthor
run buildall |
1 similar comment
yujun777
commented
Jul 7, 2026
ContributorAuthor
run buildall |
…rules (apache#62742) Problem Summary: Several Nereids rewrite rules still moved predicates containing non-idempotent (unique) functions such as rand() / uuid() / random_bytes() across operator boundaries in ways that changed query semantics. The common root cause is that a predicate like `rand() > 0.5` has an empty input-slot set, so the `containsAll(emptySet)` / `allMatch` guards used by these rules silently returned true and allowed unsafe push down / elimination. This PR adds `containsUniqueFunction()` guards to the following rules: 1. PushDownFilterThroughRepeat: skip conjuncts with unique functions. Pushing `rand() > x` below Repeat changes which rows feed each grouping set and alters aggregate results. 2. PushDownFilterThroughWindow: skip conjuncts with unique functions. Pushing a unique predicate below a window operator re-samples the base rows and changes every window-function value. 3. PushDownFilterThroughPartitionTopN: same as Window - skip unique conjuncts in the split loop. 4. PushDownFilterThroughSetOperation: do not push volatile conjuncts below `UNION DISTINCT`, `INTERSECT`, or `EXCEPT`; only `UNION ALL` keeps the original row-to-row semantics. 5. PushDownJoinOtherCondition: keep volatile ON predicates in the join when pushing them into a single child would change evaluation from per joined pair to per input row. 6. AddProjectForVolatileExpression: when a volatile expression is repeated after rewrites such as BETWEEN expansion, materialize it via a child project so repeated references share one value instead of being re-evaluated independently. 7. InferPredicates / JoinUtils.isHashJoinCondition: add the same unique function guards to avoid deriving or classifying unsafe predicates. Fix wrong results when predicates containing rand(), uuid(), random_bytes(), or uuid_numeric() are pushed across Repeat, Window, PartitionTopN, SetOperation, or Join boundaries, and when repeated volatile expressions are materialized for reuse. --------- Co-authored-by: yujun777 <yujun777@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 1d0453a)
yujun777force-pushed
the
cherry-pick-pr-62742
branch
from
July 9, 2026 08:55
6610607 to
ed074b6Compareyujun777
commented
Jul 9, 2026
ContributorAuthor
run buildall |
yujun777
commented
Jul 9, 2026
ContributorAuthor
run p0 |
hello-stephen
commented
Jul 9, 2026
Contributor
FE Regression Coverage ReportIncrement line coverage |
yujun777
commented
Jul 13, 2026
ContributorAuthor
run buildall |
hello-stephen
commented
Jul 13, 2026
Contributor
FE UT Coverage ReportIncrement line coverage |
yujun777
commented
Jul 13, 2026
ContributorAuthor
run p0 |
hello-stephen
commented
Jul 13, 2026
Contributor
FE Regression Coverage ReportIncrement line coverage |
yiguolei
approved these changes
Jul 14, 2026
Contributor
PR approved by at least one committer and no changes requested. |
Contributor
PR approved by anyone and no changes requested. |
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cherry-pick: #62742