[opt](nereids)flattern and/or by englefly · Pull Request #44574 · apache/doris

englefly · 2024-11-25T13:13:37Z

What problem does this PR solve?

And/or expressions are represented by binary trees. The depth of the tree causes stack overflow in recursive program calls. To solve this problem, this pr will flatten the binary tree when processing and/or, reducing the number of recursions. At the same time, we also rewrite some recursive programs into non-recursive forms to avoid stack overflow.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

doris-robot · 2024-11-25T13:13:43Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

englefly · 2024-11-25T13:13:44Z

run buildall

englefly · 2024-11-26T05:15:48Z

run buildall

englefly · 2024-11-26T06:16:08Z

run buildall

englefly · 2024-11-26T12:34:31Z

run buildall

englefly · 2024-11-27T06:33:37Z

run buildall

englefly · 2024-11-28T03:16:53Z

run buildall

englefly · 2024-11-28T12:16:33Z

run buildall

englefly · 2024-11-28T13:29:32Z

run buildall

englefly · 2024-11-28T14:37:49Z

run buildall

englefly · 2024-11-28T16:27:49Z

run buildall

englefly · 2024-11-29T05:58:32Z

run buildall

englefly · 2024-11-29T14:41:15Z

run buildall

englefly · 2024-11-29T15:28:13Z

run compile

englefly · 2024-11-30T03:15:19Z

run buildall

englefly · 2024-11-30T05:08:49Z

run cloud_p0

github-actions · 2024-11-30T05:16:54Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-11-30T05:16:56Z

PR approved by anyone and no changes requested.

eldenmoon

lgtm

fix prune partitions wrong, introduced by #44574

fix prune partitions wrong, introduced by apache#44574

### What problem does this PR solve? #44574 make AND's child size may exceeds 2, need consider.

fix prune partitions wrong, introduced by apache#44574

### What problem does this PR solve? apache#44574 make AND's child size may exceeds 2, need consider.

pick #41222 #46468 #44574 #48034 #50265 #47385

### What problem does this PR solve? previous pr #44574 removes Expression.checkLimit(). this function is still useful, add it back

### What problem does this PR solve? previous pr apache#44574 removes Expression.checkLimit(). this function is still useful, add it back

### What problem does this PR solve? And/or expressions are represented by binary trees. The depth of the tree causes stack overflow in recursive program calls. To solve this problem, this pr will flatten the binary tree when processing and/or, reducing the number of recursions. At the same time, we also rewrite some recursive programs into non-recursive forms to avoid stack overflow. Issue

englefly marked this pull request as draft November 25, 2024 13:13

englefly force-pushed the huge-binary-v4 branch from e811e76 to 80929c0Compare November 26, 2024 06:15

englefly marked this pull request as ready for review November 27, 2024 14:29

englefly force-pushed the huge-binary-v4 branch from af72324 to 0d50edeCompare November 28, 2024 15:10

englefly added 4 commits November 29, 2024 10:46

flattern and/or
6c35960

cancel cmd
47e7676

upate shape
4f76666

fix fold-const/cancel
97bd558

englefly force-pushed the huge-binary-v4 branch from 2f6a040 to 97bd558Compare November 29, 2024 05:58

fold const
dfa39da

1.cancel export, 2 huge-compound case
80faacd

starocean999 approved these changes Nov 30, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 30, 2024

github-actionsBot added the reviewed label Nov 30, 2024

eldenmoon approved these changes Nov 30, 2024
View reviewed changes

englefly merged commit 42a7734 into apache:masterNov 30, 2024

924060929 mentioned this pull request Feb 18, 2025
[fix](nereids) fix prune partitions wrong #48034
Merged
16 tasks

924060929 added a commit that referenced this pull request Feb 19, 2025
[fix](nereids) fix prune partitions wrong (#48034)
2a76ef9
fix prune partitions wrong, introduced by #44574

lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
[fix](nereids) fix prune partitions wrong (apache#48034)
7f7c126
fix prune partitions wrong, introduced by apache#44574

morrySnow mentioned this pull request Apr 22, 2025
[fix](nereids) fix AND extract uniform slots #50265
Merged
16 tasks

englefly pushed a commit that referenced this pull request Apr 24, 2025
[fix](nereids) fix AND extract uniform slots (#50265)
9ccfab2
### What problem does this PR solve? #44574 make AND's child size may exceeds 2, need consider.

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
[fix](nereids) fix prune partitions wrong (apache#48034)
3aac59a
fix prune partitions wrong, introduced by apache#44574

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
[fix](nereids) fix AND extract uniform slots (apache#50265)
40bad3f
### What problem does this PR solve? apache#44574 make AND's child size may exceeds 2, need consider.

englefly mentioned this pull request Jun 18, 2025
branch-3.1: [opt](nereids) infer in-predicate from or-predicate (#46468) #51869
Merged

morrySnow pushed a commit that referenced this pull request Jun 20, 2025
branch-3.1: [opt](nereids) infer in-predicate from or-predicate (#51869)
622e21e
pick #41222 #46468 #44574 #48034 #50265 #47385

morrySnow added the dev/3.1.0-merged label Jun 20, 2025

englefly mentioned this pull request Jun 26, 2025
[fix](nereids) add "Check expression limit" back #52378
Merged
16 tasks

englefly added a commit that referenced this pull request Jul 1, 2025
[fix](nereids) add "Check expression limit" back (#52378)
d58e068
### What problem does this PR solve? previous pr #44574 removes Expression.checkLimit(). this function is still useful, add it back

github-actionsBot pushed a commit that referenced this pull request Jul 1, 2025
[fix](nereids) add "Check expression limit" back (#52378)
0d9d743
### What problem does this PR solve? previous pr #44574 removes Expression.checkLimit(). this function is still useful, add it back

koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
[fix](nereids) add "Check expression limit" back (apache#52378)
82eff54
### What problem does this PR solve? previous pr apache#44574 removes Expression.checkLimit(). this function is still useful, add it back

koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
[fix](nereids) add "Check expression limit" back (apache#52378)
c752af8
### What problem does this PR solve? previous pr apache#44574 removes Expression.checkLimit(). this function is still useful, add it back

koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
[fix](nereids) add "Check expression limit" back (apache#52378)
f63ef32
### What problem does this PR solve? previous pr apache#44574 removes Expression.checkLimit(). this function is still useful, add it back

feiniaofeiafei mentioned this pull request Aug 8, 2025
[fix](nereids) set CompoundPredicate.nullableFromNereids after flattern and/or #54353
Merged
16 tasks

Uh oh!

Conversation

englefly commented Nov 25, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

doris-robot commented Nov 25, 2024

Uh oh!

englefly commented Nov 25, 2024

Uh oh!

englefly commented Nov 26, 2024

Uh oh!

englefly commented Nov 26, 2024

Uh oh!

englefly commented Nov 26, 2024

Uh oh!

englefly commented Nov 27, 2024

Uh oh!

englefly commented Nov 28, 2024

Uh oh!

englefly commented Nov 28, 2024

Uh oh!

englefly commented Nov 28, 2024

Uh oh!

englefly commented Nov 28, 2024

Uh oh!

englefly commented Nov 28, 2024

Uh oh!

englefly commented Nov 29, 2024

Uh oh!

englefly commented Nov 29, 2024

Uh oh!

englefly commented Nov 29, 2024

Uh oh!

englefly commented Nov 30, 2024

Uh oh!

englefly commented Nov 30, 2024

Uh oh!

github-actionsBot commented Nov 30, 2024

Uh oh!

github-actionsBot commented Nov 30, 2024

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

englefly commented Nov 25, 2024•
edited
Loading