Skip to content

[fix](Nereids) non-slot filter should not be push through aggregate - #25525

Merged
morrySnow merged 1 commit into
apache:masterfrom
morrySnow:push_slot_to_agg
Oct 17, 2023
Merged

[fix](Nereids) non-slot filter should not be push through aggregate#25525
morrySnow merged 1 commit into
apache:masterfrom
morrySnow:push_slot_to_agg

Conversation

@morrySnow

@morrySnowmorrySnow commented Oct 17, 2023

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@morrySnow

morrySnow commented Oct 17, 2023

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Oct 17, 2023

Copy link
Copy Markdown

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.38 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162016505 Bytes

englefly
englefly reviewed Oct 17, 2023
View reviewed changes
Comment threadregression-test/suites/nereids_rules_p0/filter_push_down/filter_push_through_aggregate.groovy
sql "set disable_nereids_rules='REWRITE_FILTER_EXPRESSION'"

// 'a' = 'b' should not be push down aggregate.
qt_do_not_push_no_slot_filter """

@engleflyengleflyOct 17, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this case can not verify that a='b' is pushed down or not.
we need to insert some tuple and check the query result. before this pr, sum column is NULL

@morrySnowmorrySnowOct 17, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a = 'b' will generate 'a' = 'b'

@englefly

englefly commented Oct 17, 2023

Copy link
Copy Markdown
Contributor

join suffers the same issue.

@morrySnow

morrySnow commented Oct 17, 2023

Copy link
Copy Markdown
ContributorAuthor

join suffers the same issue.

fix in another PR

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 17, 2023
@github-actions

github-actionsBot commented Oct 17, 2023

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Oct 17, 2023

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@morrySnow
morrySnow merged commit 9d6b2dc into apache:masterOct 17, 2023
xiaokang pushed a commit that referenced this pull request Oct 17, 2023
@morrySnow@xiaokang
[fix](Nereids) non-slot filter should not be push through aggregate (#…
1016ed2
…25525)
dutyu pushed a commit to dutyu/doris that referenced this pull request Oct 28, 2023
@morrySnow王翔宇
[fix](Nereids) non-slot filter should not be push through aggregate (a…
ee4c29c
…pache#25525)
@morrySnow
morrySnow deleted the push_slot_to_agg branch October 30, 2023 03:15
@xiaokangxiaokang mentioned this pull request Dec 4, 2023
Closed
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
@morrySnow胥剑旭
[fix](Nereids) non-slot filter should not be push through aggregate (a…
0c0c3fa
…pache#25525)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@engleflyengleflyenglefly approved these changes
+1 more reviewer
@keanji-xkeanji-xkeanji-x approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.0.3-mergedp0_wreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@morrySnow@doris-robot@englefly@keanji-x@xiaokang