Uh oh!
There was an error while loading. Please reload this page.
[Opt](multi-catalog)Disable dict filter in parquet/orc reader if have non-single conjuncts. - #44777
Conversation
doris-robot
commented
Nov 29, 2024
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
kaka11chen
commented
Nov 29, 2024
run buildall |
6b0d1b8 to
1f8ccb3Comparekaka11chen
commented
Nov 29, 2024
run buildall |
1f8ccb3 to
3c6b546Comparekaka11chen
commented
Nov 29, 2024
run buildall |
doris-robot
commented
Nov 29, 2024
TeamCity be ut coverage result: |
doris-robot
commented
Nov 29, 2024
TPC-H: Total hot run time: 40268 ms |
doris-robot
commented
Nov 29, 2024
TPC-DS: Total hot run time: 197585 ms |
doris-robot
commented
Nov 29, 2024
ClickBench: Total hot run time: 32.7 s |
kaka11chen
commented
Dec 30, 2024
run cloud_p0 |
3c6b546 to
8640b08Comparekaka11chen
commented
Apr 28, 2025
run buildall |
hello-stephen
commented
Apr 28, 2025
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Apr 28, 2025
BE Regression P0 && UT Coverage ReportIncrement line coverage Increment coverage report
|
… non-single conjuncts.
8640b08 to
1c37449Comparemorningman
commented
May 16, 2025
run buildall |
doris-robot
commented
May 16, 2025
TPC-H: Total hot run time: 33738 ms |
doris-robot
commented
May 16, 2025
TPC-DS: Total hot run time: 193042 ms |
doris-robot
commented
May 16, 2025
ClickBench: Total hot run time: 28.79 s |
doris-robot
commented
May 16, 2025
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
May 16, 2025
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
… non-single conjuncts. (apache#44777) Related PR: apache#26386 Problem Summary: Because of apache#26386, we split the conjunctions into `single_slot_filter_conjuncts` and `non_single_slot_filter_conjuncts`, where you can use the final dictionary filtering and delayed materialization of the `single_slot_filter_conjuncts` part to perform `non_single_slot_filter_conjuncts`. However, this results in fewer conditions for late materialization, so the effect will be poor.
… non-single conjuncts. (apache#44777) ### What problem does this PR solve? Related PR: apache#26386 Problem Summary: Because of apache#26386, we split the conjunctions into `single_slot_filter_conjuncts` and `non_single_slot_filter_conjuncts`, where you can use the final dictionary filtering and delayed materialization of the `single_slot_filter_conjuncts` part to perform `non_single_slot_filter_conjuncts`. However, this results in fewer conditions for late materialization, so the effect will be poor.
… non-single conjuncts. (apache#44777) Related PR: apache#26386 Problem Summary: Because of apache#26386, we split the conjunctions into `single_slot_filter_conjuncts` and `non_single_slot_filter_conjuncts`, where you can use the final dictionary filtering and delayed materialization of the `single_slot_filter_conjuncts` part to perform `non_single_slot_filter_conjuncts`. However, this results in fewer conditions for late materialization, so the effect will be poor.
… non-single conjuncts. (apache#44777) Related PR: apache#26386 Problem Summary: Because of apache#26386, we split the conjunctions into `single_slot_filter_conjuncts` and `non_single_slot_filter_conjuncts`, where you can use the final dictionary filtering and delayed materialization of the `single_slot_filter_conjuncts` part to perform `non_single_slot_filter_conjuncts`. However, this results in fewer conditions for late materialization, so the effect will be poor.
…ader if have non-single conjuncts apache#44777 (apache#52559) Cherry-picked from apache#44777
… non-single conjuncts. (apache#44777) Related PR: apache#26386 Problem Summary: Because of apache#26386, we split the conjunctions into `single_slot_filter_conjuncts` and `non_single_slot_filter_conjuncts`, where you can use the final dictionary filtering and delayed materialization of the `single_slot_filter_conjuncts` part to perform `non_single_slot_filter_conjuncts`. However, this results in fewer conditions for late materialization, so the effect will be poor.
… non-single conjuncts. (apache#320) Cherry-pick apache#44777
What problem does this PR solve?
Related PR: #26386
Problem Summary:
Because of #26386, we split the conjunctions into
single_slot_filter_conjunctsandnon_single_slot_filter_conjuncts, where you can use the final dictionary filtering and delayed materialization of thesingle_slot_filter_conjunctspart to performnon_single_slot_filter_conjuncts. However, this results in fewer conditions for late materialization, so the effect will be poor.Release note
At present, it is a matter of how many conditions there are for late materialization and whether to perform dictionary filtering on multiple columns. Because late materialization is more important, we use when there are
non_single_slot_filter_conjunctsin the filter, directly without dictionary filtering, and use the conjuncts to do late materialization.None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)