Uh oh!
There was an error while loading. Please reload this page.
[fix](mtmv) Fix compensate union wrongly when direct query is empty relation - #51700
Conversation
hello-stephen
commented
Jun 13, 2025
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
seawinde
commented
Jun 13, 2025
run buildall |
| return context.getRewritePlan(); | ||
| }, queryPlanWithUnionFilter, queryPlan), true); | ||
| }, queryPlanWithUnionFilter, queryPlan); | ||
| // need to collect table partition again |
There was a problem hiding this comment.
write why need re-collect here in comment
| // this is needed before init hook | ||
| collectTableUsedPartitions(showPlanProcess); | ||
| // this is needed before init hook, because | ||
| MaterializedViewUtils.collectTableUsedPartitions(cascadesContext.getRewritePlan(), cascadesContext); |
| }, queryPlanWithUnionFilter, queryPlan), true); | ||
| }, queryPlanWithUnionFilter, queryPlan); | ||
| // need to collect table partition again | ||
| MaterializedViewUtils.collectTableUsedPartitions(filterAddedPlan, parentCascadesContext); |
There was a problem hiding this comment.
if u need collect partition info multi times, u should sum the time used by it in summary profile
| }, queryPlanWithUnionFilter, queryPlan), true); | ||
| }, queryPlanWithUnionFilter, queryPlan); | ||
| // need to collect table partition again | ||
| MaterializedViewUtils.collectTableUsedPartitions(filterAddedPlan, parentCascadesContext); |
There was a problem hiding this comment.
only collect for sub plan is enough?
There was a problem hiding this comment.
should collect table partition for all rewritten plan by mv, have fixed it
doris-robot
commented
Jun 13, 2025
TPC-H: Total hot run time: 35905 ms |
hello-stephen
commented
Jun 13, 2025
FE UT Coverage ReportIncrement line coverage |
hello-stephen
commented
Jun 13, 2025
run external 10 |
doris-robot
commented
Jun 13, 2025
TPC-H: Total hot run time: 36238 ms |
hello-stephen
commented
Jun 16, 2025
run external 5 |
4a2a8c6 to
b95c035Compareseawinde
commented
Jun 16, 2025
run buildall |
doris-robot
commented
Jun 16, 2025
TPC-H: Total hot run time: 34052 ms |
seawinde
commented
Jun 16, 2025
run buildall |
doris-robot
commented
Jun 16, 2025
TPC-H: Total hot run time: 33763 ms |
seawinde
commented
Jun 16, 2025
run buildall |
hello-stephen
commented
Jun 16, 2025
FE UT Coverage ReportIncrement line coverage |
seawinde
commented
Jun 16, 2025
run external 10 |
doris-robot
commented
Jun 16, 2025
TPC-H: Total hot run time: 34190 ms |
doris-robot
commented
Jun 16, 2025
TPC-DS: Total hot run time: 184911 ms |
doris-robot
commented
Jun 16, 2025
ClickBench: Total hot run time: 28.89 s |
seawinde
commented
Jun 17, 2025
run buildall |
c9108e7 to
fe19d56Compareseawinde
commented
Jun 17, 2025
run buildall |
doris-robot
commented
Jun 17, 2025
TPC-H: Total hot run time: 33682 ms |
…elation (apache#51700) if query is as following which contains external table, if some partition in external table is empty, when mv partiton is invalid, this would cause compensate many union all empty relation ```sql SELECT * FROM ${catalogName}.`test_paimon_spark`.test_tb_mix_format a left join ${tableName} b on a.id=b.user_id; ```
…elation (apache#51700) if query is as following which contains external table, if some partition in external table is empty, when mv partiton is invalid, this would cause compensate many union all empty relation ```sql SELECT * FROM ${catalogName}.`test_paimon_spark`.test_tb_mix_format a left join ${tableName} b on a.id=b.user_id; ```
…elation (apache#51700) if query is as following which contains external table, if some partition in external table is empty, when mv partiton is invalid, this would cause compensate many union all empty relation ```sql SELECT * FROM ${catalogName}.`test_paimon_spark`.test_tb_mix_format a left join ${tableName} b on a.id=b.user_id; ```
…y is empty relation apache#51700 (apache#51898) Cherry-picked apache#51700
What problem does this PR solve?
if query is as following which contains external table, if some partition in external table is empty, when mv partiton is invalid, this would cause compensate many union all empty relation
Issue Number: close #xxx
Related PR: #49514
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)