Uh oh!
There was an error while loading. Please reload this page.
[fix](mtmv) Fix hudi materialized view union all rewritten plan execute fail because of invalid slot - #58643
Conversation
…te fail because of invalid slot
Thearas
commented
Dec 2, 2025
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
seawinde
commented
Dec 2, 2025
run buildall |
doris-robot
commented
Dec 2, 2025
TPC-H: Total hot run time: 34710 ms |
doris-robot
commented
Dec 2, 2025
TPC-DS: Total hot run time: 182427 ms |
doris-robot
commented
Dec 2, 2025
ClickBench: Total hot run time: 27.77 s |
hello-stephen
commented
Dec 2, 2025
FE UT Coverage ReportIncrement line coverage |
hello-stephen
commented
Dec 2, 2025
FE Regression Coverage ReportIncrement line coverage |
seawinde
commented
Dec 3, 2025
run buildall |
doris-robot
commented
Dec 3, 2025
TPC-H: Total hot run time: 34254 ms |
doris-robot
commented
Dec 3, 2025
TPC-DS: Total hot run time: 181466 ms |
doris-robot
commented
Dec 3, 2025
ClickBench: Total hot run time: 27.3 s |
hello-stephen
commented
Dec 3, 2025
FE UT Coverage ReportIncrement line coverage |
hello-stephen
commented
Dec 3, 2025
FE Regression Coverage ReportIncrement line coverage |
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
Uh oh!
There was an error while loading. Please reload this page.
…te fail because of invalid slot (apache#58643) Related PR: apache#57558apache#58413 Problem Summary: This fix addresses the following three issues: 1. When invoking the method org.apache.doris.nereids.trees.plans.logical.LogicalHudiScan#withRelationId, the output needs to be recalculated to meet expectations. 2. After compensating with a union all due to partial partition invalidation of a materialized view, during the next round of transparent rewriting, the rewriting for the child of the union allshould use the query partitioncorresponding to the specific relation id to prevent infinite loops. 3. Currently, in the `test_hudi_rewrite_mtmv` test, if the plan rewritten by the materialized view transparent rewriting is not selected by the CBO, it is difficult to troubleshoot because explain memo planis not used. Therefore, the corresponding test method is modified.
…te fail because of invalid slot (apache#58643) Related PR: apache#57558apache#58413 Problem Summary: This fix addresses the following three issues: 1. When invoking the method org.apache.doris.nereids.trees.plans.logical.LogicalHudiScan#withRelationId, the output needs to be recalculated to meet expectations. 2. After compensating with a union all due to partial partition invalidation of a materialized view, during the next round of transparent rewriting, the rewriting for the child of the union allshould use the query partitioncorresponding to the specific relation id to prevent infinite loops. 3. Currently, in the `test_hudi_rewrite_mtmv` test, if the plan rewritten by the materialized view transparent rewriting is not selected by the CBO, it is difficult to troubleshoot because explain memo planis not used. Therefore, the corresponding test method is modified.
…te fail because of invalid slot (apache#58643) ### What problem does this PR solve? Related PR: apache#57558apache#58413 Problem Summary: This fix addresses the following three issues: 1. When invoking the method org.apache.doris.nereids.trees.plans.logical.LogicalHudiScan#withRelationId, the output needs to be recalculated to meet expectations. 2. After compensating with a union all due to partial partition invalidation of a materialized view, during the next round of transparent rewriting, the rewriting for the child of the union allshould use the query partitioncorresponding to the specific relation id to prevent infinite loops. 3. Currently, in the `test_hudi_rewrite_mtmv` test, if the plan rewritten by the materialized view transparent rewriting is not selected by the CBO, it is difficult to troubleshoot because explain memo planis not used. Therefore, the corresponding test method is modified.
### What problem does this PR solve? Issue Number: N/A Related PR: apache#49514, apache#58643 Problem Summary: The existing unit test only checked stale partition mappings after the mocked executor returned, so it could not distinguish cleanup before planning from cleanup after execution. Add a focused test that simulates current planning state being written during each EXECUTE and verifies that only the current RelationId remains. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-fe-ut.sh --run org.apache.doris.nereids.trees.plans.commands.ExecuteCommandTest - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: N/A Related PR: apache#49514, apache#58643 Problem Summary: The existing unit test only checked stale partition mappings after the mocked executor returned, so it could not distinguish cleanup before planning from cleanup after execution. Add a focused test that simulates current planning state being written during each EXECUTE and verifies that only the current RelationId remains. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-fe-ut.sh --run org.apache.doris.nereids.trees.plans.commands.ExecuteCommandTest - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: N/A Related PR: apache#49514, apache#58643 Problem Summary: The existing unit test only checked stale partition mappings after the mocked executor returned, so it could not distinguish cleanup before planning from cleanup after execution. Add a focused test that simulates current planning state being written during each EXECUTE and verifies that only the current RelationId remains. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-fe-ut.sh --run org.apache.doris.nereids.trees.plans.commands.ExecuteCommandTest - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: N/A Related PR: apache#49514, apache#58643 Problem Summary: The existing unit test only checked stale partition mappings after the mocked executor returned, so it could not distinguish cleanup before planning from cleanup after execution. Add a focused test that simulates current planning state being written during each EXECUTE and verifies that only the current RelationId remains. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-fe-ut.sh --run org.apache.doris.nereids.trees.plans.commands.ExecuteCommandTest - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: N/A Related PR: apache#49514, apache#58643 Problem Summary: Proxy lazy parsing installed a new StatementContext only on ConnectContext, so query retries reset the executor old context while planning reused stale materialized-view state. Streaming insert preflight and task planning also entered initPlan as separate first attempts on the same context, leaving preflight MV outputs available to task planning. Synchronize proxy executors with the parsed LogicalPlanAdapter context and reset the existing MV-only planning-attempt state at every insert planning attempt, including first entry. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-fe-ut.sh --run org.apache.doris.qe.StmtExecutorTest,org.apache.doris.nereids.trees.plans.commands.ExecuteCommandTest,org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommandRetryTest,org.apache.doris.nereids.rules.exploration.mv.PartitionCompensatorTest (51 passed) - Behavior changed: Yes, proxy retries and reused insert planning contexts discard stale MV planning-attempt state. - Does this need documentation: No
### What problem does this PR solve? Issue Number: N/A Related PR: apache#49514, apache#58643 Problem Summary: The existing unit test only checked stale partition mappings after the mocked executor returned, so it could not distinguish cleanup before planning from cleanup after execution. Add a focused test that simulates current planning state being written during each EXECUTE and verifies that only the current RelationId remains. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-fe-ut.sh --run org.apache.doris.nereids.trees.plans.commands.ExecuteCommandTest - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: N/A Related PR: apache#49514, apache#58643 Problem Summary: Proxy lazy parsing installed a new StatementContext only on ConnectContext, so query retries reset the executor old context while planning reused stale materialized-view state. Streaming insert preflight and task planning also entered initPlan as separate first attempts on the same context, leaving preflight MV outputs available to task planning. Synchronize proxy executors with the parsed LogicalPlanAdapter context and reset the existing MV-only planning-attempt state at every insert planning attempt, including first entry. ### Release note None ### Check List (For Author) - Test: Unit Test - ./run-fe-ut.sh --run org.apache.doris.qe.StmtExecutorTest,org.apache.doris.nereids.trees.plans.commands.ExecuteCommandTest,org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommandRetryTest,org.apache.doris.nereids.rules.exploration.mv.PartitionCompensatorTest (51 passed) - Behavior changed: Yes, proxy retries and reused insert planning contexts discard stale MV planning-attempt state. - Does this need documentation: No
What problem does this PR solve?
This fix addresses the following three issues:
test_hudi_rewrite_mtmvtest, if the plan rewritten by the materialized view transparent rewriting is not selected by the CBO, it is difficult to troubleshoot because explain memo planis not used. Therefore, the corresponding test method is modified.Issue Number: close #xxx
Related PR:
#57558
#58413
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)