[fix](partition_prune) Move the pruning of predicates that are always true after partition pruning into the PlanPostProcessor by feiniaofeiafei · Pull Request #63111 · apache/doris

feiniaofeiafei · 2026-05-09T12:05:08Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #57169

Problem Summary:
After partition pruning, predicates that are evaluated to constant true are removed as an optimization (introduced by #57169). However, this introduced a bug in materialized view rewriting: when such predicates are removed, they are not compensated above the rewritten materialized view. Since the materialized view itself is not partitioned, this leads to incorrect query results.

This PR fixes the issue by moving the removal of constant-true predicates to the planPostProcessors phase. This ensures that the predicate removal does not interfere with materialized view rewriting, preserving correctness while still retaining the optimization benefit.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-05-09T12:05:14Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

feiniaofeiafei · 2026-05-09T12:06:42Z

run buildall

feiniaofeiafei · 2026-05-09T12:39:53Z

run buildall

feiniaofeiafei · 2026-05-09T12:44:39Z

run buildall

feiniaofeiafei · 2026-05-11T07:10:32Z

run buildall

hello-stephen · 2026-05-11T07:28:37Z

TPC-H: Total hot run time: 29511 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7836b856c1b48e3431234a73507a0d7ed23a80d8, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17934	3893	3849	3849
q2	q3	10733	870	602	602
q4	4665	462	344	344
q5	7451	1304	1152	1152
q6	184	168	145	145
q7	897	953	743	743
q8	9310	1401	1236	1236
q9	5552	5423	5320	5320
q10	6262	2076	1817	1817
q11	461	262	252	252
q12	630	403	290	290
q13	18126	3254	2725	2725
q14	294	284	259	259
q15	q16	855	859	780	780
q17	971	1009	731	731
q18	6474	5786	5614	5614
q19	1158	1206	1106	1106
q20	494	395	255	255
q21	5040	2352	1920	1920
q22	474	425	371	371
Total cold run time: 97965 ms
Total hot run time: 29511 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4813	4672	4871	4672
q2	q3	4682	4751	4180	4180
q4	2142	2177	1380	1380
q5	4944	4992	5260	4992
q6	197	162	129	129
q7	2076	1793	1582	1582
q8	3317	3082	3122	3082
q9	8461	8449	8460	8449
q10	4450	4448	4234	4234
q11	581	426	411	411
q12	704	777	521	521
q13	3252	3495	3054	3054
q14	328	314	280	280
q15	q16	757	804	703	703
q17	1331	1306	1244	1244
q18	7937	7222	6956	6956
q19	1142	1117	1141	1117
q20	2215	2270	2000	2000
q21	6130	5359	4859	4859
q22	542	510	414	414
Total cold run time: 60001 ms
Total hot run time: 54259 ms

hello-stephen · 2026-05-11T07:39:32Z

TPC-DS: Total hot run time: 170534 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7836b856c1b48e3431234a73507a0d7ed23a80d8, data reload: false
query5	4336	645	505	505
query6	327	216	200	200
query7	4241	564	307	307
query8	333	255	218	218
query9	8796	4007	3990	3990
query10	451	350	294	294
query11	5798	2389	2180	2180
query12	180	132	128	128
query13	1279	610	419	419
query14	5976	5407	5067	5067
query14_1	4393	4412	4382	4382
query15	219	204	187	187
query16	1007	454	360	360
query17	1161	778	640	640
query18	2503	496	376	376
query19	227	213	175	175
query20	143	134	136	134
query21	215	141	118	118
query22	13757	13494	13351	13351
query23	17178	16427	16082	16082
query23_1	16122	16234	16107	16107
query24	7487	1756	1353	1353
query24_1	1356	1344	1363	1344
query25	612	549	474	474
query26	1313	320	172	172
query27	2679	591	350	350
query28	4435	1979	1981	1979
query29	1020	660	559	559
query30	312	241	203	203
query31	1124	1078	936	936
query32	90	82	78	78
query33	569	366	311	311
query34	1157	1147	634	634
query35	764	794	709	709
query36	1334	1358	1147	1147
query37	146	103	93	93
query38	3221	3112	3048	3048
query39	913	938	891	891
query39_1	876	883	898	883
query40	229	154	131	131
query41	65	63	62	62
query42	108	107	105	105
query43	317	324	284	284
query44 query45	211	193	190	190
query46	1089	1202	723	723
query47	2281	2371	2148	2148
query48	403	415	305	305
query49	643	561	433	433
query50	719	285	221	221
query51	4309	4307	4199	4199
query52	106	112	94	94
query53	252	278	211	211
query54	306	269	256	256
query55	94	88	86	86
query56	300	311	292	292
query57	1422	1406	1314	1314
query58	295	277	264	264
query59	1588	1613	1435	1435
query60	348	359	325	325
query61	158	154	153	153
query62	665	617	565	565
query63	239	213	211	211
query64	2450	819	688	688
query65 query66	1749	539	400	400
query67	29935	29857	29819	29819
query68 query69	464	348	302	302
query70	1018	1027	1001	1001
query71	308	278	275	275
query72	2905	2717	2436	2436
query73	816	769	424	424
query74	5050	4950	4726	4726
query75	2754	2629	2333	2333
query76	2282	1102	787	787
query77	415	425	337	337
query78	12976	12959	12316	12316
query79	1477	976	753	753
query80	845	595	484	484
query81	485	278	241	241
query82	1334	156	119	119
query83	357	276	243	243
query84	269	135	108	108
query85	962	525	443	443
query86	436	321	307	307
query87	3405	3329	3227	3227
query88	3487	2655	2631	2631
query89	440	383	337	337
query90	1884	181	175	175
query91	177	157	137	137
query92	80	85	72	72
query93	973	958	556	556
query94	618	335	304	304
query95	653	371	342	342
query96	983	789	344	344
query97	2708	2688	2598	2598
query98	241	230	246	230
query99	1130	1136	1027	1027
Total cold run time: 253271 ms
Total hot run time: 170534 ms

feiniaofeiafei · 2026-05-11T13:24:14Z

run buildall

hello-stephen · 2026-05-11T14:52:49Z

TPC-H: Total hot run time: 29722 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c4285dd1e1e0b0a698bc81ff430ac62bff58cb5b, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17703	3965	3920	3920
q2	q3	10724	867	606	606
q4	4669	458	345	345
q5	7459	1312	1141	1141
q6	192	182	144	144
q7	930	933	760	760
q8	9487	1411	1285	1285
q9	5873	5448	5330	5330
q10	6286	2086	1804	1804
q11	475	279	262	262
q12	685	410	295	295
q13	18209	3318	2745	2745
q14	301	287	259	259
q15	q16	899	875	787	787
q17	1048	1130	824	824
q18	6557	5739	5671	5671
q19	1616	1269	1092	1092
q20	531	393	263	263
q21	4616	2284	1889	1889
q22	420	352	300	300
Total cold run time: 98680 ms
Total hot run time: 29722 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4284	4263	4181	4181
q2	q3	4623	4774	4180	4180
q4	2119	2176	1384	1384
q5	4950	5017	5262	5017
q6	187	165	132	132
q7	2042	2127	1653	1653
q8	3465	3220	3139	3139
q9	8494	8468	8456	8456
q10	4440	4498	4255	4255
q11	624	434	418	418
q12	705	747	516	516
q13	3248	3569	2863	2863
q14	306	310	275	275
q15	q16	793	793	698	698
q17	1349	1316	1350	1316
q18	8058	7207	7164	7164
q19	1172	1155	1180	1155
q20	2244	2265	1974	1974
q21	6256	5538	4892	4892
q22	560	525	428	428
Total cold run time: 59919 ms
Total hot run time: 54096 ms

hello-stephen · 2026-05-11T15:03:47Z

TPC-DS: Total hot run time: 170733 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c4285dd1e1e0b0a698bc81ff430ac62bff58cb5b, data reload: false
query5	4340	641	523	523
query6	325	209	201	201
query7	4213	578	299	299
query8	327	244	220	220
query9	8809	4002	3984	3984
query10	471	350	301	301
query11	6036	2365	2214	2214
query12	183	130	125	125
query13	1324	637	434	434
query14	6796	5364	5080	5080
query14_1	4381	4342	4372	4342
query15	214	209	181	181
query16	1000	444	428	428
query17	1161	759	630	630
query18	2748	489	368	368
query19	263	210	178	178
query20	142	134	133	133
query21	221	141	123	123
query22	13668	13972	14528	13972
query23	17490	16603	16197	16197
query23_1	16395	16244	16568	16244
query24	7886	1786	1367	1367
query24_1	1348	1344	1390	1344
query25	618	476	432	432
query26	1314	308	166	166
query27	2675	576	334	334
query28	4341	1926	1914	1914
query29	987	621	512	512
query30	295	237	195	195
query31	1108	1057	921	921
query32	79	71	67	67
query33	530	340	292	292
query34	1165	1110	646	646
query35	755	773	649	649
query36	1333	1388	1214	1214
query37	147	96	127	96
query38	3151	3112	3086	3086
query39	921	930	889	889
query39_1	882	868	868	868
query40	228	155	139	139
query41	68	59	60	59
query42	110	105	107	105
query43	330	339	302	302
query44 query45	222	208	199	199
query46	1115	1225	725	725
query47	2386	2389	2366	2366
query48	426	411	291	291
query49	625	545	421	421
query50	710	281	214	214
query51	4342	4311	4245	4245
query52	106	103	92	92
query53	246	266	203	203
query54	308	277	245	245
query55	91	86	84	84
query56	304	315	289	289
query57	1413	1398	1318	1318
query58	294	268	258	258
query59	1539	1658	1383	1383
query60	345	338	317	317
query61	158	154	156	154
query62	683	619	574	574
query63	243	197	202	197
query64	2370	835	669	669
query65 query66	1671	519	379	379
query67	29909	29923	29770	29770
query68 query69	446	346	303	303
query70	989	982	952	952
query71	300	279	265	265
query72	2916	2725	2547	2547
query73	839	745	408	408
query74	5085	4907	4702	4702
query75	2780	2644	2334	2334
query76	2290	1124	759	759
query77	406	404	348	348
query78	13035	12933	12393	12393
query79	1382	956	731	731
query80	1198	581	485	485
query81	521	285	241	241
query82	451	157	121	121
query83	382	266	241	241
query84	259	136	113	113
query85	882	520	455	455
query86	435	314	307	307
query87	3388	3314	3250	3250
query88	3497	2659	2635	2635
query89	430	379	341	341
query90	2048	182	180	180
query91	179	172	139	139
query92	79	76	70	70
query93	1121	958	547	547
query94	694	340	292	292
query95	638	474	349	349
query96	1035	780	324	324
query97	2725	2738	2609	2609
query98	248	234	224	224
query99	1121	1130	995	995
Total cold run time: 254890 ms
Total hot run time: 170733 ms

hello-stephen · 2026-05-11T21:22:28Z

FE Regression Coverage Report

Increment line coverage 83.78% (93/111) 🎉
Increment coverage report
Complete coverage report

feiniaofeiafei · 2026-05-12T02:12:07Z

run p0

foxtail463 · 2026-05-12T06:14:13Z

I agree with the core idea: partition-prunable predicates should not be removed before MV rewrite, otherwise MV matching may see an incomplete query predicate set and produce an unsafe rewrite.

One concern is where the deferred-removal state is stored. The current implementation records PartitionPrunablePredicate in StatementContext and later matches it back to a PhysicalOlapScan by TableIdentifier, selected partition ids, and slot-name remapping. This works for the current case, but the proof is actually scan-local:

for this scan, under this selected partition set, these conjuncts are guaranteed to be true.

Putting that proof in StatementContext makes the lifetime and ownership less explicit, especially when the same table appears multiple times, scans are rebuilt, or selected partitions are changed by later rewrites.

A cleaner structure may be to attach this information to the scan itself, e.g. PartitionPruneInfo on LogicalOlapScan / PhysicalOlapScan:

classPartitionPruneInfo {
Set<Long> provedPartitionIds;
List<Slot> partitionSlots;
Set<Expression> prunableConjuncts;
}

Then the post processor can simply inspect scan.getPartitionPruneInfo() and remove a conjunct only when:

partitionPruneInfo.getProvedPartitionIds().containsAll(scan.getSelectedPartitionIds())

This keeps the invariant local to the scan and avoids using StatementContext as a side table for plan-node-specific proof.

…nstead of StatementContext ### What problem does this PR solve? Problem Summary: Move PartitionPrunablePredicate info from StatementContext (keyed by TableIdentifier) onto LogicalOlapScan / PhysicalOlapScan so it follows the scan through with*/deepCopy and is included in equals. Also propagate the info onto the MV scan rewritten by SyncMaterializationContext#getScanPlan so the post-processor can still strip the deferred conjuncts after sync MV rewrite. ### Release note None ### Check List (For Author) - Test: Regression test (prune_predicates_mv_test.groovy) - Behavior changed: No - Does this need documentation: No Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

hello-stephen · 2026-05-12T07:27:38Z

FE Regression Coverage Report

Increment line coverage 50.82% (93/183) 🎉
Increment coverage report
Complete coverage report

feiniaofeiafei · 2026-05-12T07:37:56Z

run buildall

One olap scan can carry at most one PartitionPrunablePredicate, so use Optional instead of Set. Simplifies merge in PruneOlapScanPartition and iteration in PrunePartitionPredicate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

feiniaofeiafei · 2026-05-12T07:40:51Z

run buildall

hello-stephen · 2026-05-12T08:23:07Z

TPC-H: Total hot run time: 29486 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4c371b5d965b0bd3c957c8196c6be17956b96535, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17646	3874	3870	3870
q2	q3	10723	861	613	613
q4	4657	448	355	355
q5	7458	1336	1145	1145
q6	178	163	136	136
q7	930	939	744	744
q8	9304	1409	1276	1276
q9	5641	5383	5354	5354
q10	6263	2059	1809	1809
q11	452	266	255	255
q12	626	409	286	286
q13	18085	3316	2699	2699
q14	295	282	259	259
q15	q16	872	874	790	790
q17	983	985	790	790
q18	6539	5640	5628	5628
q19	1180	1280	1032	1032
q20	493	406	272	272
q21	4479	2268	1871	1871
q22	436	351	302	302
Total cold run time: 97240 ms
Total hot run time: 29486 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4153	4071	4087	4071
q2	q3	4613	4763	4177	4177
q4	2101	2153	1380	1380
q5	4951	5015	5222	5015
q6	196	165	131	131
q7	2020	1796	1703	1703
q8	3621	3235	3309	3235
q9	8547	8523	8512	8512
q10	4461	4512	4250	4250
q11	614	426	416	416
q12	693	756	532	532
q13	3556	3560	2973	2973
q14	321	337	294	294
q15	q16	820	763	692	692
q17	1406	1314	1262	1262
q18	7979	7230	7068	7068
q19	1133	1164	1164	1164
q20	2224	2209	1927	1927
q21	6094	5427	4891	4891
q22	567	493	419	419
Total cold run time: 60070 ms
Total hot run time: 54112 ms

hello-stephen · 2026-05-12T08:34:02Z

TPC-DS: Total hot run time: 169360 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4c371b5d965b0bd3c957c8196c6be17956b96535, data reload: false
query5	4325	642	511	511
query6	326	221	219	219
query7	4235	565	332	332
query8	320	230	218	218
query9	8845	3975	3970	3970
query10	443	356	297	297
query11	5839	2553	2219	2219
query12	181	130	128	128
query13	1357	580	462	462
query14	5996	5358	4998	4998
query14_1	4349	4337	4356	4337
query15	205	202	183	183
query16	1004	457	435	435
query17	997	739	618	618
query18	2432	486	356	356
query19	240	201	159	159
query20	137	130	127	127
query21	211	137	120	120
query22	13577	13591	13307	13307
query23	17159	16380	16518	16380
query23_1	16328	16375	16267	16267
query24	7718	1827	1405	1405
query24_1	1415	1400	1441	1400
query25	667	537	515	515
query26	1457	320	186	186
query27	2691	592	346	346
query28	4454	1989	1975	1975
query29	1026	643	546	546
query30	301	230	200	200
query31	1102	1076	935	935
query32	90	75	75	75
query33	587	333	290	290
query34	1179	1123	644	644
query35	767	766	676	676
query36	1312	1349	1131	1131
query37	153	96	86	86
query38	3224	3160	3060	3060
query39	920	920	884	884
query39_1	853	867	861	861
query40	231	154	135	135
query41	62	62	60	60
query42	109	107	104	104
query43	314	320	275	275
query44 query45	218	204	189	189
query46	1037	1202	725	725
query47	2319	2321	2149	2149
query48	420	424	294	294
query49	632	526	432	432
query50	740	295	210	210
query51	4327	4403	4232	4232
query52	106	107	94	94
query53	248	271	203	203
query54	297	271	250	250
query55	91	87	83	83
query56	295	299	297	297
query57	1432	1399	1295	1295
query58	290	264	293	264
query59	1575	1587	1401	1401
query60	347	348	331	331
query61	201	151	151	151
query62	668	617	566	566
query63	239	210	199	199
query64	2459	830	694	694
query65 query66	1749	501	392	392
query67	30003	29285	29894	29285
query68 query69	461	334	298	298
query70	1024	1009	964	964
query71	302	274	275	274
query72	3028	2732	2296	2296
query73	835	763	431	431
query74	5086	4893	4754	4754
query75	2766	2658	2319	2319
query76	2302	1123	739	739
query77	411	418	343	343
query78	13104	12902	12242	12242
query79	1470	941	737	737
query80	664	603	484	484
query81	452	278	244	244
query82	1369	164	126	126
query83	359	284	249	249
query84	301	143	113	113
query85	869	595	536	536
query86	407	347	311	311
query87	3440	3357	3217	3217
query88	3550	2688	2658	2658
query89	453	389	337	337
query90	1882	182	183	182
query91	176	171	134	134
query92	81	78	73	73
query93	975	957	545	545
query94	544	350	300	300
query95	648	385	343	343
query96	1100	798	336	336
query97	2696	2679	2539	2539
query98	242	236	227	227
query99	1079	1126	1024	1024
Total cold run time: 253937 ms
Total hot run time: 169360 ms

hello-stephen · 2026-05-12T09:16:22Z

FE UT Coverage Report

Increment line coverage 51.20% (64/125) 🎉
Increment coverage report
Complete coverage report

feiniaofeiafei · 2026-05-12T10:21:36Z

run feut

hello-stephen · 2026-05-12T10:35:12Z

FE Regression Coverage Report

Increment line coverage 75.20% (94/125) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-05-12T11:48:02Z

FE UT Coverage Report

Increment line coverage 51.20% (64/125) 🎉
Increment coverage report
Complete coverage report

morrySnow · 2026-05-13T11:06:13Z

/review

feiniaofeiafei · 2026-05-14T06:43:17Z

run buildall

hello-stephen · 2026-05-14T07:00:27Z

TPC-H: Total hot run time: 29585 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2d5071c247addf7c1280f4c8401675858dedc9f7, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17651	3927	3875	3875
q2	q3	10730	876	617	617
q4	4660	462	341	341
q5	7450	1349	1129	1129
q6	185	165	136	136
q7	909	953	762	762
q8	9300	1408	1263	1263
q9	5632	5388	5332	5332
q10	6246	2089	1821	1821
q11	472	264	261	261
q12	637	415	289	289
q13	18104	3241	2746	2746
q14	311	286	267	267
q15	q16	898	883	799	799
q17	963	1007	669	669
q18	6433	5815	5597	5597
q19	1301	1222	1094	1094
q20	513	391	273	273
q21	4845	2433	1984	1984
q22	481	383	330	330
Total cold run time: 97721 ms
Total hot run time: 29585 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4709	4785	4853	4785
q2	q3	4685	4796	4189	4189
q4	2138	2220	1408	1408
q5	4980	5068	5298	5068
q6	196	162	137	137
q7	2060	1781	1621	1621
q8	3325	3118	3131	3118
q9	8433	8474	8485	8474
q10	4497	4502	4278	4278
q11	620	424	411	411
q12	751	775	563	563
q13	3273	3564	3010	3010
q14	315	309	273	273
q15	q16	922	805	703	703
q17	1336	1304	1252	1252
q18	8057	7268	7275	7268
q19	1166	1122	1163	1122
q20	2235	2204	1957	1957
q21	6163	5487	4932	4932
q22	538	505	415	415
Total cold run time: 60399 ms
Total hot run time: 54984 ms

hello-stephen · 2026-05-14T07:11:35Z

TPC-DS: Total hot run time: 171194 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2d5071c247addf7c1280f4c8401675858dedc9f7, data reload: false
query5	4327	653	511	511
query6	325	218	202	202
query7	4286	575	303	303
query8	328	230	226	226
query9	8826	3973	3998	3973
query10	450	329	289	289
query11	5761	2399	2209	2209
query12	184	132	126	126
query13	1265	585	424	424
query14	5906	5305	5029	5029
query14_1	4322	4327	4292	4292
query15	211	202	187	187
query16	1023	450	372	372
query17	1149	743	620	620
query18	2453	507	378	378
query19	214	201	158	158
query20	148	130	128	128
query21	224	145	124	124
query22	13632	13635	13383	13383
query23	17207	16356	16805	16356
query23_1	16465	16412	16405	16405
query24	7510	1790	1352	1352
query24_1	1342	1360	1357	1357
query25	548	491	440	440
query26	1330	316	165	165
query27	2693	599	318	318
query28	4429	1903	1898	1898
query29	1008	662	500	500
query30	299	234	199	199
query31	1101	1067	931	931
query32	89	70	68	68
query33	534	339	283	283
query34	1154	1131	634	634
query35	772	794	671	671
query36	1356	1341	1222	1222
query37	149	94	84	84
query38	3178	3180	3076	3076
query39	945	967	912	912
query39_1	856	877	890	877
query40	234	155	135	135
query41	63	60	58	58
query42	110	106	105	105
query43	317	321	278	278
query44 query45	208	202	190	190
query46	1084	1181	734	734
query47	2372	2344	2191	2191
query48	406	420	299	299
query49	623	523	429	429
query50	737	281	217	217
query51	4264	4282	4218	4218
query52	108	105	92	92
query53	263	272	195	195
query54	312	266	248	248
query55	93	89	84	84
query56	329	306	301	301
query57	1448	1408	1341	1341
query58	301	265	251	251
query59	1556	1602	1419	1419
query60	336	329	314	314
query61	164	161	162	161
query62	681	621	582	582
query63	258	199	205	199
query64	2399	824	725	725
query65 query66	1766	541	411	411
query67	30045	30028	29837	29837
query68 query69	475	345	315	315
query70	1043	1028	1012	1012
query71	301	270	275	270
query72	3127	2872	2527	2527
query73	815	773	409	409
query74	5056	4950	4719	4719
query75	2758	2649	2323	2323
query76	2297	1119	766	766
query77	422	419	355	355
query78	12908	12992	12407	12407
query79	1528	1043	738	738
query80	1347	571	478	478
query81	522	283	240	240
query82	1189	160	127	127
query83	339	280	245	245
query84	264	138	109	109
query85	900	520	447	447
query86	456	331	317	317
query87	3439	3363	3234	3234
query88	3498	2640	2580	2580
query89	435	385	344	344
query90	1908	177	176	176
query91	183	167	139	139
query92	81	72	71	71
query93	1020	956	559	559
query94	728	335	312	312
query95	664	377	447	377
query96	999	746	343	343
query97	2708	2696	2593	2593
query98	242	235	226	226
query99	1126	1118	991	991
Total cold run time: 254103 ms
Total hot run time: 171194 ms

github-actions · 2026-05-14T07:18:07Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-05-14T07:18:10Z

PR approved by anyone and no changes requested.

hello-stephen · 2026-05-14T08:00:03Z

FE UT Coverage Report

Increment line coverage 51.20% (64/125) 🎉
Increment coverage report
Complete coverage report

feiniaofeiafei · 2026-05-14T08:59:17Z

/review

hello-stephen · 2026-05-14T09:18:11Z

FE Regression Coverage Report

Increment line coverage 8.65% (94/1087) 🎉
Increment coverage report
Complete coverage report

924060929 · 2026-05-14T10:09:22Z

/review

github-actions

Code review completed. I did not find a critical/blocking issue to request changes.

Critical checkpoint conclusions:

Goal and tests: The PR defers removal of partition-prunable predicates until after MV rewrite and carries the necessary metadata through logical/physical OLAP scans. The added regression suite covers async MV, partitioned MV, and sync MV alias cases; unit coverage for PartitionPruner.pruneWithResult remains focused on predicate-result metadata.
Scope and focus: The implementation is reasonably focused on partition predicate pruning/MV rewrite interaction. The existing package-location review thread was already addressed by moving PartitionPrunablePredicate to trees.plans, so I did not repeat it.
Concurrency/lifecycle: The changes are optimizer plan metadata and post-processing only; I did not identify new shared mutable state, lock ordering, or lifecycle hazards.
Configuration/compatibility: No new configs, storage formats, Thrift/protocol fields, or rolling-upgrade compatibility concerns were introduced.
Parallel paths: OLAP scan logical-to-physical propagation and sync MV scan propagation are covered. External file scan partition pruning is not part of this behavior and remains separate.
Conditional checks: skipPrunePredicate and delete-statement exclusions are preserved both when recording and when removing deferred predicates. The post-processor also verifies that recorded selected partitions still cover the final scan partitions before removing predicates.
Test coverage/results: Regression output is present and deterministic via order_qt; no additional user-provided focus points were supplied. I did not run the tests in this review runner.
Observability: No new observability requirement found; this is an optimizer rewrite/post-process change.
Transaction/persistence/data writes: Not applicable.
FE/BE variable passing: Not applicable.
Performance: The added post-processor work is limited to physical filters directly over OLAP scans with recorded metadata; no obvious hot-path anti-pattern was found.

User focus: .opencode-review.yiiWoK/review_focus.txt says no additional focus was provided; no extra focus-specific issue was found.

… are alwaystrue after partition pruning into the PlanPostProcessor #63111 (#63464) picked from #63111 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… are alwaystrue after partition pruning into the PlanPostProcessor #63111 (#63467) picked from #63111 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… true after partition pruning into the PlanPostProcessor (apache#63111) ### What problem does this PR solve? Related PR: apache#57169 Problem Summary: After partition pruning, predicates that are evaluated to constant true are removed as an optimization (introduced by apache#57169). However, this introduced a bug in materialized view rewriting: when such predicates are removed, they are not compensated above the rewritten materialized view. Since the materialized view itself is not partitioned, this leads to incorrect query results. This PR fixes the issue by moving the removal of constant-true predicates to the planPostProcessors phase. This ensures that the predicate removal does not interfere with materialized view rewriting, preserving correctness while still retaining the optimization benefit. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix
990efa5

fix
b2a3401

fix
7836b85

fix
c4285dd

feiniaofeiafei force-pushed the fix/cir-20138-mv-partition-predicate-compensate branch from d817a84 to 4c371b5Compare May 12, 2026 07:40

924060929 approved these changes May 14, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label May 14, 2026

github-actionsBot added the reviewed label May 14, 2026

github-actionsBot reviewed May 14, 2026
View reviewed changes

morrySnow merged commit fb27aeb into apache:masterMay 14, 2026
31 checks passed

morrySnow added dev/4.0.x dev/4.1.x labels May 14, 2026

github-actionsBot added dev/4.0.x-conflict dev/4.1.x-conflict labels May 14, 2026

yiguolei added dev/4.1.2-merged and removed dev/4.1.x dev/4.1.x-conflict labels May 22, 2026

yiguolei added dev/4.0.6-merged and removed dev/4.0.x dev/4.0.x-conflict labels May 22, 2026

yiguolei mentioned this pull request Jun 14, 2026
4.1.2 Release Notes #64485
Closed

Uh oh!

Conversation

feiniaofeiafei commented May 9, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented May 9, 2026

Uh oh!

feiniaofeiafei commented May 9, 2026

Uh oh!

feiniaofeiafei commented May 9, 2026

Uh oh!

feiniaofeiafei commented May 9, 2026

Uh oh!

feiniaofeiafei commented May 11, 2026

Uh oh!

hello-stephen commented May 11, 2026

Uh oh!

hello-stephen commented May 11, 2026

Uh oh!

feiniaofeiafei commented May 11, 2026

Uh oh!

hello-stephen commented May 11, 2026

Uh oh!

hello-stephen commented May 11, 2026

Uh oh!

hello-stephen commented May 11, 2026

FE Regression Coverage Report

Uh oh!

feiniaofeiafei commented May 12, 2026

Uh oh!

foxtail463 commented May 12, 2026

Uh oh!

hello-stephen commented May 12, 2026

FE Regression Coverage Report

Uh oh!

feiniaofeiafei commented May 12, 2026

Uh oh!

feiniaofeiafei commented May 12, 2026

Uh oh!

hello-stephen commented May 12, 2026

Uh oh!

hello-stephen commented May 12, 2026

Uh oh!

hello-stephen commented May 12, 2026

FE UT Coverage Report

Uh oh!

feiniaofeiafei commented May 12, 2026

Uh oh!

hello-stephen commented May 12, 2026

FE Regression Coverage Report

Uh oh!

hello-stephen commented May 12, 2026

FE UT Coverage Report

Uh oh!

morrySnow commented May 13, 2026

Uh oh!

feiniaofeiafei commented May 14, 2026

Uh oh!

hello-stephen commented May 14, 2026

Uh oh!

hello-stephen commented May 14, 2026

Uh oh!

github-actionsBot commented May 14, 2026

Uh oh!

github-actionsBot commented May 14, 2026

Uh oh!

hello-stephen commented May 14, 2026

FE UT Coverage Report

Uh oh!

feiniaofeiafei commented May 14, 2026

Uh oh!

hello-stephen commented May 14, 2026

FE Regression Coverage Report

Uh oh!

924060929 commented May 14, 2026

Uh oh!

feiniaofeiafei commented May 9, 2026•
edited
Loading