[fix](mtmv) Fix partition trace fail when create partition mv with view by seawinde · Pull Request #57958 · apache/doris

seawinde · 2025-11-12T11:27:32Z

What problem does this PR solve?

Problem Summary:

Fix partition trace fail when create partition mv with view and date_trunc
Fix err if use data_trunc('day', col) partition column when create partition materialized view

the mv def sql would be success after this fix

CREATEVIEWlineitem_daily_summary_viewASSELECT DATE_TRUNC('day', L_SHIPDATE) AS ship_date,
L_RETURNFLAG,
L_LINESTATUS,
COUNT(*) AS order_count,
SUM(L_QUANTITY) AS total_quantity,
SUM(L_EXTENDEDPRICE) AS total_price,
AVG(L_DISCOUNT) AS avg_discount
FROM lineitem
WHERE L_SHIPDATE IS NOT NULLGROUP BY ship_date, L_RETURNFLAG, L_LINESTATUS;

mv def is as following

SELECT
ship_date,
L_RETURNFLAG,
SUM(order_count) AS total_orders,
SUM(total_quantity) AS sum_quantity,
SUM(total_price) AS sum_price,
AVG(avg_discount) AS average_discount
FROM lineitem_daily_summary_view
GROUP BY ship_date, L_RETURNFLAG
ORDER BY ship_date, L_RETURNFLAG, total_orders, sum_quantity, sum_price;

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-11-12T11:27:36Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

seawinde · 2025-11-12T11:30:14Z

run buildall

doris-robot · 2025-11-12T11:50:13Z

TPC-H: Total hot run time: 34270 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 854cccd3d1f5ef5b090e4f83a7276df3f0fde30d, data reload: false
------ Round 1 ----------------------------------
q1	17627	5279	5103	5103
q2	2109	309	202	202
q3	10208	1297	748	748
q4	10218	872	367	367
q5	7530	2418	2302	2302
q6	186	170	135	135
q7	939	783	639	639
q8	9351	1356	1098	1098
q9	6968	5162	5157	5157
q10	6826	2223	1818	1818
q11	484	300	276	276
q12	341	366	230	230
q13	17774	3645	2997	2997
q14	243	255	217	217
q15	576	510	510	510
q16	1036	995	933	933
q17	588	869	364	364
q18	7480	7200	7112	7112
q19	1223	947	558	558
q20	349	342	235	235
q21	3789	2565	2272	2272
q22	1062	1065	997	997
Total cold run time: 106907 ms
Total hot run time: 34270 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5209	5151	5110	5110
q2	249	334	251	251
q3	2175	2666	2277	2277
q4	1359	1755	1317	1317
q5	4166	4357	4660	4357
q6	216	171	152	152
q7	2033	2007	1892	1892
q8	2673	2611	2563	2563
q9	7385	7338	7368	7338
q10	3077	3255	2811	2811
q11	606	537	509	509
q12	722	787	623	623
q13	3516	3983	3568	3568
q14	289	289	280	280
q15	544	509	505	505
q16	1033	1120	1111	1111
q17	1273	1652	1370	1370
q18	7901	7582	7673	7582
q19	846	856	978	856
q20	2004	2084	1913	1913
q21	5067	4487	4318	4318
q22	1102	1048	1015	1015
Total cold run time: 53445 ms
Total hot run time: 51718 ms

doris-robot · 2025-11-12T12:01:57Z

TPC-DS: Total hot run time: 187562 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 854cccd3d1f5ef5b090e4f83a7276df3f0fde30d, data reload: false
query1	1050	431	398	398
query2	6563	1690	1739	1690
query3	6763	219	219	219
query4	26258	23732	23388	23388
query5	4953	607	463	463
query6	339	235	220	220
query7	4664	494	298	298
query8	316	253	240	240
query9	8724	2580	2612	2580
query10	494	350	294	294
query11	15523	15143	14845	14845
query12	165	117	111	111
query13	1673	586	441	441
query14	10901	9250	9151	9151
query15	203	194	174	174
query16	7742	701	519	519
query17	1293	823	613	613
query18	2016	412	319	319
query19	208	204	169	169
query20	128	130	121	121
query21	216	133	113	113
query22	4028	4057	4023	4023
query23	34096	32907	33112	32907
query24	8563	2415	2443	2415
query25	592	503	449	449
query26	1235	276	160	160
query27	2735	499	350	350
query28	4354	2251	2193	2193
query29	817	596	480	480
query30	298	224	195	195
query31	886	826	730	730
query32	87	74	75	74
query33	625	376	321	321
query34	789	851	519	519
query35	815	868	761	761
query36	963	1006	883	883
query37	124	113	89	89
query38	3579	3531	3508	3508
query39	1472	1410	1405	1405
query40	219	136	119	119
query41	68	62	61	61
query42	127	117	115	115
query43	493	492	451	451
query44	1231	743	746	743
query45	186	180	177	177
query46	884	986	640	640
query47	1765	1776	1707	1707
query48	406	428	318	318
query49	797	539	426	426
query50	633	693	405	405
query51	3930	3930	3865	3865
query52	113	117	107	107
query53	236	268	192	192
query54	310	300	295	295
query55	88	89	83	83
query56	334	305	318	305
query57	1178	1207	1137	1137
query58	289	282	279	279
query59	2580	2632	2619	2619
query60	360	341	323	323
query61	165	158	160	158
query62	834	721	690	690
query63	230	189	187	187
query64	4590	1286	974	974
query65	4029	3923	3958	3923
query66	1126	449	358	358
query67	15283	15193	14781	14781
query68	8481	926	603	603
query69	491	326	314	314
query70	1350	1332	1313	1313
query71	494	342	322	322
query72	6257	4863	5100	4863
query73	664	582	362	362
query74	8860	9180	8636	8636
query75	3560	3313	2823	2823
query76	3458	1150	730	730
query77	743	400	314	314
query78	9667	9731	8838	8838
query79	2692	836	592	592
query80	701	561	527	527
query81	508	257	233	233
query82	684	161	128	128
query83	276	267	253	253
query84	260	112	94	94
query85	925	488	447	447
query86	380	301	285	285
query87	3707	3715	3653	3653
query88	3822	2294	2263	2263
query89	394	323	305	305
query90	2041	221	227	221
query91	183	164	134	134
query92	88	64	65	64
query93	2505	966	634	634
query94	702	451	338	338
query95	405	328	316	316
query96	496	570	279	279
query97	2928	2959	2876	2876
query98	242	227	216	216
query99	1350	1411	1320	1320
Total cold run time: 278048 ms
Total hot run time: 187562 ms

doris-robot · 2025-11-12T12:07:04Z

ClickBench: Total hot run time: 27.53 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 854cccd3d1f5ef5b090e4f83a7276df3f0fde30d, data reload: false
query1	0.05	0.05	0.05
query2	0.10	0.05	0.06
query3	0.26	0.09	0.08
query4	1.60	0.11	0.12
query5	0.27	0.25	0.26
query6	1.18	0.64	0.64
query7	0.04	0.03	0.02
query8	0.05	0.04	0.05
query9	0.58	0.52	0.51
query10	0.57	0.57	0.57
query11	0.17	0.11	0.11
query12	0.16	0.12	0.12
query13	0.62	0.61	0.60
query14	1.01	0.98	0.98
query15	0.86	0.83	0.83
query16	0.39	0.38	0.40
query17	1.06	1.02	1.03
query18	0.22	0.19	0.19
query19	1.92	1.82	1.80
query20	0.02	0.01	0.01
query21	15.47	0.17	0.14
query22	5.19	0.08	0.05
query23	15.64	0.26	0.11
query24	2.55	0.54	0.81
query25	0.08	0.07	0.06
query26	0.15	0.13	0.13
query27	0.08	0.05	0.06
query28	4.93	1.14	0.96
query29	12.54	3.96	3.25
query30	0.29	0.13	0.12
query31	2.82	0.59	0.38
query32	3.23	0.56	0.47
query33	3.08	3.02	3.12
query34	15.97	5.17	4.49
query35	4.54	4.56	4.65
query36	0.68	0.50	0.48
query37	0.10	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.91 s
Total hot run time: 27.53 s

hello-stephen · 2025-11-12T14:10:08Z

FE Regression Coverage Report

Increment line coverage 81.82% (9/11) 🎉
Increment coverage report
Complete coverage report

seawinde · 2025-11-13T03:02:19Z

run buildall

doris-robot · 2025-11-13T03:27:05Z

TPC-H: Total hot run time: 34327 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 85366fb5b9e6da92a66540cc48615a2e392bd435, data reload: false
------ Round 1 ----------------------------------
q1	17606	5132	5041	5041
q2	2058	313	215	215
q3	10206	1289	719	719
q4	10261	969	385	385
q5	7470	2397	2359	2359
q6	178	167	136	136
q7	932	769	631	631
q8	9363	1331	1071	1071
q9	6980	5143	5327	5143
q10	6835	2232	1798	1798
q11	516	291	278	278
q12	338	371	241	241
q13	17760	3656	3054	3054
q14	236	237	215	215
q15	577	538	510	510
q16	997	999	951	951
q17	578	879	370	370
q18	7517	7300	7146	7146
q19	1238	953	571	571
q20	350	359	241	241
q21	3878	3219	2295	2295
q22	1064	1028	957	957
Total cold run time: 106938 ms
Total hot run time: 34327 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5154	5121	5116	5116
q2	255	321	229	229
q3	2167	2674	2316	2316
q4	1364	1817	1330	1330
q5	4244	4510	4655	4510
q6	208	172	131	131
q7	2035	2031	1851	1851
q8	2640	2581	2578	2578
q9	7364	7297	7220	7220
q10	3070	3292	2860	2860
q11	626	532	520	520
q12	706	815	660	660
q13	3560	3961	3307	3307
q14	277	297	278	278
q15	549	501	508	501
q16	1087	1093	1138	1093
q17	1187	1549	1431	1431
q18	7938	7719	7588	7588
q19	805	905	1019	905
q20	2011	2051	1919	1919
q21	5034	4427	4371	4371
q22	1087	1081	990	990
Total cold run time: 53368 ms
Total hot run time: 51704 ms

doris-robot · 2025-11-13T03:38:45Z

TPC-DS: Total hot run time: 187354 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 85366fb5b9e6da92a66540cc48615a2e392bd435, data reload: false
query1	1042	394	396	394
query2	6596	1723	1692	1692
query3	6764	230	225	225
query4	26201	23664	23041	23041
query5	4402	635	518	518
query6	355	229	212	212
query7	4640	498	295	295
query8	295	285	250	250
query9	8680	2598	2620	2598
query10	515	333	280	280
query11	15683	15040	14883	14883
query12	178	117	114	114
query13	1680	557	432	432
query14	10471	9146	9170	9146
query15	203	192	174	174
query16	7179	707	549	549
query17	1267	785	625	625
query18	2003	446	336	336
query19	217	213	183	183
query20	130	133	122	122
query21	218	137	115	115
query22	4012	4161	4024	4024
query23	33976	33026	32773	32773
query24	8508	2433	2443	2433
query25	642	574	475	475
query26	1251	281	167	167
query27	2746	505	355	355
query28	4375	2237	2203	2203
query29	854	638	529	529
query30	303	229	200	200
query31	925	813	730	730
query32	87	79	75	75
query33	620	401	346	346
query34	810	851	525	525
query35	855	832	737	737
query36	950	983	908	908
query37	122	124	90	90
query38	3534	3519	3484	3484
query39	1465	1408	1424	1408
query40	230	133	122	122
query41	69	65	66	65
query42	129	115	112	112
query43	494	489	481	481
query44	1253	753	753	753
query45	189	185	178	178
query46	887	995	633	633
query47	1800	1833	1741	1741
query48	397	426	321	321
query49	845	559	421	421
query50	639	672	408	408
query51	3854	3945	3880	3880
query52	113	111	102	102
query53	236	262	198	198
query54	310	289	281	281
query55	95	86	81	81
query56	320	333	329	329
query57	1200	1213	1106	1106
query58	288	275	280	275
query59	2579	2721	2555	2555
query60	351	362	327	327
query61	164	163	159	159
query62	826	720	656	656
query63	234	195	192	192
query64	4583	1208	864	864
query65	4014	3980	3957	3957
query66	1178	443	331	331
query67	15221	15238	14927	14927
query68	8462	928	600	600
query69	495	328	292	292
query70	1373	1254	1298	1254
query71	494	338	333	333
query72	5879	4898	4939	4898
query73	703	564	357	357
query74	9203	9112	8811	8811
query75	3982	3324	2834	2834
query76	3774	1162	743	743
query77	814	413	324	324
query78	9479	9688	8770	8770
query79	2508	844	594	594
query80	685	563	499	499
query81	504	263	230	230
query82	469	162	133	133
query83	300	257	253	253
query84	307	120	98	98
query85	923	474	433	433
query86	342	336	308	308
query87	3749	3706	3601	3601
query88	2862	2253	2225	2225
query89	423	333	296	296
query90	1950	222	215	215
query91	171	168	138	138
query92	88	65	63	63
query93	1566	973	645	645
query94	727	448	355	355
query95	403	325	301	301
query96	484	563	276	276
query97	2938	2970	2880	2880
query98	231	217	208	208
query99	1424	1429	1286	1286
Total cold run time: 274964 ms
Total hot run time: 187354 ms

doris-robot · 2025-11-13T03:43:53Z

ClickBench: Total hot run time: 27.97 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 85366fb5b9e6da92a66540cc48615a2e392bd435, data reload: false
query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.25	0.08	0.09
query4	1.61	0.12	0.11
query5	0.26	0.25	0.27
query6	1.17	0.66	0.65
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.59	0.53	0.52
query10	0.57	0.58	0.57
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.62	0.60	0.61
query14	1.02	1.00	0.99
query15	0.84	0.83	0.82
query16	0.40	0.40	0.40
query17	1.01	1.05	1.05
query18	0.21	0.20	0.20
query19	1.94	1.84	1.90
query20	0.02	0.01	0.01
query21	15.45	0.18	0.13
query22	5.13	0.07	0.05
query23	15.70	0.25	0.11
query24	3.08	0.91	0.90
query25	0.07	0.07	0.08
query26	0.15	0.14	0.13
query27	0.07	0.06	0.05
query28	5.55	1.12	0.91
query29	12.56	3.84	3.20
query30	0.29	0.15	0.12
query31	2.82	0.59	0.38
query32	3.23	0.54	0.46
query33	3.04	3.03	3.12
query34	15.90	5.19	4.57
query35	4.56	4.58	4.55
query36	0.68	0.50	0.49
query37	0.10	0.06	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.15	0.13
query41	0.08	0.04	0.03
query42	0.05	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 99.88 s
Total hot run time: 27.97 s

hello-stephen · 2025-11-13T04:15:04Z

FE UT Coverage Report

Increment line coverage 27.27% (3/11) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-11-13T05:31:43Z

FE Regression Coverage Report

Increment line coverage 72.73% (8/11) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-11-13T10:28:21Z

FE Regression Coverage Report

Increment line coverage 72.73% (8/11) 🎉
Increment coverage report
Complete coverage report

github-actions · 2025-11-17T06:45:05Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-11-17T06:45:08Z

PR approved by anyone and no changes requested.

…ew (apache#57958) ### What problem does this PR solve? Related PR: apache#56423 apache#56958 Problem Summary: 1. Fix partition trace fail when create partition mv with view and date_trunc 2. Fix err if use data_trunc('day', col) partition column when create partition materialized view the mv def sql would be success after this fix CREATE VIEW lineitem_daily_summary_view AS SELECT DATE_TRUNC('day', L_SHIPDATE) AS ship_date, L_RETURNFLAG, L_LINESTATUS, COUNT(*) AS order_count, SUM(L_QUANTITY) AS total_quantity, SUM(L_EXTENDEDPRICE) AS total_price, AVG(L_DISCOUNT) AS avg_discount FROM lineitem WHERE L_SHIPDATE IS NOT NULL GROUP BY ship_date, L_RETURNFLAG, L_LINESTATUS; mv def is as following SELECT ship_date, L_RETURNFLAG, SUM(order_count) AS total_orders, SUM(total_quantity) AS sum_quantity, SUM(total_price) AS sum_price, AVG(avg_discount) AS average_discount FROM lineitem_daily_summary_view GROUP BY ship_date, L_RETURNFLAG ORDER BY ship_date, L_RETURNFLAG, total_orders, sum_quantity, sum_price;

pick: #56958 #57713 #57759 #57681 #57958 --------- Co-authored-by: seawinde <daydayup005@yeah.net> Co-authored-by: seawinde <wusi@selectdb.com>

…ew (apache#57958) ### What problem does this PR solve? Related PR: apache#56423 apache#56958 Problem Summary: 1. Fix partition trace fail when create partition mv with view and date_trunc 2. Fix err if use data_trunc('day', col) partition column when create partition materialized view the mv def sql would be success after this fix CREATE VIEW lineitem_daily_summary_view AS SELECT DATE_TRUNC('day', L_SHIPDATE) AS ship_date, L_RETURNFLAG, L_LINESTATUS, COUNT(*) AS order_count, SUM(L_QUANTITY) AS total_quantity, SUM(L_EXTENDEDPRICE) AS total_price, AVG(L_DISCOUNT) AS avg_discount FROM lineitem WHERE L_SHIPDATE IS NOT NULL GROUP BY ship_date, L_RETURNFLAG, L_LINESTATUS; mv def is as following SELECT ship_date, L_RETURNFLAG, SUM(order_count) AS total_orders, SUM(total_quantity) AS sum_quantity, SUM(total_price) AS sum_price, AVG(avg_discount) AS average_discount FROM lineitem_daily_summary_view GROUP BY ship_date, L_RETURNFLAG ORDER BY ship_date, L_RETURNFLAG, total_orders, sum_quantity, sum_price;

seawinde added 3 commits November 13, 2025 10:50

[fix](mtmv) Fix partition trace fail when create partition mv with view
3ed394e

fix data_trunc('day', slot)
472f2a1

add ut
2d007b8

seawinde force-pushed the fix_partition_trace_fail_contain_view branch from 451281d to 2d007b8Compare November 13, 2025 02:51

add ut
85366fb

seawinde changed the title ~~[draft](mtmv) Fix partition trace fail when create partition mv with view~~[fix](mtmv) Fix partition trace fail when create partition mv with viewNov 13, 2025

morrySnow approved these changes Nov 17, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 17, 2025

github-actionsBot added the reviewed label Nov 17, 2025

zddr approved these changes Nov 17, 2025
View reviewed changes

morrySnow merged commit ca1346d into apache:masterNov 17, 2025
32 checks passed

zddr mentioned this pull request Nov 19, 2025
branch-4.0:[feature](mtmv)MTMV refresh support multi pct tables #58140
Merged

yiguolei pushed a commit that referenced this pull request Nov 20, 2025
branch-4.0:[feature](mtmv)MTMV refresh support multi pct tables (#58140)
1d63c19
pick: #56958 #57713 #57759 #57681 #57958 --------- Co-authored-by: seawinde <daydayup005@yeah.net> Co-authored-by: seawinde <wusi@selectdb.com>

yiguolei added the dev/4.0.2-merged label Nov 20, 2025

Uh oh!

Conversation

seawinde commented Nov 12, 2025• edited by morrySnow Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Nov 12, 2025

Uh oh!

seawinde commented Nov 12, 2025

Uh oh!

doris-robot commented Nov 12, 2025

Uh oh!

doris-robot commented Nov 12, 2025

Uh oh!

doris-robot commented Nov 12, 2025

Uh oh!

hello-stephen commented Nov 12, 2025

FE Regression Coverage Report

Uh oh!

seawinde commented Nov 13, 2025

Uh oh!

doris-robot commented Nov 13, 2025

Uh oh!

doris-robot commented Nov 13, 2025

Uh oh!

doris-robot commented Nov 13, 2025

Uh oh!

hello-stephen commented Nov 13, 2025

FE UT Coverage Report

Uh oh!

hello-stephen commented Nov 13, 2025

FE Regression Coverage Report

Uh oh!

hello-stephen commented Nov 13, 2025

FE Regression Coverage Report

Uh oh!

github-actionsBot commented Nov 17, 2025

Uh oh!

github-actionsBot commented Nov 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

seawinde commented Nov 12, 2025•
edited by morrySnow
Loading