[feature](mtmv) Support querying rewrite by materialized view when DML such as insert and insert overwrite by seawinde · Pull Request #37561 · apache/doris

seawinde · 2024-07-09T12:38:51Z

Proposed changes

Support querying rewrite by materialized view when DML such as insert and insert overwrite , into outfile etc.
this is default enable, you can disable it by set enable_dml_materialized_view_rewrite = false
and the enable_materialized_view_rewrite is only control DQL.

It would not rewrite by materialized view when query sql use external table.
If you want rewrite by materialized view when use external table in DML,
you should set enable_dml_materialized_view_rewrite_when_base_table_unawareness = true;
this is default false.

this pr is replaced by #38115

doris-robot · 2024-07-09T12:38:57Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

seawinde · 2024-07-09T12:39:01Z

run buildall

seawinde · 2024-07-09T13:27:18Z

run buildall

seawinde · 2024-07-10T02:35:54Z

run buildall

doris-robot · 2024-07-10T02:55:31Z

TPC-H: Total hot run time: 40869 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a703fb97db1ab2a3f83309aa97aa9a748dcf123c, data reload: false
------ Round 1 ----------------------------------
q1	17878	4490	4436	4436
q2	2374	190	190	190
q3	10457	1167	1170	1167
q4	10197	852	798	798
q5	7484	2682	2625	2625
q6	224	139	143	139
q7	951	629	621	621
q8	9232	2097	2099	2097
q9	8972	6527	6503	6503
q10	8919	3727	3743	3727
q11	455	244	247	244
q12	406	241	239	239
q13	17766	3031	3013	3013
q14	264	232	234	232
q15	538	484	498	484
q16	460	392	381	381
q17	982	678	606	606
q18	8155	7663	7422	7422
q19	1703	1389	1331	1331
q20	684	318	327	318
q21	4970	3946	4209	3946
q22	420	360	350	350
Total cold run time: 113491 ms
Total hot run time: 40869 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4324	4261	4287	4261
q2	367	266	252	252
q3	3033	2750	2791	2750
q4	1910	1593	1553	1553
q5	5305	5290	5308	5290
q6	226	134	137	134
q7	2124	1797	1783	1783
q8	3239	3364	3345	3345
q9	8362	8346	8336	8336
q10	3956	3669	3738	3669
q11	575	511	486	486
q12	800	619	597	597
q13	17470	3025	3022	3022
q14	303	273	276	273
q15	524	471	478	471
q16	481	414	418	414
q17	1811	1497	1494	1494
q18	7697	7609	7426	7426
q19	1704	1633	1571	1571
q20	1995	1817	1813	1813
q21	4978	4882	4828	4828
q22	672	542	561	542
Total cold run time: 71856 ms
Total hot run time: 54310 ms

doris-robot · 2024-07-10T03:07:13Z

TPC-DS: Total hot run time: 175187 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a703fb97db1ab2a3f83309aa97aa9a748dcf123c, data reload: false
query1	919	372	362	362
query2	6451	2492	2462	2462
query3	6677	203	221	203
query4	28183	17572	17190	17190
query5	4176	486	499	486
query6	263	174	168	168
query7	4591	307	305	305
query8	354	301	299	299
query9	8451	2403	2381	2381
query10	470	282	289	282
query11	11873	9955	10127	9955
query12	137	93	85	85
query13	1663	387	391	387
query14	10360	7827	7858	7827
query15	240	193	195	193
query16	7957	313	329	313
query17	1803	555	546	546
query18	2016	292	291	291
query19	200	161	158	158
query20	94	83	89	83
query21	215	135	128	128
query22	4345	4190	4229	4190
query23	34045	33230	33186	33186
query24	12035	2907	2855	2855
query25	686	397	403	397
query26	1783	160	160	160
query27	2889	278	285	278
query28	7750	2076	2069	2069
query29	1106	665	644	644
query30	288	151	152	151
query31	987	734	774	734
query32	101	56	59	56
query33	792	329	323	323
query34	915	492	502	492
query35	713	586	572	572
query36	1114	994	968	968
query37	295	86	83	83
query38	2870	2717	2744	2717
query39	881	817	819	817
query40	282	128	124	124
query41	56	53	53	53
query42	128	103	105	103
query43	625	568	560	560
query44	1187	744	738	738
query45	201	172	168	168
query46	1116	751	774	751
query47	1849	1774	1770	1770
query48	385	302	316	302
query49	1241	443	437	437
query50	791	413	413	413
query51	6803	6736	6667	6667
query52	111	93	100	93
query53	370	302	296	296
query54	1008	464	460	460
query55	80	77	78	77
query56	348	296	305	296
query57	1178	1047	1058	1047
query58	264	271	390	271
query59	3549	3386	3376	3376
query60	316	294	294	294
query61	97	99	95	95
query62	833	668	653	653
query63	339	317	291	291
query64	10474	2226	7486	2226
query65	3179	3159	3137	3137
query66	1376	333	337	333
query67	15722	14817	14845	14817
query68	8108	549	554	549
query69	778	480	396	396
query70	1265	1170	1099	1099
query71	525	285	284	284
query72	8764	5305	5482	5305
query73	792	328	327	327
query74	5942	5501	5575	5501
query75	5325	2683	2688	2683
query76	5037	967	947	947
query77	798	302	351	302
query78	9526	9085	8951	8951
query79	7405	514	530	514
query80	1007	479	484	479
query81	583	221	227	221
query82	760	136	138	136
query83	334	175	171	171
query84	278	91	90	90
query85	1401	311	305	305
query86	449	365	306	306
query87	3304	3096	3041	3041
query88	4420	2464	2435	2435
query89	515	382	382	382
query90	1987	190	188	188
query91	131	103	105	103
query92	63	49	51	49
query93	5577	517	518	517
query94	1239	212	208	208
query95	419	313	314	313
query96	625	274	275	274
query97	3189	3012	3038	3012
query98	231	199	199	199
query99	1567	1275	1256	1256
Total cold run time: 306940 ms
Total hot run time: 175187 ms

doris-robot · 2024-07-10T03:12:45Z

ClickBench: Total hot run time: 30.38 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a703fb97db1ab2a3f83309aa97aa9a748dcf123c, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.23	0.06	0.05
query4	1.69	0.07	0.08
query5	0.50	0.49	0.48
query6	1.16	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.54	0.55	0.53
query11	0.15	0.12	0.11
query12	0.16	0.12	0.12
query13	0.60	0.59	0.59
query14	0.77	0.77	0.79
query15	0.86	0.83	0.81
query16	0.37	0.37	0.35
query17	0.96	1.02	0.98
query18	0.23	0.22	0.21
query19	1.79	1.69	1.72
query20	0.02	0.01	0.01
query21	15.42	0.75	0.66
query22	4.12	8.35	1.61
query23	18.29	1.46	1.42
query24	2.17	0.23	0.23
query25	0.16	0.09	0.08
query26	0.31	0.21	0.20
query27	0.46	0.22	0.22
query28	13.30	1.01	1.00
query29	12.63	3.32	3.28
query30	0.25	0.06	0.06
query31	2.87	0.39	0.39
query32	3.28	0.47	0.46
query33	2.88	2.93	2.89
query34	17.03	4.35	4.39
query35	4.41	4.43	4.42
query36	0.65	0.46	0.47
query37	0.19	0.14	0.15
query38	0.16	0.14	0.14
query39	0.05	0.03	0.04
query40	0.15	0.13	0.12
query41	0.08	0.05	0.05
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.72 s
Total hot run time: 30.38 s

zfr9527

LGTM

github-actions · 2024-07-10T07:04:41Z

PR approved by anyone and no changes requested.

seawinde · 2024-07-10T08:55:36Z

run buildall

doris-robot · 2024-07-10T09:33:43Z

TPC-H: Total hot run time: 40186 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 46f975614d3b33c725ac7986741f7cc1b4e8a0ab, data reload: false
------ Round 1 ----------------------------------
q1	17637	4406	4318	4318
q2	2014	194	190	190
q3	10444	1201	1124	1124
q4	10187	839	723	723
q5	7528	2699	2600	2600
q6	228	140	139	139
q7	966	602	610	602
q8	9231	2086	2136	2086
q9	8984	6550	6585	6550
q10	9009	3729	3805	3729
q11	520	232	238	232
q12	466	240	231	231
q13	17758	2998	3007	2998
q14	276	227	227	227
q15	546	482	481	481
q16	521	394	384	384
q17	978	744	784	744
q18	8022	7509	7431	7431
q19	8332	1472	1541	1472
q20	672	324	323	323
q21	4940	3262	3943	3262
q22	415	340	347	340
Total cold run time: 119674 ms
Total hot run time: 40186 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4505	4348	4330	4330
q2	390	256	281	256
q3	3167	3010	2922	2922
q4	2020	1776	1733	1733
q5	5623	5524	5628	5524
q6	239	138	146	138
q7	2247	1867	1844	1844
q8	3388	3494	3487	3487
q9	8786	9003	8851	8851
q10	4134	3839	3942	3839
q11	635	555	535	535
q12	844	694	683	683
q13	16279	3306	3399	3306
q14	346	302	297	297
q15	556	520	509	509
q16	513	481	487	481
q17	1862	1588	1572	1572
q18	8948	8071	7848	7848
q19	1837	1600	1646	1600
q20	2888	1906	1839	1839
q21	7174	5056	4843	4843
q22	600	560	576	560
Total cold run time: 76981 ms
Total hot run time: 56997 ms

doris-robot · 2024-07-10T09:44:49Z

TPC-DS: Total hot run time: 172714 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 46f975614d3b33c725ac7986741f7cc1b4e8a0ab, data reload: false
query1	907	376	370	370
query2	6454	2508	2510	2508
query3	6634	210	215	210
query4	27886	17624	17188	17188
query5	3602	466	479	466
query6	291	201	175	175
query7	4581	298	280	280
query8	332	309	289	289
query9	8535	2361	2341	2341
query10	428	270	264	264
query11	12265	10029	10080	10029
query12	117	88	82	82
query13	1634	383	381	381
query14	10165	7555	6180	6180
query15	228	190	181	181
query16	7786	327	307	307
query17	1775	555	514	514
query18	1954	281	270	270
query19	188	151	147	147
query20	93	80	81	80
query21	206	127	127	127
query22	4404	4002	3952	3952
query23	33754	33930	33655	33655
query24	10966	2894	2794	2794
query25	648	418	406	406
query26	704	148	148	148
query27	2291	287	278	278
query28	6175	2108	2117	2108
query29	912	646	667	646
query30	253	152	155	152
query31	977	774	742	742
query32	105	54	54	54
query33	686	287	299	287
query34	877	490	502	490
query35	668	557	566	557
query36	1147	949	973	949
query37	146	89	81	81
query38	2968	2864	2803	2803
query39	928	826	809	809
query40	201	124	121	121
query41	54	53	57	53
query42	115	97	101	97
query43	608	547	537	537
query44	1134	731	747	731
query45	197	167	161	161
query46	1062	722	693	693
query47	1855	1795	1782	1782
query48	382	299	310	299
query49	849	405	413	405
query50	791	390	393	390
query51	6839	6729	6755	6729
query52	111	98	96	96
query53	363	285	290	285
query54	873	462	449	449
query55	77	76	73	73
query56	306	284	263	263
query57	1137	1046	1029	1029
query58	263	249	281	249
query59	3357	3323	3363	3323
query60	303	276	276	276
query61	102	117	119	117
query62	786	648	658	648
query63	329	292	301	292
query64	9255	2196	1623	1623
query65	3527	3132	3100	3100
query66	819	332	328	328
query67	15498	14983	14978	14978
query68	4531	532	542	532
query69	555	422	329	329
query70	1191	1127	1065	1065
query71	414	276	286	276
query72	6876	5235	5634	5235
query73	748	334	329	329
query74	5971	5494	5500	5494
query75	3384	2687	2731	2687
query76	2207	984	912	912
query77	419	288	297	288
query78	9491	8915	9172	8915
query79	2271	517	515	515
query80	2468	467	475	467
query81	601	217	218	217
query82	877	141	139	139
query83	311	170	179	170
query84	272	97	163	97
query85	1847	313	300	300
query86	380	320	319	319
query87	3285	3093	3109	3093
query88	3905	2443	2466	2443
query89	478	391	389	389
query90	1626	186	182	182
query91	131	107	103	103
query92	59	48	48	48
query93	2544	516	510	510
query94	764	209	209	209
query95	399	319	310	310
query96	600	271	274	271
query97	3200	3043	3016	3016
query98	225	208	198	198
query99	1570	1229	1248	1229
Total cold run time: 279463 ms
Total hot run time: 172714 ms

doris-robot · 2024-07-10T09:50:21Z

ClickBench: Total hot run time: 30.31 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 46f975614d3b33c725ac7986741f7cc1b4e8a0ab, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.51	0.49	0.50
query6	1.13	0.74	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.50	0.49
query10	0.56	0.55	0.54
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.60	0.58
query14	0.77	0.77	0.79
query15	0.86	0.83	0.81
query16	0.36	0.35	0.38
query17	1.02	0.97	0.98
query18	0.23	0.21	0.22
query19	1.78	1.75	1.75
query20	0.01	0.01	0.01
query21	15.39	0.76	0.68
query22	3.88	7.76	1.64
query23	18.30	1.39	1.29
query24	2.09	0.24	0.21
query25	0.16	0.08	0.08
query26	0.30	0.21	0.20
query27	0.46	0.23	0.22
query28	13.32	1.02	1.00
query29	12.64	3.30	3.32
query30	0.25	0.06	0.07
query31	2.85	0.38	0.39
query32	3.29	0.48	0.47
query33	2.86	2.91	2.86
query34	17.05	4.34	4.40
query35	4.43	4.38	4.40
query36	0.64	0.48	0.47
query37	0.18	0.15	0.15
query38	0.16	0.14	0.15
query39	0.04	0.03	0.03
query40	0.16	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.42 s
Total hot run time: 30.31 s

seawinde · 2024-07-10T12:08:31Z

run buildall

doris-robot · 2024-07-10T12:46:26Z

TPC-H: Total hot run time: 39985 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d0e868cdb1365bf918fedc78e5e2564758a49e29, data reload: false
------ Round 1 ----------------------------------
q1	17623	4328	4276	4276
q2	2014	191	184	184
q3	10458	1216	1066	1066
q4	10195	755	830	755
q5	7487	2704	2648	2648
q6	222	142	142	142
q7	973	594	618	594
q8	9225	2053	2097	2053
q9	8910	6527	6485	6485
q10	9021	3720	3732	3720
q11	458	234	242	234
q12	504	229	237	229
q13	18109	2987	2996	2987
q14	278	233	230	230
q15	525	490	492	490
q16	498	399	383	383
q17	963	675	731	675
q18	7922	7390	7461	7390
q19	3520	1517	1444	1444
q20	643	316	327	316
q21	5109	3357	4045	3357
q22	406	332	327	327
Total cold run time: 115063 ms
Total hot run time: 39985 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4393	4234	4277	4234
q2	374	270	272	270
q3	2975	2763	2882	2763
q4	1966	1783	1712	1712
q5	5606	5510	5509	5509
q6	234	136	135	135
q7	2177	1832	1888	1832
q8	3314	3423	3472	3423
q9	8674	8705	8800	8705
q10	4095	3937	3744	3744
q11	577	491	477	477
q12	785	656	633	633
q13	16051	3238	3176	3176
q14	302	289	291	289
q15	529	499	479	479
q16	487	422	419	419
q17	1801	1531	1531	1531
q18	8112	7928	7772	7772
q19	1777	1514	1625	1514
q20	2142	1940	1872	1872
q21	5230	5006	4957	4957
q22	647	544	549	544
Total cold run time: 72248 ms
Total hot run time: 55990 ms

doris-robot · 2024-07-10T12:57:31Z

TPC-DS: Total hot run time: 171118 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d0e868cdb1365bf918fedc78e5e2564758a49e29, data reload: false
query1	941	379	373	373
query2	6469	2354	2399	2354
query3	6633	209	219	209
query4	28401	17454	17249	17249
query5	3695	485	479	479
query6	256	172	185	172
query7	4590	289	286	286
query8	319	303	304	303
query9	8434	2406	2406	2406
query10	439	301	277	277
query11	12154	10144	10283	10144
query12	119	86	85	85
query13	1669	370	371	370
query14	10150	7943	6760	6760
query15	238	191	184	184
query16	7157	312	300	300
query17	1833	543	522	522
query18	1779	285	294	285
query19	191	145	164	145
query20	93	80	84	80
query21	217	126	125	125
query22	4409	4143	4035	4035
query23	34028	33474	33693	33474
query24	10589	2870	2848	2848
query25	599	376	403	376
query26	704	159	149	149
query27	2244	286	282	282
query28	6258	2145	2138	2138
query29	864	639	645	639
query30	254	152	159	152
query31	964	759	756	756
query32	94	53	55	53
query33	655	326	337	326
query34	911	500	489	489
query35	688	593	570	570
query36	1144	998	986	986
query37	150	86	82	82
query38	2969	2869	2827	2827
query39	892	828	813	813
query40	201	124	120	120
query41	54	49	54	49
query42	114	102	101	101
query43	594	532	533	532
query44	1077	765	738	738
query45	195	160	157	157
query46	1081	709	745	709
query47	1874	1777	1782	1777
query48	378	304	301	301
query49	833	408	409	408
query50	774	397	396	396
query51	6929	6881	6731	6731
query52	107	95	100	95
query53	375	288	290	288
query54	907	457	456	456
query55	79	76	77	76
query56	307	287	312	287
query57	1162	1066	1069	1066
query58	248	247	271	247
query59	3373	3096	3240	3096
query60	315	277	268	268
query61	98	95	94	94
query62	800	655	669	655
query63	316	292	288	288
query64	9123	2181	1679	1679
query65	3181	3095	3099	3095
query66	763	328	334	328
query67	15656	15038	14931	14931
query68	6166	549	538	538
query69	643	512	368	368
query70	1116	1092	1127	1092
query71	447	274	276	274
query72	7000	4038	2690	2690
query73	765	326	325	325
query74	5960	5584	5559	5559
query75	3670	2714	2670	2670
query76	3225	910	978	910
query77	615	308	312	308
query78	9509	9022	10345	9022
query79	2281	530	523	523
query80	2011	468	466	466
query81	602	217	222	217
query82	461	128	138	128
query83	269	166	170	166
query84	272	90	89	89
query85	1276	336	303	303
query86	461	305	315	305
query87	3295	3129	3202	3129
query88	3325	2472	2459	2459
query89	483	379	387	379
query90	1755	189	186	186
query91	135	105	104	104
query92	66	48	54	48
query93	1007	503	516	503
query94	1105	211	207	207
query95	408	320	317	317
query96	584	275	280	275
query97	3249	3005	3025	3005
query98	226	196	199	196
query99	1706	1286	1288	1286
Total cold run time: 279200 ms
Total hot run time: 171118 ms

doris-robot · 2024-07-10T13:03:03Z

ClickBench: Total hot run time: 30.77 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d0e868cdb1365bf918fedc78e5e2564758a49e29, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.67	0.09	0.10
query5	0.52	0.47	0.48
query6	1.12	0.74	0.74
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.56	0.49	0.50
query10	0.56	0.55	0.56
query11	0.15	0.12	0.11
query12	0.14	0.13	0.13
query13	0.59	0.59	0.58
query14	0.75	0.77	0.81
query15	0.85	0.82	0.82
query16	0.38	0.37	0.36
query17	0.96	0.97	0.98
query18	0.23	0.22	0.22
query19	1.89	1.77	1.81
query20	0.02	0.01	0.01
query21	15.40	0.77	0.66
query22	3.94	6.99	1.85
query23	18.27	1.39	1.28
query24	2.53	0.23	0.22
query25	0.15	0.08	0.09
query26	0.36	0.22	0.21
query27	0.56	0.23	0.23
query28	13.34	1.03	1.00
query29	12.60	3.39	3.38
query30	0.25	0.06	0.06
query31	2.87	0.40	0.40
query32	3.26	0.47	0.48
query33	2.87	2.93	2.91
query34	17.07	4.41	4.31
query35	4.41	4.44	4.43
query36	0.66	0.46	0.48
query37	0.19	0.16	0.14
query38	0.16	0.15	0.15
query39	0.05	0.03	0.04
query40	0.15	0.13	0.12
query41	0.10	0.04	0.05
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 110.1 s
Total hot run time: 30.77 s

seawinde · 2024-07-11T06:15:44Z

run buildall

doris-robot · 2024-07-11T06:54:13Z

TPC-H: Total hot run time: 39725 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 99fd8b5550202bf0a3acfcabcd8f26b9b2f1428e, data reload: false
------ Round 1 ----------------------------------
q1	17620	4366	4296	4296
q2	2006	190	191	190
q3	10453	1193	1074	1074
q4	10195	778	879	778
q5	7541	2662	2660	2660
q6	221	138	135	135
q7	965	609	596	596
q8	9216	2077	2058	2058
q9	8609	6570	6548	6548
q10	8879	3784	3734	3734
q11	450	235	235	235
q12	427	234	228	228
q13	18787	2980	3003	2980
q14	283	241	233	233
q15	524	469	483	469
q16	517	377	373	373
q17	955	674	646	646
q18	7938	7432	7300	7300
q19	6460	1385	1415	1385
q20	654	323	332	323
q21	4909	3158	3206	3158
q22	391	335	326	326
Total cold run time: 118000 ms
Total hot run time: 39725 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4400	4271	4289	4271
q2	375	265	267	265
q3	3032	2913	2917	2913
q4	2019	1662	1731	1662
q5	5648	5574	5406	5406
q6	219	134	129	129
q7	2252	1889	1842	1842
q8	3237	3409	3381	3381
q9	8760	8797	8821	8797
q10	4147	3756	3804	3756
q11	611	515	558	515
q12	831	683	656	656
q13	16689	3186	3181	3181
q14	323	317	302	302
q15	534	476	474	474
q16	496	416	467	416
q17	1787	1516	1486	1486
q18	8081	7783	7896	7783
q19	2879	1640	1626	1626
q20	2139	1892	1857	1857
q21	5071	4859	4722	4722
q22	663	564	572	564
Total cold run time: 74193 ms
Total hot run time: 56004 ms

doris-robot · 2024-07-11T07:05:20Z

TPC-DS: Total hot run time: 174677 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 99fd8b5550202bf0a3acfcabcd8f26b9b2f1428e, data reload: false
query1	899	384	372	372
query2	6447	2524	2439	2439
query3	6654	205	212	205
query4	28440	17648	17523	17523
query5	3579	477	471	471
query6	285	202	157	157
query7	4591	291	283	283
query8	321	291	305	291
query9	8681	2415	2385	2385
query10	428	299	266	266
query11	10664	10051	10093	10051
query12	133	82	82	82
query13	1644	362	360	360
query14	10076	7163	7688	7163
query15	236	184	186	184
query16	7695	320	307	307
query17	1806	547	516	516
query18	1902	285	270	270
query19	206	149	145	145
query20	87	82	82	82
query21	210	127	127	127
query22	4349	4002	3989	3989
query23	33986	33678	33579	33579
query24	11028	2984	2991	2984
query25	584	402	383	383
query26	709	153	145	145
query27	2354	276	272	272
query28	5937	2154	2143	2143
query29	892	629	669	629
query30	250	151	150	150
query31	954	777	770	770
query32	97	57	53	53
query33	719	300	298	298
query34	885	487	500	487
query35	676	598	570	570
query36	1152	969	1011	969
query37	137	89	94	89
query38	2940	2878	2813	2813
query39	924	833	855	833
query40	214	120	122	120
query41	52	49	54	49
query42	119	98	98	98
query43	639	577	554	554
query44	1142	730	762	730
query45	214	170	174	170
query46	1086	759	741	741
query47	1844	1799	1773	1773
query48	364	309	287	287
query49	820	404	420	404
query50	775	388	380	380
query51	6865	6815	6710	6710
query52	119	96	96	96
query53	359	285	285	285
query54	896	438	435	435
query55	78	77	74	74
query56	296	268	278	268
query57	1131	1053	1084	1053
query58	249	262	266	262
query59	3409	3186	3186	3186
query60	311	276	281	276
query61	98	143	98	98
query62	786	657	642	642
query63	314	282	282	282
query64	9119	2180	1646	1646
query65	3170	3089	3103	3089
query66	749	320	334	320
query67	15398	15035	15103	15035
query68	4520	542	528	528
query69	521	332	360	332
query70	1201	1105	1142	1105
query71	369	287	281	281
query72	7036	5890	5639	5639
query73	744	322	317	317
query74	5961	5578	5560	5560
query75	3403	2690	2704	2690
query76	2125	880	935	880
query77	440	295	306	295
query78	9560	9049	8986	8986
query79	2135	532	522	522
query80	2559	472	467	467
query81	578	224	226	224
query82	811	137	137	137
query83	280	173	169	169
query84	249	85	85	85
query85	1959	315	293	293
query86	474	328	335	328
query87	3349	3133	3183	3133
query88	3763	2370	2360	2360
query89	468	377	377	377
query90	1761	190	188	188
query91	129	102	106	102
query92	64	51	48	48
query93	2379	515	491	491
query94	1166	276	212	212
query95	407	311	309	309
query96	601	277	269	269
query97	3220	3009	3032	3009
query98	220	202	193	193
query99	1531	1253	1250	1250
Total cold run time: 278156 ms
Total hot run time: 174677 ms

doris-robot · 2024-07-11T07:10:52Z

ClickBench: Total hot run time: 30.71 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 99fd8b5550202bf0a3acfcabcd8f26b9b2f1428e, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.68	0.07	0.07
query5	0.50	0.51	0.47
query6	1.16	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.54	0.49	0.48
query10	0.55	0.53	0.54
query11	0.15	0.11	0.11
query12	0.14	0.12	0.12
query13	0.59	0.58	0.59
query14	0.79	0.77	0.82
query15	0.84	0.82	0.81
query16	0.36	0.38	0.37
query17	0.98	0.98	1.00
query18	0.22	0.21	0.22
query19	1.78	1.84	1.71
query20	0.01	0.01	0.01
query21	15.41	0.75	0.66
query22	4.32	6.93	2.12
query23	18.28	1.31	1.34
query24	2.20	0.23	0.22
query25	0.15	0.08	0.10
query26	0.31	0.21	0.21
query27	0.46	0.23	0.23
query28	13.23	1.02	0.99
query29	12.61	3.32	3.27
query30	0.26	0.06	0.05
query31	2.89	0.39	0.39
query32	3.25	0.48	0.47
query33	2.91	2.90	2.86
query34	17.08	4.29	4.37
query35	4.42	4.45	4.40
query36	0.66	0.50	0.48
query37	0.20	0.15	0.16
query38	0.16	0.16	0.15
query39	0.05	0.04	0.03
query40	0.16	0.12	0.13
query41	0.09	0.05	0.05
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.91 s
Total hot run time: 30.71 s

seawinde · 2024-07-12T03:59:44Z

run buildall

doris-robot · 2024-07-12T04:35:52Z

TPC-H: Total hot run time: 40142 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b6576447eba08cc9f193dd5703388b95bd645a04, data reload: false
------ Round 1 ----------------------------------
q1	17629	4285	4230	4230
q2	2023	195	186	186
q3	10478	1204	1103	1103
q4	10417	903	779	779
q5	7533	2698	2675	2675
q6	221	136	138	136
q7	963	600	606	600
q8	9232	2122	2072	2072
q9	8867	6659	6626	6626
q10	8855	3847	3795	3795
q11	461	232	238	232
q12	474	232	227	227
q13	19879	3006	3002	3002
q14	280	233	236	233
q15	531	485	484	484
q16	495	384	379	379
q17	968	690	694	690
q18	8087	7445	7428	7428
q19	5951	1474	1418	1418
q20	684	344	353	344
q21	5091	3146	3270	3146
q22	411	362	357	357
Total cold run time: 119530 ms
Total hot run time: 40142 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4536	4375	4336	4336
q2	370	278	268	268
q3	3288	3103	3079	3079
q4	2113	1862	1759	1759
q5	5734	5554	5562	5554
q6	231	142	135	135
q7	2232	1848	1865	1848
q8	3340	3434	3428	3428
q9	8772	8896	8791	8791
q10	4079	3748	3864	3748
q11	621	500	494	494
q12	814	648	634	634
q13	17227	3151	3182	3151
q14	331	282	297	282
q15	534	494	488	488
q16	483	442	443	442
q17	1808	1524	1540	1524
q18	8089	7887	8032	7887
q19	1828	1657	1523	1523
q20	2662	1859	1837	1837
q21	9842	4696	4884	4696
q22	656	564	549	549
Total cold run time: 79590 ms
Total hot run time: 56453 ms

doris-robot · 2024-07-12T04:47:03Z

TPC-DS: Total hot run time: 174149 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b6576447eba08cc9f193dd5703388b95bd645a04, data reload: false
query1	928	376	364	364
query2	6364	2349	2426	2349
query3	6638	213	221	213
query4	28495	17626	17556	17556
query5	3499	511	492	492
query6	268	189	158	158
query7	4591	294	293	293
query8	311	291	289	289
query9	8447	2445	2429	2429
query10	436	293	274	274
query11	10734	10302	10104	10104
query12	116	88	86	86
query13	1633	384	374	374
query14	10146	7344	6407	6407
query15	243	192	190	190
query16	7586	335	319	319
query17	1723	571	559	559
query18	1257	272	270	270
query19	197	152	144	144
query20	88	84	83	83
query21	209	125	122	122
query22	4220	4129	4035	4035
query23	34033	34195	33624	33624
query24	10705	2901	2922	2901
query25	618	386	400	386
query26	1104	158	153	153
query27	2383	270	281	270
query28	7568	2161	2134	2134
query29	873	652	622	622
query30	258	151	152	151
query31	992	778	752	752
query32	99	53	64	53
query33	736	339	329	329
query34	896	508	512	508
query35	700	587	609	587
query36	1138	1001	957	957
query37	139	84	79	79
query38	2942	2885	2775	2775
query39	860	813	809	809
query40	200	118	138	118
query41	51	52	51	51
query42	117	100	94	94
query43	596	545	548	545
query44	1186	718	727	718
query45	197	161	166	161
query46	1093	732	768	732
query47	1899	1784	1809	1784
query48	367	302	313	302
query49	824	404	406	404
query50	780	397	401	397
query51	6916	6811	6815	6811
query52	110	94	98	94
query53	374	293	291	291
query54	878	457	456	456
query55	77	73	75	73
query56	284	260	271	260
query57	1145	1055	1037	1037
query58	245	247	256	247
query59	3344	3238	3086	3086
query60	313	267	273	267
query61	130	90	96	90
query62	767	648	637	637
query63	321	297	286	286
query64	9370	2230	1658	1658
query65	3157	3097	3165	3097
query66	694	337	332	332
query67	15400	15036	14861	14861
query68	4553	549	568	549
query69	496	383	337	337
query70	1169	1050	1153	1050
query71	380	279	281	279
query72	6793	5565	5680	5565
query73	745	334	328	328
query74	6030	5524	5503	5503
query75	3383	2702	2688	2688
query76	2513	967	893	893
query77	427	306	308	306
query78	11292	10616	9290	9290
query79	2842	524	538	524
query80	911	471	476	471
query81	583	221	221	221
query82	590	135	129	129
query83	349	169	164	164
query84	282	85	88	85
query85	1324	302	302	302
query86	475	314	287	287
query87	3367	3150	3091	3091
query88	4076	2471	2452	2452
query89	470	393	394	393
query90	1784	190	191	190
query91	138	101	102	101
query92	60	50	50	50
query93	3166	526	515	515
query94	888	215	214	214
query95	403	317	325	317
query96	609	282	281	281
query97	3206	3040	3098	3040
query98	210	201	197	197
query99	1660	1248	1292	1248
Total cold run time: 280182 ms
Total hot run time: 174149 ms

doris-robot · 2024-07-12T04:52:35Z

ClickBench: Total hot run time: 30.7 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b6576447eba08cc9f193dd5703388b95bd645a04, data reload: false
query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.24	0.06	0.07
query4	1.64	0.08	0.07
query5	0.51	0.49	0.50
query6	1.15	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.05	0.04
query9	0.55	0.48	0.49
query10	0.55	0.54	0.55
query11	0.16	0.11	0.12
query12	0.15	0.13	0.12
query13	0.59	0.60	0.57
query14	0.76	0.77	0.77
query15	0.85	0.81	0.82
query16	0.36	0.37	0.38
query17	0.96	1.03	1.04
query18	0.22	0.21	0.22
query19	1.81	1.63	1.72
query20	0.01	0.01	0.01
query21	15.40	0.75	0.64
query22	4.46	6.84	2.14
query23	18.26	1.34	1.20
query24	2.20	0.23	0.22
query25	0.16	0.08	0.08
query26	0.30	0.21	0.21
query27	0.45	0.23	0.23
query28	13.24	1.01	1.00
query29	12.59	3.29	3.31
query30	0.24	0.06	0.05
query31	2.88	0.39	0.40
query32	3.26	0.49	0.49
query33	2.95	2.93	2.89
query34	17.12	4.30	4.31
query35	4.41	4.40	4.38
query36	0.65	0.50	0.49
query37	0.19	0.16	0.16
query38	0.15	0.16	0.14
query39	0.04	0.04	0.04
query40	0.16	0.14	0.12
query41	0.09	0.05	0.04
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.99 s
Total hot run time: 30.7 s

seawinde · 2024-07-16T02:17:22Z

run buildall

doris-robot · 2024-07-16T02:44:45Z

TPC-H: Total hot run time: 40096 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b6576447eba08cc9f193dd5703388b95bd645a04, data reload: false
------ Round 1 ----------------------------------
q1	17892	4386	4286	4286
q2	2042	193	191	191
q3	10728	1277	1172	1172
q4	10260	811	758	758
q5	9203	2740	2702	2702
q6	227	145	145	145
q7	982	623	613	613
q8	9580	2093	2126	2093
q9	9026	6638	6603	6603
q10	8822	3805	3794	3794
q11	464	240	254	240
q12	395	233	232	232
q13	17771	2989	3026	2989
q14	285	234	245	234
q15	517	486	494	486
q16	528	376	376	376
q17	955	632	670	632
q18	8219	7475	7332	7332
q19	6507	1426	1451	1426
q20	660	327	329	327
q21	4986	3891	3176	3176
q22	348	289	291	289
Total cold run time: 120397 ms
Total hot run time: 40096 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4393	4233	4272	4233
q2	368	266	259	259
q3	3176	2973	2933	2933
q4	2021	1718	1674	1674
q5	5450	5589	5602	5589
q6	231	147	148	147
q7	2207	1857	1876	1857
q8	3246	3346	3370	3346
q9	8772	8757	8678	8678
q10	4093	3899	3914	3899
q11	636	503	480	480
q12	810	617	647	617
q13	17743	3215	3221	3215
q14	312	279	303	279
q15	525	486	490	486
q16	507	453	443	443
q17	1817	1537	1522	1522
q18	8165	7833	7524	7524
q19	1701	1426	1623	1426
q20	2272	1838	1854	1838
q21	5075	5664	4846	4846
q22	585	506	508	506
Total cold run time: 74105 ms
Total hot run time: 55797 ms

seawinde · 2024-07-17T05:52:57Z

run buildall

github-actions · 2024-07-17T05:55:47Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-17T05:56:24Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-17T06:30:59Z

TPC-H: Total hot run time: 39702 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit caf5a24a854cf797a391b588a26f763ee65a870d, data reload: false
------ Round 1 ----------------------------------
q1	17606	4424	4278	4278
q2	2015	188	182	182
q3	10464	1195	1116	1116
q4	10209	761	848	761
q5	7551	2672	2647	2647
q6	216	132	133	132
q7	951	593	601	593
q8	9211	2088	2043	2043
q9	8936	6578	6544	6544
q10	8625	3746	3830	3746
q11	461	226	238	226
q12	412	217	222	217
q13	17907	2984	3024	2984
q14	275	234	236	234
q15	540	472	504	472
q16	497	387	373	373
q17	956	655	713	655
q18	7953	7465	7369	7369
q19	7246	1384	1426	1384
q20	666	322	309	309
q21	4915	3255	3156	3156
q22	348	281	281	281
Total cold run time: 117960 ms
Total hot run time: 39702 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4352	4255	4212	4212
q2	363	271	274	271
q3	2955	2902	2919	2902
q4	1980	1653	1717	1653
q5	5616	5523	5474	5474
q6	231	135	131	131
q7	2187	1894	1854	1854
q8	3235	3421	3434	3421
q9	8703	8798	8891	8798
q10	4022	3942	3771	3771
q11	573	508	504	504
q12	815	664	637	637
q13	16236	3153	3177	3153
q14	344	281	291	281
q15	529	489	495	489
q16	485	441	429	429
q17	1820	1567	1506	1506
q18	8186	8207	7943	7943
q19	1772	1527	1449	1449
q20	2194	1865	1878	1865
q21	9094	4888	4813	4813
q22	620	489	518	489
Total cold run time: 76312 ms
Total hot run time: 56045 ms

doris-robot · 2024-07-17T06:42:22Z

TPC-DS: Total hot run time: 173964 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit caf5a24a854cf797a391b588a26f763ee65a870d, data reload: false
query1	906	388	359	359
query2	6443	1992	1908	1908
query3	6639	218	221	218
query4	28489	17616	17423	17423
query5	3675	477	491	477
query6	261	172	181	172
query7	4585	292	288	288
query8	253	205	198	198
query9	8532	2380	2374	2374
query10	432	276	267	267
query11	10683	9961	10174	9961
query12	115	90	79	79
query13	1627	364	349	349
query14	10092	7710	7596	7596
query15	206	171	169	169
query16	7357	311	315	311
query17	1372	549	516	516
query18	1919	283	266	266
query19	197	143	146	143
query20	85	83	78	78
query21	199	126	127	126
query22	4340	4115	4209	4115
query23	33940	33610	33598	33598
query24	11338	2942	2882	2882
query25	605	378	392	378
query26	820	153	147	147
query27	2360	281	286	281
query28	5756	2057	2045	2045
query29	881	653	652	652
query30	257	156	155	155
query31	994	799	735	735
query32	99	52	55	52
query33	751	296	298	296
query34	1013	492	506	492
query35	710	591	573	573
query36	1130	986	1026	986
query37	146	83	90	83
query38	2989	2869	2889	2869
query39	900	864	842	842
query40	212	124	125	124
query41	48	45	48	45
query42	117	103	105	103
query43	503	458	481	458
query44	1242	721	745	721
query45	192	166	163	163
query46	1097	732	719	719
query47	1845	1817	1805	1805
query48	359	298	302	298
query49	857	427	433	427
query50	778	387	391	387
query51	6922	6911	6786	6786
query52	109	96	96	96
query53	360	287	290	287
query54	916	462	450	450
query55	78	76	74	74
query56	308	278	280	278
query57	1160	1064	1049	1049
query58	263	260	260	260
query59	3134	2733	2588	2588
query60	325	297	298	297
query61	114	112	113	112
query62	813	677	656	656
query63	316	288	287	287
query64	9300	2277	1753	1753
query65	3154	3089	3117	3089
query66	784	328	338	328
query67	15665	15173	14901	14901
query68	4591	540	548	540
query69	615	426	352	352
query70	1205	1131	1202	1131
query71	439	288	282	282
query72	7945	5849	5605	5605
query73	754	323	324	323
query74	6105	5739	5650	5650
query75	3586	2708	2683	2683
query76	2686	954	894	894
query77	672	310	307	307
query78	12326	9555	9073	9073
query79	10988	533	525	525
query80	1363	477	464	464
query81	585	222	228	222
query82	587	132	133	132
query83	336	166	171	166
query84	279	84	87	84
query85	723	321	301	301
query86	453	291	314	291
query87	3335	3088	3140	3088
query88	5812	2456	2390	2390
query89	493	380	369	369
query90	2123	191	193	191
query91	129	98	99	98
query92	75	48	50	48
query93	4537	500	502	500
query94	1306	215	214	214
query95	415	327	315	315
query96	609	281	281	281
query97	3213	2990	3020	2990
query98	233	199	195	195
query99	1517	1284	1260	1260
Total cold run time: 294102 ms
Total hot run time: 173964 ms

doris-robot · 2024-07-17T06:47:53Z

ClickBench: Total hot run time: 31.09 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit caf5a24a854cf797a391b588a26f763ee65a870d, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.09	0.08
query5	0.49	0.48	0.49
query6	1.14	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.53	0.49	0.48
query10	0.54	0.54	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.58	0.58
query14	0.77	0.79	0.77
query15	0.87	0.81	0.82
query16	0.37	0.36	0.37
query17	1.03	0.96	0.99
query18	0.23	0.22	0.21
query19	1.82	1.70	1.72
query20	0.01	0.00	0.01
query21	15.41	0.77	0.66
query22	4.25	6.77	2.40
query23	18.29	1.36	1.30
query24	2.08	0.23	0.22
query25	0.15	0.08	0.08
query26	0.30	0.22	0.21
query27	0.45	0.23	0.23
query28	13.29	1.01	1.01
query29	12.60	3.32	3.32
query30	0.24	0.06	0.05
query31	2.86	0.39	0.41
query32	3.28	0.48	0.47
query33	2.88	2.89	2.88
query34	17.07	4.39	4.36
query35	4.37	4.38	4.41
query36	0.67	0.46	0.50
query37	0.19	0.16	0.16
query38	0.15	0.15	0.15
query39	0.05	0.03	0.04
query40	0.14	0.13	0.12
query41	0.09	0.05	0.04
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.68 s
Total hot run time: 31.09 s

morrySnow · 2024-07-17T06:57:33Z

 LOG.debug("Start analyze plan");
 }
 keepOrShowPlanProcess(showPlanProcess, () -> cascadesContext.newAnalyzer().analyze());
+ if (this.cascadesContext.getConnectContext().getSessionVariable().isEnableDmlMaterializedViewRewrite()


u should add a analysis rule and check do this in rule. move hooks from NereidsPlanner into StatementContext

morrySnow · 2024-07-17T07:00:47Z

+ = "enable_dml_materialized_view_rewrite";
+
+ public static final String ENABLE_DML_MATERIALIZED_VIEW_REWRITE_WHEN_BASE_TABLE_UNAWARENESS
+ = "enable_dml_materialized_view_rewrite_when_base_table_unawareness";


unawareness? is it a standard phrases?

seawinde · 2024-07-17T12:33:29Z

run buildall

doris-robot · 2024-07-17T12:50:59Z

TPC-H: Total hot run time: 39975 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 901c1f4599a032c1a1e9eb5600acf755368cf368, data reload: false
------ Round 1 ----------------------------------
q1	18108	5114	4424	4424
q2	2565	207	196	196
q3	11501	1210	1092	1092
q4	10853	786	803	786
q5	7852	2826	2660	2660
q6	226	144	144	144
q7	981	607	599	599
q8	9209	2069	2057	2057
q9	8780	6505	6596	6505
q10	8765	3790	3789	3789
q11	438	240	243	240
q12	394	233	233	233
q13	17758	2991	3043	2991
q14	280	226	237	226
q15	526	477	478	477
q16	510	378	378	378
q17	969	633	792	633
q18	8009	7444	7416	7416
q19	8340	1325	1387	1325
q20	672	332	341	332
q21	4894	3228	3187	3187
q22	349	285	286	285
Total cold run time: 121979 ms
Total hot run time: 39975 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4461	4297	4262	4262
q2	383	268	275	268
q3	3037	2760	2795	2760
q4	1870	1623	1623	1623
q5	5328	5286	5283	5283
q6	221	134	135	134
q7	2096	1702	1738	1702
q8	3208	3350	3343	3343
q9	8458	8343	8391	8343
q10	3875	3630	3709	3630
q11	593	492	488	488
q12	800	608	600	600
q13	17435	2954	2953	2953
q14	305	266	269	266
q15	518	468	471	468
q16	462	414	434	414
q17	1766	1501	1474	1474
q18	7652	7517	7335	7335
q19	3357	1508	1437	1437
q20	1979	1790	1756	1756
q21	4772	4649	4694	4649
q22	576	516	525	516
Total cold run time: 73152 ms
Total hot run time: 53704 ms

doris-robot · 2024-07-17T13:02:03Z

TPC-DS: Total hot run time: 172991 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 901c1f4599a032c1a1e9eb5600acf755368cf368, data reload: false
query1	914	380	365	365
query2	6462	1921	1863	1863
query3	6652	208	221	208
query4	20096	17389	17295	17295
query5	4229	492	499	492
query6	276	183	172	172
query7	4610	292	296	292
query8	247	204	191	191
query9	8347	2391	2365	2365
query10	445	285	267	267
query11	11786	10009	10017	10009
query12	131	83	83	83
query13	1627	362	356	356
query14	10108	7854	7635	7635
query15	220	169	170	169
query16	7704	334	310	310
query17	1791	544	518	518
query18	1722	273	271	271
query19	191	148	148	148
query20	91	82	79	79
query21	209	127	127	127
query22	4276	4107	4066	4066
query23	33696	33002	33060	33002
query24	11100	2839	2868	2839
query25	644	367	361	361
query26	1519	147	150	147
query27	2876	272	274	272
query28	7549	1986	1969	1969
query29	984	643	608	608
query30	289	150	144	144
query31	954	763	728	728
query32	97	53	54	53
query33	784	321	319	319
query34	910	492	492	492
query35	698	578	577	577
query36	1068	947	928	928
query37	155	79	82	79
query38	2861	2762	2742	2742
query39	863	813	844	813
query40	283	121	121	121
query41	49	45	46	45
query42	118	100	98	98
query43	508	491	458	458
query44	1206	737	720	720
query45	188	157	160	157
query46	1093	734	733	733
query47	1834	1797	1742	1742
query48	356	287	285	285
query49	1173	415	414	414
query50	772	392	405	392
query51	6885	6804	6805	6804
query52	107	94	104	94
query53	355	298	295	295
query54	987	443	482	443
query55	78	75	76	75
query56	299	278	262	262
query57	1166	1038	1093	1038
query58	260	235	251	235
query59	2780	2678	2677	2677
query60	290	278	280	278
query61	96	93	96	93
query62	809	655	653	653
query63	326	294	294	294
query64	10587	2253	1676	1676
query65	3174	3101	3098	3098
query66	1338	352	334	334
query67	15430	14954	14988	14954
query68	4619	544	555	544
query69	476	328	325	325
query70	1179	1089	1167	1089
query71	400	280	280	280
query72	7011	6033	5583	5583
query73	762	332	333	332
query74	6113	5715	5603	5603
query75	3427	2710	2663	2663
query76	2546	953	944	944
query77	481	311	301	301
query78	11361	9323	12868	9323
query79	2055	532	527	527
query80	1583	488	475	475
query81	578	221	225	221
query82	281	135	134	134
query83	277	166	168	166
query84	279	92	87	87
query85	791	302	306	302
query86	458	331	313	313
query87	3298	3154	3121	3121
query88	3044	2439	2369	2369
query89	482	380	375	375
query90	1958	199	200	199
query91	132	100	101	100
query92	61	51	51	51
query93	1002	506	505	505
query94	1174	217	222	217
query95	407	322	326	322
query96	593	274	268	268
query97	3233	2996	3054	2996
query98	220	199	189	189
query99	1650	1245	1258	1245
Total cold run time: 273659 ms
Total hot run time: 172991 ms

doris-robot · 2024-07-17T13:07:33Z

ClickBench: Total hot run time: 30.88 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 901c1f4599a032c1a1e9eb5600acf755368cf368, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.06
query4	1.67	0.07	0.09
query5	0.50	0.51	0.48
query6	1.15	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.49	0.49
query10	0.54	0.53	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.58
query14	0.76	0.76	0.79
query15	0.83	0.80	0.81
query16	0.36	0.37	0.36
query17	0.96	0.96	1.00
query18	0.23	0.22	0.22
query19	1.82	1.69	1.66
query20	0.01	0.03	0.01
query21	15.40	0.75	0.66
query22	4.78	6.02	2.35
query23	18.29	1.36	1.23
query24	2.16	0.22	0.23
query25	0.16	0.09	0.08
query26	0.30	0.23	0.21
query27	0.46	0.23	0.23
query28	13.17	1.02	0.99
query29	12.58	3.36	3.33
query30	0.25	0.06	0.06
query31	2.86	0.38	0.39
query32	3.28	0.48	0.46
query33	2.92	2.90	2.89
query34	17.05	4.30	4.36
query35	4.43	4.40	4.47
query36	0.64	0.47	0.48
query37	0.19	0.16	0.15
query38	0.15	0.14	0.15
query39	0.05	0.03	0.04
query40	0.15	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.16 s
Total hot run time: 30.88 s

seawinde · 2024-07-18T04:34:59Z

run buildall

doris-robot · 2024-07-18T05:20:23Z

TPC-H: Total hot run time: 39795 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ddf54d58e28a9cf1152aaf6ea363994841d66eec, data reload: false
------ Round 1 ----------------------------------
q1	17608	4371	4253	4253
q2	2020	198	191	191
q3	10533	1212	1167	1167
q4	10195	836	808	808
q5	7512	2707	2698	2698
q6	215	133	139	133
q7	957	598	591	591
q8	9213	2075	2095	2075
q9	8788	6540	6539	6539
q10	8791	3784	3761	3761
q11	452	236	235	235
q12	433	231	221	221
q13	17779	2973	2991	2973
q14	269	235	243	235
q15	528	489	488	488
q16	496	380	388	380
q17	979	655	674	655
q18	7960	7549	7330	7330
q19	6960	1413	1282	1282
q20	688	345	337	337
q21	4813	3163	3282	3163
q22	345	298	280	280
Total cold run time: 117534 ms
Total hot run time: 39795 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4380	4227	4221	4221
q2	365	261	259	259
q3	3014	2920	2939	2920
q4	2072	1697	1764	1697
q5	5619	5494	5501	5494
q6	219	142	135	135
q7	2227	1904	1883	1883
q8	3243	3391	3439	3391
q9	8753	8823	8860	8823
q10	4162	3812	3876	3812
q11	591	493	498	493
q12	840	630	646	630
q13	17226	3169	3141	3141
q14	329	291	280	280
q15	542	487	475	475
q16	493	424	430	424
q17	1798	1523	1514	1514
q18	8126	7902	7817	7817
q19	1779	1609	1567	1567
q20	2117	1875	1827	1827
q21	5197	5078	4870	4870
q22	588	517	510	510
Total cold run time: 73680 ms
Total hot run time: 56183 ms

doris-robot · 2024-07-18T05:31:30Z

TPC-DS: Total hot run time: 174140 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ddf54d58e28a9cf1152aaf6ea363994841d66eec, data reload: false
query1	921	376	364	364
query2	6473	1866	1838	1838
query3	6638	234	213	213
query4	28358	17457	17415	17415
query5	3647	492	479	479
query6	286	171	173	171
query7	4601	305	294	294
query8	251	194	203	194
query9	8591	2465	2436	2436
query10	439	308	277	277
query11	10680	10112	10136	10112
query12	117	87	84	84
query13	1653	376	372	372
query14	10278	7867	6849	6849
query15	236	168	163	163
query16	7687	338	312	312
query17	1787	560	526	526
query18	1956	275	287	275
query19	189	146	146	146
query20	87	82	83	82
query21	202	136	136	136
query22	4364	4199	3992	3992
query23	34209	33846	34086	33846
query24	10711	2979	2910	2910
query25	611	387	393	387
query26	702	155	157	155
query27	2242	281	285	281
query28	6081	2071	2064	2064
query29	889	632	652	632
query30	263	158	161	158
query31	973	741	774	741
query32	97	58	56	56
query33	652	308	309	308
query34	894	489	502	489
query35	689	563	577	563
query36	1131	996	959	959
query37	142	87	88	87
query38	2936	2855	2830	2830
query39	907	823	837	823
query40	199	123	125	123
query41	45	44	48	44
query42	111	101	97	97
query43	512	453	476	453
query44	1079	716	719	716
query45	192	164	162	162
query46	1085	728	708	708
query47	1865	1785	1774	1774
query48	379	295	304	295
query49	822	407	413	407
query50	774	398	394	394
query51	6916	6895	6787	6787
query52	103	89	97	89
query53	357	298	290	290
query54	871	461	447	447
query55	75	76	72	72
query56	283	277	267	267
query57	1116	1073	1063	1063
query58	264	251	288	251
query59	2850	2603	2657	2603
query60	305	282	310	282
query61	100	93	92	92
query62	786	635	623	623
query63	324	289	292	289
query64	9139	2235	1671	1671
query65	3188	3128	3112	3112
query66	698	326	327	326
query67	15552	15071	14921	14921
query68	4555	543	557	543
query69	445	326	338	326
query70	1189	1159	1096	1096
query71	378	285	283	283
query72	6871	6252	6315	6252
query73	754	326	323	323
query74	6139	5696	5744	5696
query75	3373	2694	2674	2674
query76	2726	1022	915	915
query77	437	313	303	303
query78	9852	9737	9063	9063
query79	2238	528	519	519
query80	1882	483	480	480
query81	584	226	219	219
query82	825	133	136	133
query83	275	166	171	166
query84	275	88	89	88
query85	1979	317	307	307
query86	475	305	338	305
query87	3374	3123	3098	3098
query88	4073	2445	2433	2433
query89	478	411	390	390
query90	1766	199	194	194
query91	129	102	105	102
query92	60	50	52	50
query93	2401	514	514	514
query94	1071	218	218	218
query95	419	316	319	316
query96	599	276	271	271
query97	3204	3062	3042	3042
query98	214	276	196	196
query99	1506	1248	1248	1248
Total cold run time: 278106 ms
Total hot run time: 174140 ms

doris-robot · 2024-07-18T05:37:02Z

ClickBench: Total hot run time: 30.64 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ddf54d58e28a9cf1152aaf6ea363994841d66eec, data reload: false
query1	0.04	0.04	0.02
query2	0.08	0.04	0.03
query3	0.22	0.05	0.05
query4	1.67	0.08	0.08
query5	0.50	0.48	0.52
query6	1.13	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.50	0.50
query10	0.55	0.55	0.55
query11	0.16	0.11	0.12
query12	0.16	0.13	0.13
query13	0.60	0.58	0.58
query14	0.76	0.78	0.77
query15	0.86	0.81	0.82
query16	0.37	0.37	0.37
query17	0.99	1.00	0.97
query18	0.23	0.21	0.21
query19	1.80	1.69	1.69
query20	0.01	0.01	0.02
query21	15.39	0.78	0.66
query22	4.66	6.93	1.98
query23	18.28	1.30	1.33
query24	2.07	0.23	0.24
query25	0.15	0.09	0.09
query26	0.29	0.22	0.21
query27	0.45	0.23	0.23
query28	13.30	1.02	1.01
query29	12.64	3.26	3.29
query30	0.26	0.07	0.06
query31	2.85	0.40	0.37
query32	3.28	0.47	0.48
query33	2.85	2.96	2.87
query34	16.99	4.31	4.45
query35	4.40	4.43	4.45
query36	0.66	0.47	0.45
query37	0.18	0.16	0.15
query38	0.15	0.15	0.15
query39	0.04	0.03	0.03
query40	0.15	0.12	0.13
query41	0.09	0.05	0.05
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.97 s
Total hot run time: 30.64 s

github-actions · 2024-07-18T13:42:17Z

PR approved by at least one committer and no changes requested.

morrySnow · 2024-07-18T13:46:18Z

+ @Override
+ public List<Rule> buildRules() {
+ return ImmutableList.of(
+ RuleType.INIT_MATERIALIZATION_HOOK_FOR_FILE_SINK.build(logicalFileSink()


could use any().when(LogicalTableSink.class::isInstanceOf)

…sert and insert overwrite dml

seawinde · 2024-07-19T03:02:26Z

this is replaced by #38115, this branch rebase wrongly, it doesn't modify the meta etc
So close this pr

zfr9527 approved these changes Jul 10, 2024
View reviewed changes

github-actionsBot added the reviewed label Jul 10, 2024

seawinde force-pushed the ddl_enable_mv_rewrite branch from 46f9756 to d0e868cCompare July 10, 2024 11:59

zddr reviewed Jul 12, 2024
View reviewed changes

morrySnow added the dev/2.1.x label Jul 17, 2024

morrySnow reviewed Jul 17, 2024
View reviewed changes

zddr approved these changes Jul 18, 2024
View reviewed changes

morrySnow previously approved these changes Jul 18, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 18, 2024

morrySnow self-requested a review July 18, 2024 13:45

morrySnow reviewed Jul 18, 2024
View reviewed changes

seawinde dismissed morrySnow’s stale review via ac8d705July 19, 2024 02:56

[feature](mtmv) Support querying rewrite by materialized view when in…
ad9f822
…sert and insert overwrite dml

seawinde force-pushed the ddl_enable_mv_rewrite branch from ac8d705 to ad9f822Compare July 19, 2024 02:57

seawinde closed this Jul 19, 2024

seawinde mentioned this pull request Jul 19, 2024
[feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml #38115
Merged

morrySnow removed meta-change dev/2.1.x labels Jul 19, 2024

Uh oh!

Conversation

seawinde commented Jul 9, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Jul 9, 2024

Uh oh!

seawinde commented Jul 9, 2024

Uh oh!

seawinde commented Jul 9, 2024

Uh oh!

seawinde commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

zfr9527 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 10, 2024

Uh oh!

seawinde commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

seawinde commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

doris-robot commented Jul 10, 2024

Uh oh!

seawinde commented Jul 11, 2024

Uh oh!

doris-robot commented Jul 11, 2024

Uh oh!

doris-robot commented Jul 11, 2024

Uh oh!

doris-robot commented Jul 11, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

seawinde commented Jul 12, 2024

Uh oh!

doris-robot commented Jul 12, 2024

Uh oh!

doris-robot commented Jul 12, 2024

Uh oh!

doris-robot commented Jul 12, 2024

Uh oh!

seawinde commented Jul 16, 2024

Uh oh!

doris-robot commented Jul 16, 2024

Uh oh!

seawinde commented Jul 17, 2024

Uh oh!

github-actionsBot commented Jul 17, 2024

Uh oh!

github-actionsBot commented Jul 17, 2024

Uh oh!

doris-robot commented Jul 17, 2024

Uh oh!

doris-robot commented Jul 17, 2024

Uh oh!

doris-robot commented Jul 17, 2024

Uh oh!

morrySnowJul 17, 2024

Choose a reason for hiding this comment

seawinde commented Jul 9, 2024•
edited
Loading

seawinde commented Jul 19, 2024•
edited
Loading