Skip to content

[fix](mtmv)mtmv disable dynamic partition - #35686

Merged
morrySnow merged 6 commits into
apache:masterfrom
zddr:mv_disable_dynamic_partition
Jun 4, 2024
Merged

[fix](mtmv)mtmv disable dynamic partition#35686
morrySnow merged 6 commits into
apache:masterfrom
zddr:mv_disable_dynamic_partition

Conversation

@zddr

@zddrzddr commented May 31, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

  1. MTMV do not allow setting dynamic partition related properties
  2. add case for show data

@doris-robot

doris-robot commented May 31, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zddr

zddr commented May 31, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented May 31, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 42038 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a3e78fc1488997aac8ffba32a6034ae45d141fd4, data reload: false
------ Round 1 ----------------------------------
q1	17610	4373	4286	4286
q2	2021	203	194	194
q3	10441	1208	1256	1208
q4	10215	777	865	777
q5	7484	2926	2761	2761
q6	226	134	153	134
q7	951	628	608	608
q8	9232	2115	2127	2115
q9	9246	6695	6731	6695
q10	9391	3888	3874	3874
q11	441	242	241	241
q12	469	225	243	225
q13	17297	3169	3282	3169
q14	277	236	230	230
q15	516	478	489	478
q16	481	382	396	382
q17	988	671	706	671
q18	8444	7750	7825	7750
q19	6245	1619	1619	1619
q20	639	320	302	302
q21	5104	3990	4044	3990
q22	391	329	339	329
Total cold run time: 118109 ms
Total hot run time: 42038 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4549	4409	4406	4406
q2	389	274	265	265
q3	3166	2941	2897	2897
q4	1907	1612	1590	1590
q5	5471	5515	5479	5479
q6	213	129	129	129
q7	2168	1864	1854	1854
q8	3237	3375	3388	3375
q9	8615	8692	8676	8676
q10	3929	3732	3865	3732
q11	594	489	505	489
q12	814	625	643	625
q13	16288	3151	3181	3151
q14	309	274	272	272
q15	537	475	471	471
q16	474	448	443	443
q17	1806	1516	1478	1478
q18	7683	7682	7397	7397
q19	1659	1591	1569	1569
q20	2043	1805	1812	1805
q21	9300	4672	4666	4666
q22	609	527	534	527
Total cold run time: 75760 ms
Total hot run time: 55296 ms

@doris-robot

doris-robot commented May 31, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 170590 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a3e78fc1488997aac8ffba32a6034ae45d141fd4, data reload: false
query1	921	385	385	385
query2	6450	2485	2537	2485
query3	6635	205	204	204
query4	19151	17586	17443	17443
query5	4115	424	430	424
query6	241	159	152	152
query7	4578	317	302	302
query8	327	283	287	283
query9	8442	2387	2377	2377
query10	448	310	284	284
query11	11112	10067	9984	9984
query12	132	91	88	88
query13	1649	371	359	359
query14	9022	7644	7612	7612
query15	239	188	187	187
query16	7763	261	268	261
query17	1716	543	535	535
query18	1855	269	272	269
query19	218	156	159	156
query20	95	86	85	85
query21	189	133	133	133
query22	4172	4186	3981	3981
query23	33610	33319	33190	33190
query24	8637	2828	2860	2828
query25	590	356	356	356
query26	703	158	157	157
query27	2212	326	326	326
query28	5613	2048	2065	2048
query29	856	605	617	605
query30	253	152	153	152
query31	979	761	749	749
query32	97	52	55	52
query33	641	290	284	284
query34	859	481	496	481
query35	711	611	602	602
query36	1072	911	912	911
query37	101	67	70	67
query38	2933	2782	2772	2772
query39	850	780	815	780
query40	194	129	129	129
query41	51	47	49	47
query42	104	99	100	99
query43	595	563	572	563
query44	1070	726	750	726
query45	182	168	177	168
query46	1068	716	714	714
query47	1839	1771	1747	1747
query48	376	294	292	292
query49	841	382	379	379
query50	773	396	385	385
query51	6743	6608	6672	6608
query52	106	90	91	90
query53	351	292	318	292
query54	768	434	417	417
query55	72	71	72	71
query56	270	241	247	241
query57	1131	1026	1040	1026
query58	228	210	211	210
query59	3440	3128	3135	3128
query60	273	265	254	254
query61	92	88	89	88
query62	593	443	453	443
query63	310	290	308	290
query64	8613	2162	1748	1748
query65	3192	3097	3128	3097
query66	795	330	326	326
query67	15114	14907	14720	14720
query68	4498	541	535	535
query69	432	269	279	269
query70	1162	1046	1075	1046
query71	373	266	268	266
query72	7667	5401	2726	2726
query73	701	321	322	321
query74	5987	5605	5577	5577
query75	3265	2641	2632	2632
query76	2330	1027	959	959
query77	409	265	263	263
query78	10407	10050	9691	9691
query79	2398	514	507	507
query80	1127	450	437	437
query81	536	221	219	219
query82	731	92	91	91
query83	255	172	174	172
query84	248	89	88	88
query85	1611	283	291	283
query86	497	329	294	294
query87	3272	3148	3205	3148
query88	4247	2355	2351	2351
query89	478	387	383	383
query90	2002	191	196	191
query91	127	96	98	96
query92	60	47	50	47
query93	2041	510	506	506
query94	1241	190	186	186
query95	395	313	314	313
query96	582	266	264	264
query97	3190	3018	3042	3018
query98	245	218	208	208
query99	1120	856	872	856
Total cold run time: 263005 ms
Total hot run time: 170590 ms

@doris-robot

doris-robot commented May 31, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.04 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a3e78fc1488997aac8ffba32a6034ae45d141fd4, data reload: false
query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.07
query5	0.50	0.49	0.49
query6	1.13	0.73	0.73
query7	0.01	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.50
query10	0.55	0.55	0.54
query11	0.17	0.12	0.12
query12	0.15	0.12	0.12
query13	0.60	0.59	0.59
query14	0.79	0.78	0.79
query15	0.83	0.82	0.80
query16	0.36	0.38	0.37
query17	0.93	0.96	1.03
query18	0.21	0.24	0.24
query19	1.79	1.68	1.65
query20	0.02	0.01	0.01
query21	15.50	0.70	0.66
query22	4.01	6.68	2.44
query23	18.33	1.41	1.34
query24	2.13	0.21	0.20
query25	0.15	0.09	0.08
query26	0.27	0.16	0.16
query27	0.09	0.08	0.08
query28	13.30	1.03	0.99
query29	13.33	3.30	3.26
query30	0.24	0.06	0.05
query31	2.89	0.38	0.38
query32	3.28	0.47	0.46
query33	2.86	2.93	2.86
query34	17.02	4.44	4.42
query35	4.56	4.47	4.60
query36	0.70	0.49	0.46
query37	0.17	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.16	0.14	0.14
query41	0.10	0.05	0.04
query42	0.05	0.04	0.05
query43	0.04	0.03	0.04
Total cold run time: 110.05 s
Total hot run time: 31.04 s

@zddr

zddr commented May 31, 2024

Copy link
Copy Markdown
ContributorAuthor

run feut

@github-actions

github-actionsBot commented May 31, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@cambyzju

cambyzju commented May 31, 2024

Copy link
Copy Markdown
Contributor

branch-2.0 also have this problem?

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 3, 2024
@github-actions

github-actionsBot commented Jun 3, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@zddr

zddr commented Jun 4, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Jun 4, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41756 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 249fe5d69b7e5056a43e0f83b1de5be01a82c386, data reload: false
------ Round 1 ----------------------------------
q1	17619	4412	4288	4288
q2	2022	195	204	195
q3	10423	1216	1178	1178
q4	10198	763	820	763
q5	7491	2744	2774	2744
q6	221	139	141	139
q7	964	613	617	613
q8	9227	2179	2128	2128
q9	9445	6677	6754	6677
q10	9732	3978	3877	3877
q11	432	262	251	251
q12	524	247	239	239
q13	18495	3246	3171	3171
q14	276	219	222	219
q15	520	490	465	465
q16	516	411	401	401
q17	1001	664	616	616
q18	8442	7723	7776	7723
q19	7585	1322	1569	1322
q20	639	334	325	325
q21	5263	4074	4108	4074
q22	407	348	356	348
Total cold run time: 121442 ms
Total hot run time: 41756 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4577	4503	4467	4467
q2	374	274	262	262
q3	3178	3006	2777	2777
q4	1889	1650	1662	1650
q5	5488	5500	5498	5498
q6	222	131	130	130
q7	2187	1832	1817	1817
q8	3245	3392	3390	3390
q9	8644	8729	8690	8690
q10	4082	3786	3731	3731
q11	590	486	492	486
q12	800	635	589	589
q13	15888	3178	3168	3168
q14	299	268	269	268
q15	510	478	512	478
q16	479	416	442	416
q17	1844	1513	1502	1502
q18	7981	7562	7452	7452
q19	1694	1566	1473	1473
q20	2082	1787	1785	1785
q21	4913	4604	4627	4604
q22	624	539	548	539
Total cold run time: 71590 ms
Total hot run time: 55172 ms

@doris-robot

doris-robot commented Jun 4, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 172121 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 249fe5d69b7e5056a43e0f83b1de5be01a82c386, data reload: false
query1	932	391	376	376
query2	6458	2339	2348	2339
query3	6642	209	208	208
query4	20194	17242	17255	17242
query5	4134	449	451	449
query6	257	153	153	153
query7	4590	301	293	293
query8	326	282	293	282
query9	8653	2471	2471	2471
query10	444	311	282	282
query11	10430	9968	9962	9962
query12	134	90	97	90
query13	1639	357	370	357
query14	10174	6850	7413	6850
query15	225	187	197	187
query16	7555	274	270	270
query17	1391	520	513	513
query18	1930	280	269	269
query19	204	151	148	148
query20	97	86	85	85
query21	209	130	126	126
query22	4337	3936	3980	3936
query23	33520	33053	32952	32952
query24	11380	2855	2837	2837
query25	614	359	360	359
query26	1201	154	153	153
query27	2672	320	315	315
query28	7061	2081	2066	2066
query29	894	598	600	598
query30	268	148	157	148
query31	979	731	718	718
query32	87	52	54	52
query33	752	291	273	273
query34	972	494	479	479
query35	733	619	585	585
query36	1071	915	883	883
query37	148	67	68	67
query38	2878	2779	2757	2757
query39	878	780	795	780
query40	201	122	122	122
query41	52	50	50	50
query42	119	97	97	97
query43	589	565	552	552
query44	1224	732	740	732
query45	205	175	165	165
query46	1069	750	716	716
query47	1887	1748	1802	1748
query48	363	298	290	290
query49	1069	399	406	399
query50	822	391	402	391
query51	6901	6833	6757	6757
query52	110	87	88	87
query53	348	282	282	282
query54	908	452	449	449
query55	74	74	72	72
query56	277	251	264	251
query57	1163	1049	1074	1049
query58	255	258	230	230
query59	3513	3242	3022	3022
query60	293	269	261	261
query61	89	84	86	84
query62	629	447	435	435
query63	313	297	287	287
query64	8884	2163	1698	1698
query65	3181	3096	3121	3096
query66	963	342	328	328
query67	15233	14946	14954	14946
query68	4507	540	545	540
query69	469	323	319	319
query70	1064	1119	1121	1119
query71	388	293	282	282
query72	7292	5618	5413	5413
query73	763	324	327	324
query74	5899	5419	5475	5419
query75	3328	2672	2672	2672
query76	2235	965	950	950
query77	461	292	289	289
query78	10389	9878	9673	9673
query79	1782	520	514	514
query80	2514	462	444	444
query81	547	219	222	219
query82	1067	101	104	101
query83	301	173	175	173
query84	265	85	86	85
query85	1191	259	248	248
query86	401	324	287	287
query87	3243	3079	3108	3079
query88	3138	2427	2421	2421
query89	478	403	382	382
query90	1754	193	193	193
query91	128	95	97	95
query92	59	50	52	50
query93	1559	514	500	500
query94	1252	185	183	183
query95	394	309	308	308
query96	584	273	267	267
query97	3199	3040	3033	3033
query98	232	215	213	213
query99	1145	820	833	820
Total cold run time: 269825 ms
Total hot run time: 172121 ms

@doris-robot

doris-robot commented Jun 4, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.18 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 249fe5d69b7e5056a43e0f83b1de5be01a82c386, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.09	0.09
query5	0.54	0.49	0.48
query6	1.12	0.72	0.74
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.52	0.49	0.48
query10	0.55	0.54	0.54
query11	0.15	0.11	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.60
query14	0.77	0.77	0.80
query15	0.82	0.82	0.81
query16	0.35	0.37	0.37
query17	0.98	1.01	1.00
query18	0.21	0.26	0.24
query19	1.79	1.72	1.66
query20	0.01	0.02	0.01
query21	15.50	0.70	0.68
query22	4.49	6.35	2.40
query23	18.32	1.43	1.28
query24	1.95	0.22	0.22
query25	0.15	0.10	0.08
query26	0.27	0.17	0.17
query27	0.08	0.08	0.08
query28	13.26	1.01	1.00
query29	13.39	3.33	3.32
query30	0.24	0.07	0.06
query31	2.86	0.39	0.39
query32	3.28	0.47	0.46
query33	2.88	2.94	2.96
query34	17.08	4.43	4.37
query35	4.51	4.52	4.64
query36	0.65	0.47	0.46
query37	0.18	0.16	0.16
query38	0.15	0.15	0.14
query39	0.04	0.04	0.03
query40	0.16	0.14	0.15
query41	0.08	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.03	0.04
Total cold run time: 110.26 s
Total hot run time: 31.18 s

@morrySnow
morrySnow merged commit 4e85439 into apache:masterJun 4, 2024
seawinde pushed a commit to seawinde/doris that referenced this pull request Jun 5, 2024
@zddr@seawinde
[fix](mtmv) mtmv disable dynamic partition (apache#35686)
033d9c8
1. MTMV do not allow setting dynamic partition related properties
2. add case for show data
morningman pushed a commit to morningman/doris that referenced this pull request Jun 6, 2024
@zddr@morningman
[fix](mtmv) mtmv disable dynamic partition (apache#35686)
6500ae0
1. MTMV do not allow setting dynamic partition related properties
2. add case for show data
@morningmanmorningman mentioned this pull request Jun 6, 2024
Merged
morningman added a commit that referenced this pull request Jun 6, 2024
@morningman@zddr
[fix](mtmv) mtmv disable dynamic partition (#35686) (#35949)
104fcea
bp #35686
Co-authored-by: zhangdong <493738387@qq.com>
dataroaring pushed a commit that referenced this pull request Jun 7, 2024
@zddr@dataroaring
[fix](mtmv) mtmv disable dynamic partition (#35686)
6300c5d
1. MTMV do not allow setting dynamic partition related properties
2. add case for show data
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@zddr@HappenLee
[fix](mtmv) mtmv disable dynamic partition (apache#35686)
95f55b8
1. MTMV do not allow setting dynamic partition related properties
2. add case for show data
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnowmorrySnowmorrySnow approved these changes
@seawindeseawindeseawinde approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.4-mergeddev/3.0.0-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zddr@doris-robot@cambyzju@morrySnow@seawinde@morningman@dataroaring