[enhance](mtmv)During cache generation, no longer hold the write lock for mtmv by zddr · Pull Request #40402 · apache/doris

zddr · 2024-09-05T03:33:40Z

During cache generation, no longer hold the write Lock for mv to avoid changes in the logic of cache generation in the future, internal calls to other locks, and deadlocks

doris-robot · 2024-09-05T03:33:45Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

zddr · 2024-09-05T03:33:54Z

run buildall

doris-robot · 2024-09-05T04:07:32Z

TPC-H: Total hot run time: 38193 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 51cac12dbc1d22090b293fcede75cc2785c38b02, data reload: false
------ Round 1 ----------------------------------
q1	17744	4656	4312	4312
q2	2025	191	183	183
q3	11489	1000	1143	1000
q4	10507	820	821	820
q5	7767	2838	2856	2838
q6	226	141	145	141
q7	950	612	615	612
q8	9322	2042	2074	2042
q9	7182	6513	6529	6513
q10	7032	2212	2178	2178
q11	458	240	235	235
q12	395	226	227	226
q13	18794	3111	3080	3080
q14	277	244	241	241
q15	538	471	482	471
q16	621	506	503	503
q17	992	723	722	722
q18	7474	6860	6817	6817
q19	1391	1008	1085	1008
q20	665	330	328	328
q21	3827	3000	2943	2943
q22	1124	1026	980	980
Total cold run time: 110800 ms
Total hot run time: 38193 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4400	4303	4286	4286
q2	387	280	264	264
q3	2885	2664	2708	2664
q4	1990	1677	1666	1666
q5	5708	5706	5798	5706
q6	227	132	141	132
q7	2195	1864	1853	1853
q8	3270	3414	3490	3414
q9	8873	8881	8816	8816
q10	3641	3377	3316	3316
q11	605	511	505	505
q12	823	665	679	665
q13	16342	3321	3304	3304
q14	333	285	291	285
q15	524	486	493	486
q16	624	584	574	574
q17	1851	1543	1504	1504
q18	8000	7919	7881	7881
q19	1735	1649	1600	1600
q20	2137	1915	1952	1915
q21	5788	5451	5475	5451
q22	1089	1047	1070	1047
Total cold run time: 73427 ms
Total hot run time: 57334 ms

doris-robot · 2024-09-05T04:19:49Z

TPC-DS: Total hot run time: 193320 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 51cac12dbc1d22090b293fcede75cc2785c38b02, data reload: false
query1	1273	877	907	877
query2	6295	2000	1887	1887
query3	10611	3877	4027	3877
query4	59587	23709	23294	23294
query5	5405	489	499	489
query6	397	178	163	163
query7	5769	307	296	296
query8	306	216	205	205
query9	9042	2497	2483	2483
query10	500	274	262	262
query11	17953	15048	15242	15048
query12	149	110	105	105
query13	1593	391	390	390
query14	11160	7455	7267	7267
query15	232	175	185	175
query16	7614	437	477	437
query17	1166	567	574	567
query18	2008	308	296	296
query19	293	162	165	162
query20	127	115	126	115
query21	208	104	106	104
query22	4628	4463	4603	4463
query23	34193	33790	33297	33297
query24	5947	2827	2828	2827
query25	558	396	401	396
query26	697	162	160	160
query27	1802	281	281	281
query28	3663	2080	2051	2051
query29	725	414	424	414
query30	237	160	164	160
query31	944	772	786	772
query32	85	54	58	54
query33	459	299	295	295
query34	864	467	487	467
query35	830	724	734	724
query36	1074	942	948	942
query37	156	102	83	83
query38	4118	3844	3917	3844
query39	1443	1445	1396	1396
query40	204	117	117	117
query41	47	45	46	45
query42	120	100	98	98
query43	518	496	473	473
query44	1091	779	750	750
query45	198	168	163	163
query46	1092	760	743	743
query47	1905	1802	1821	1802
query48	379	303	303	303
query49	788	451	458	451
query50	853	429	432	429
query51	6994	6800	6777	6777
query52	102	86	90	86
query53	259	181	183	181
query54	579	462	462	462
query55	77	80	75	75
query56	292	265	268	265
query57	1185	1075	1059	1059
query58	239	235	237	235
query59	3053	2866	2839	2839
query60	307	275	284	275
query61	124	124	124	124
query62	761	673	650	650
query63	221	191	188	188
query64	2938	671	699	671
query65	3216	3141	3157	3141
query66	688	340	337	337
query67	15338	15398	15343	15343
query68	4393	572	565	565
query69	414	276	283	276
query70	1201	1028	1060	1028
query71	347	281	285	281
query72	6562	4054	4036	4036
query73	760	331	322	322
query74	9172	8819	8849	8819
query75	3370	2682	2722	2682
query76	1710	1082	951	951
query77	564	338	312	312
query78	11102	9560	9538	9538
query79	1806	520	531	520
query80	926	523	514	514
query81	581	238	237	237
query82	561	150	146	146
query83	212	148	148	148
query84	262	76	124	76
query85	703	287	286	286
query86	416	282	301	282
query87	4445	4273	4278	4273
query88	3352	2393	2367	2367
query89	389	289	287	287
query90	2025	196	195	195
query91	125	97	97	97
query92	63	48	50	48
query93	1596	544	534	534
query94	811	289	283	283
query95	361	290	258	258
query96	596	268	266	266
query97	3223	3089	3043	3043
query98	214	200	198	198
query99	1550	1290	1245	1245
Total cold run time: 311491 ms
Total hot run time: 193320 ms

doris-robot · 2024-09-05T04:25:26Z

ClickBench: Total hot run time: 31.95 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 51cac12dbc1d22090b293fcede75cc2785c38b02, data reload: false
query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.07	0.08
query5	0.51	0.50	0.49
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.56	0.56	0.54
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.60	0.58	0.58
query14	2.04	2.05	2.06
query15	0.85	0.82	0.81
query16	0.34	0.36	0.36
query17	0.99	0.97	1.01
query18	0.22	0.20	0.22
query19	1.87	1.87	1.81
query20	0.01	0.01	0.00
query21	15.39	0.68	0.66
query22	4.85	6.11	1.88
query23	18.25	1.42	1.30
query24	2.08	0.23	0.21
query25	0.15	0.08	0.08
query26	0.26	0.19	0.18
query27	0.07	0.08	0.07
query28	13.33	1.01	1.00
query29	12.65	3.37	3.32
query30	0.25	0.05	0.06
query31	2.88	0.42	0.41
query32	3.23	0.47	0.48
query33	2.99	2.98	3.02
query34	17.12	4.34	4.41
query35	4.43	4.42	4.54
query36	0.65	0.47	0.46
query37	0.19	0.16	0.16
query38	0.14	0.14	0.14
query39	0.05	0.04	0.04
query40	0.17	0.13	0.13
query41	0.09	0.06	0.05
query42	0.07	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 111.41 s
Total hot run time: 31.95 s

github-actions · 2024-09-05T04:26:02Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-09-05T04:26:05Z

PR approved by anyone and no changes requested.

zddr · 2024-09-05T06:29:41Z

run external

morrySnow · 2024-09-05T06:29:46Z

run external

…k for mtmv (apache#40402) During cache generation, no longer hold the write Lock for mv to avoid changes in the logic of cache generation in the future, internal calls to other locks, and deadlocks

#40489) …k for mtmv (#40402) pick: #40402

…k for mtmv (#40402) During cache generation, no longer hold the write Lock for mv to avoid changes in the logic of cache generation in the future, internal calls to other locks, and deadlocks

…k for mtmv (apache#40402) During cache generation, no longer hold the write Lock for mv to avoid changes in the logic of cache generation in the future, internal calls to other locks, and deadlocks

1
51cac12

morrySnow approved these changes Sep 5, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2024

github-actionsBot added the reviewed label Sep 5, 2024

seawinde approved these changes Sep 5, 2024
View reviewed changes

morrySnow merged commit e23696c into apache:masterSep 6, 2024

morrySnow added the dev/2.1.x label Sep 6, 2024

zddr mentioned this pull request Sep 6, 2024
[enhance](mtmv) During cache generation, no longer hold the write loc… #40489
Merged

yiguolei pushed a commit that referenced this pull request Sep 9, 2024
[enhance](mtmv) During cache generation, no longer hold the write loc… (
01a651e
#40489) …k for mtmv (#40402) pick: #40402

yiguolei added dev/2.1.7-merged and removed dev/2.1.x labels Sep 9, 2024

dataroaring added the dev/3.0.2-merged label Sep 26, 2024

yiguolei mentioned this pull request Nov 6, 2024
Release Note 2.1.7 #43319
Closed

yiguolei mentioned this pull request Jan 19, 2025
Release Note 2.1.8 #47198
Closed

Navigation Menu

Uh oh!

[enhance](mtmv)During cache generation, no longer hold the write lock for mtmv - #40402

[enhance](mtmv)During cache generation, no longer hold the write lock for mtmv#40402
morrySnow merged 1 commit into
apache:masterfrom
zddr:cache_without_writelock

zddr commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

zddr commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

zddr commented Sep 5, 2024

Uh oh!

morrySnow commented Sep 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

zddr commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

zddr commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

zddr commented Sep 5, 2024

Uh oh!

morrySnow commented Sep 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants