[Fix]Fix cgroup v2 init by wangbo · Pull Request #39991 · apache/doris

wangbo · 2024-08-27T09:29:59Z

Proposed changes

1 Add a step to write pid to query path's cgroup.procs, otherwrise write thread id may failed.
2 Move some cgroup v2's public file to parent class to init init them once.

doris-robot · 2024-08-27T09:30:04Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

wangbo · 2024-08-27T09:30:06Z

run buildall

github-actions · 2024-08-27T09:36:40Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-27T10:37:19Z

TPC-H: Total hot run time: 38412 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 984c9aac6443ba67c905f22ae1f93bf6528eafcc, data reload: false
------ Round 1 ----------------------------------
q1	18173	4592	4503	4503
q2	2809	187	190	187
q3	11728	1139	1170	1139
q4	10733	756	778	756
q5	8691	2916	2833	2833
q6	226	141	137	137
q7	980	636	614	614
q8	9345	2091	2056	2056
q9	7025	6616	6606	6606
q10	7007	2194	2178	2178
q11	460	245	253	245
q12	406	234	231	231
q13	17777	3022	3016	3016
q14	276	241	250	241
q15	530	498	491	491
q16	504	391	398	391
q17	992	683	675	675
q18	7517	6939	6926	6926
q19	1387	992	1115	992
q20	697	336	333	333
q21	4092	3129	2870	2870
q22	1093	992	1013	992
Total cold run time: 112448 ms
Total hot run time: 38412 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4396	4346	4319	4319
q2	384	273	281	273
q3	2904	2635	2651	2635
q4	1950	1662	1601	1601
q5	5431	5411	5441	5411
q6	220	135	134	134
q7	2140	1748	1770	1748
q8	3228	3412	3399	3399
q9	8472	8454	8511	8454
q10	3465	3206	3233	3206
q11	596	507	512	507
q12	804	611	606	606
q13	13067	3087	3084	3084
q14	313	270	268	268
q15	538	497	475	475
q16	486	429	427	427
q17	1811	1528	1490	1490
q18	7786	7374	7390	7374
q19	1714	1682	1646	1646
q20	2048	1836	1841	1836
q21	5388	5088	5370	5088
q22	1091	1059	1012	1012
Total cold run time: 68232 ms
Total hot run time: 54993 ms

doris-robot · 2024-08-27T10:49:05Z

TPC-DS: Total hot run time: 188848 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 984c9aac6443ba67c905f22ae1f93bf6528eafcc, data reload: false
query1	915	384	370	370
query2	6465	1862	1822	1822
query3	6652	208	230	208
query4	34247	23227	23257	23227
query5	4187	515	524	515
query6	260	166	167	166
query7	4583	302	296	296
query8	268	216	216	216
query9	8601	2530	2525	2525
query10	445	283	304	283
query11	17729	15097	15034	15034
query12	157	101	97	97
query13	1614	384	370	370
query14	8955	7036	7502	7036
query15	270	170	172	170
query16	7745	432	453	432
query17	1591	591	555	555
query18	1813	296	288	288
query19	330	144	148	144
query20	124	109	110	109
query21	214	106	101	101
query22	4602	4041	4278	4041
query23	33954	33262	33281	33262
query24	11352	2990	2983	2983
query25	668	413	436	413
query26	1242	162	161	161
query27	2605	285	284	284
query28	7499	2088	2079	2079
query29	795	423	418	418
query30	304	158	148	148
query31	1003	744	788	744
query32	104	59	62	59
query33	754	292	291	291
query34	965	510	502	502
query35	872	742	699	699
query36	1086	934	946	934
query37	162	90	83	83
query38	4002	3911	3945	3911
query39	1436	1401	1350	1350
query40	210	120	118	118
query41	78	46	45	45
query42	122	99	100	99
query43	505	454	465	454
query44	1299	759	752	752
query45	199	173	170	170
query46	1129	795	783	783
query47	1909	1812	1822	1812
query48	370	299	296	296
query49	1059	457	437	437
query50	824	419	444	419
query51	7258	7108	6957	6957
query52	102	95	95	95
query53	272	193	187	187
query54	883	462	460	460
query55	81	76	80	76
query56	288	264	254	254
query57	1184	1065	1032	1032
query58	247	237	238	237
query59	2924	2747	2816	2747
query60	305	318	274	274
query61	103	99	105	99
query62	853	676	672	672
query63	231	194	188	188
query64	4277	682	640	640
query65	3242	3175	3166	3166
query66	1355	344	356	344
query67	15748	15249	15387	15249
query68	3574	592	596	592
query69	420	287	289	287
query70	1188	1053	1123	1053
query71	357	284	336	284
query72	6435	4042	3968	3968
query73	760	335	335	335
query74	9284	8790	8971	8790
query75	3444	2737	2752	2737
query76	2217	1139	984	984
query77	523	345	348	345
query78	9638	9988	9678	9678
query79	1103	553	555	553
query80	727	523	530	523
query81	552	225	228	225
query82	244	143	137	137
query83	232	158	159	158
query84	235	77	80	77
query85	780	288	287	287
query86	325	299	293	293
query87	4409	4188	4301	4188
query88	2972	2362	2344	2344
query89	393	299	291	291
query90	1932	204	202	202
query91	142	113	117	113
query92	70	54	53	53
query93	1076	567	555	555
query94	909	316	321	316
query95	379	281	358	281
query96	602	274	272	272
query97	3194	3092	3060	3060
query98	220	206	197	197
query99	1499	1282	1282	1282
Total cold run time: 287462 ms
Total hot run time: 188848 ms

doris-robot · 2024-08-27T10:54:33Z

ClickBench: Total hot run time: 31.57 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 984c9aac6443ba67c905f22ae1f93bf6528eafcc, data reload: false
query1	0.05	0.04	0.04
query2	0.09	0.05	0.04
query3	0.22	0.05	0.05
query4	1.69	0.09	0.08
query5	0.49	0.50	0.48
query6	1.12	0.74	0.73
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.56	0.48	0.50
query10	0.53	0.55	0.54
query11	0.15	0.13	0.11
query12	0.15	0.12	0.13
query13	0.62	0.59	0.58
query14	0.77	0.79	0.77
query15	0.90	0.82	0.82
query16	0.37	0.37	0.36
query17	1.05	1.01	1.00
query18	0.21	0.19	0.20
query19	1.88	1.76	1.85
query20	0.01	0.01	0.01
query21	15.39	0.66	0.65
query22	4.40	5.72	2.74
query23	18.31	1.39	1.23
query24	2.16	0.22	0.22
query25	0.14	0.08	0.08
query26	0.28	0.18	0.19
query27	0.08	0.08	0.08
query28	13.20	1.01	1.02
query29	12.57	3.34	3.36
query30	0.25	0.06	0.05
query31	2.87	0.40	0.40
query32	3.24	0.49	0.48
query33	3.00	3.00	3.00
query34	17.01	4.41	4.38
query35	4.51	4.43	4.45
query36	0.67	0.49	0.50
query37	0.19	0.15	0.15
query38	0.14	0.14	0.14
query39	0.05	0.04	0.04
query40	0.17	0.13	0.14
query41	0.09	0.05	0.05
query42	0.06	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.77 s
Total hot run time: 31.57 s

wangbo · 2024-08-27T11:58:17Z

run buildall

github-actions · 2024-08-27T12:03:39Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-08-27T12:28:00Z

clang-tidy review says "All clean, LGTM! 👍"

wangbo · 2024-08-27T12:33:15Z

run buildall

github-actions · 2024-08-27T12:37:27Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-27T13:35:27Z

TPC-H: Total hot run time: 38078 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5cc067f58269a12467490fb8cc766cfb00950110, data reload: false
------ Round 1 ----------------------------------
q1	18076	4802	4429	4429
q2	2645	215	190	190
q3	11421	1149	1146	1146
q4	10841	762	722	722
q5	8268	3248	2889	2889
q6	229	142	142	142
q7	970	615	600	600
q8	9345	2080	2061	2061
q9	7118	6525	6540	6525
q10	7031	2160	2237	2160
q11	479	249	251	249
q12	396	227	221	221
q13	18687	3002	3027	3002
q14	278	235	232	232
q15	516	481	492	481
q16	491	400	392	392
q17	1000	633	782	633
q18	7500	6881	6866	6866
q19	1393	935	992	935
q20	685	353	339	339
q21	4022	2891	2858	2858
q22	1118	1006	1052	1006
Total cold run time: 112509 ms
Total hot run time: 38078 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4357	4320	4244	4244
q2	365	265	259	259
q3	2859	2688	2660	2660
q4	1943	1708	1658	1658
q5	5416	5387	5406	5387
q6	219	132	132	132
q7	2075	1768	1720	1720
q8	3185	3358	3352	3352
q9	8466	8503	8446	8446
q10	3430	3235	3227	3227
q11	616	499	498	498
q12	802	589	582	582
q13	10284	3066	3038	3038
q14	320	268	279	268
q15	526	496	487	487
q16	469	424	423	423
q17	1768	1496	1488	1488
q18	7879	7571	7512	7512
q19	1661	1492	1514	1492
q20	2094	1835	1825	1825
q21	5359	5316	5205	5205
q22	1141	1033	1032	1032
Total cold run time: 65234 ms
Total hot run time: 54935 ms

doris-robot · 2024-08-27T13:47:17Z

TPC-DS: Total hot run time: 188159 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5cc067f58269a12467490fb8cc766cfb00950110, data reload: false
query1	926	376	363	363
query2	6462	1888	1843	1843
query3	6646	216	218	216
query4	34350	23183	23053	23053
query5	4141	511	488	488
query6	250	163	171	163
query7	4589	291	294	291
query8	262	209	209	209
query9	8499	2494	2469	2469
query10	447	277	279	277
query11	17758	15045	14993	14993
query12	148	98	99	98
query13	1647	383	375	375
query14	9383	7340	7400	7340
query15	270	171	177	171
query16	7574	473	485	473
query17	1651	571	566	566
query18	1324	310	292	292
query19	336	149	153	149
query20	118	112	115	112
query21	216	106	104	104
query22	4434	4200	4217	4200
query23	34171	33414	33345	33345
query24	11125	2861	2865	2861
query25	654	402	411	402
query26	1269	159	162	159
query27	2558	290	282	282
query28	7366	2075	2053	2053
query29	831	438	430	430
query30	307	160	151	151
query31	981	768	789	768
query32	100	59	61	59
query33	757	290	309	290
query34	950	476	501	476
query35	881	739	717	717
query36	1110	935	936	935
query37	157	87	85	85
query38	4051	3864	3897	3864
query39	1457	1525	1400	1400
query40	266	120	119	119
query41	50	49	47	47
query42	114	102	100	100
query43	502	458	463	458
query44	1190	755	750	750
query45	202	165	165	165
query46	1098	737	745	737
query47	1866	1788	1799	1788
query48	369	303	294	294
query49	1071	449	441	441
query50	808	417	424	417
query51	7199	7126	6991	6991
query52	101	89	89	89
query53	252	186	183	183
query54	1070	474	478	474
query55	84	77	81	77
query56	280	275	261	261
query57	1194	1065	1067	1065
query58	251	246	271	246
query59	3125	2698	2786	2698
query60	310	283	283	283
query61	129	124	144	124
query62	835	668	649	649
query63	218	188	187	187
query64	5306	816	682	682
query65	3221	3150	3155	3150
query66	1275	338	340	338
query67	15739	15242	15328	15242
query68	4647	554	555	554
query69	494	266	277	266
query70	1199	1109	1118	1109
query71	425	280	280	280
query72	7441	3912	3980	3912
query73	761	333	347	333
query74	9317	8800	8793	8793
query75	3609	2654	2669	2654
query76	3218	1069	1031	1031
query77	618	318	317	317
query78	9688	9104	9145	9104
query79	1869	532	542	532
query80	996	513	498	498
query81	591	226	227	226
query82	977	136	141	136
query83	272	154	149	149
query84	244	77	89	77
query85	1293	337	289	289
query86	380	312	284	284
query87	4394	4227	4424	4227
query88	2904	2338	2329	2329
query89	395	294	283	283
query90	1844	195	191	191
query91	126	101	96	96
query92	66	51	50	50
query93	1419	552	542	542
query94	970	302	306	302
query95	354	261	265	261
query96	600	268	268	268
query97	3153	3100	3054	3054
query98	217	217	195	195
query99	1632	1240	1283	1240
Total cold run time: 294394 ms
Total hot run time: 188159 ms

doris-robot · 2024-08-27T13:52:48Z

ClickBench: Total hot run time: 30.96 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5cc067f58269a12467490fb8cc766cfb00950110, data reload: false
query1	0.05	0.04	0.04
query2	0.08	0.04	0.05
query3	0.22	0.05	0.05
query4	1.68	0.07	0.08
query5	0.49	0.49	0.49
query6	1.13	0.73	0.73
query7	0.01	0.01	0.01
query8	0.04	0.04	0.05
query9	0.55	0.48	0.50
query10	0.55	0.55	0.55
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.63	0.59	0.58
query14	0.77	0.78	0.76
query15	0.84	0.82	0.83
query16	0.36	0.37	0.37
query17	0.96	0.99	1.02
query18	0.21	0.21	0.21
query19	1.91	1.75	1.82
query20	0.01	0.01	0.01
query21	15.43	0.65	0.65
query22	4.79	6.82	2.21
query23	18.31	1.32	1.25
query24	2.11	0.25	0.22
query25	0.15	0.09	0.09
query26	0.27	0.18	0.18
query27	0.08	0.07	0.07
query28	13.21	1.02	0.99
query29	12.59	3.38	3.34
query30	0.25	0.06	0.05
query31	2.88	0.40	0.38
query32	3.28	0.49	0.48
query33	2.96	2.99	3.00
query34	16.92	4.35	4.35
query35	4.42	4.42	4.43
query36	0.65	0.47	0.51
query37	0.19	0.15	0.15
query38	0.15	0.15	0.15
query39	0.04	0.04	0.04
query40	0.16	0.13	0.14
query41	0.10	0.05	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.81 s
Total hot run time: 30.96 s

github-actions · 2024-09-06T03:38:52Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-09-06T03:38:54Z

PR approved by anyone and no changes requested.

xinyiZzz

LGTM

## Proposed changes 1 Add a step to write pid to query path's cgroup.procs, otherwrise write thread id may failed. 2 Move some cgroup v2's public file to parent class to init init them once.

## Proposed changes pick #39991 #39374 #36663

## Proposed changes 1 Add a step to write pid to query path's cgroup.procs, otherwrise write thread id may failed. 2 Move some cgroup v2's public file to parent class to init init them once.

wangbo force-pushed the 0827_fix_cg_v2 branch from 984c9aa to ed148d7Compare August 27, 2024 11:57

wangbo force-pushed the 0827_fix_cg_v2 branch from ed148d7 to 648c78dCompare August 27, 2024 12:21

Fix cgroup v2 init
5cc067f

wangbo force-pushed the 0827_fix_cg_v2 branch from 648c78d to 5cc067fCompare August 27, 2024 12:30

yiguolei approved these changes Sep 6, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 6, 2024

github-actionsBot added the reviewed label Sep 6, 2024

xinyiZzz approved these changes Sep 6, 2024
View reviewed changes

wangbo merged commit 7f744ec into apache:masterSep 6, 2024

yiguolei added the dev/2.1.x label Sep 6, 2024

dataroaring added the dev/3.0.3-merged label Oct 13, 2024

wangbo mentioned this pull request Oct 25, 2024
[pick]support cgroup v2 #42465
Merged

yiguolei pushed a commit that referenced this pull request Oct 25, 2024
[pick]support cgroup v2 (#42465)
9eef393
## Proposed changes pick #39991 #39374 #36663

wm1581066 added dev/2.1.7-merged and removed dev/2.1.x labels Oct 25, 2024

Uh oh!

Conversation

wangbo commented Aug 27, 2024

Proposed changes

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

wangbo commented Aug 27, 2024

Uh oh!

github-actionsBot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

wangbo commented Aug 27, 2024

Uh oh!

github-actionsBot commented Aug 27, 2024

Uh oh!

github-actionsBot commented Aug 27, 2024

Uh oh!

wangbo commented Aug 27, 2024

Uh oh!

github-actionsBot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

github-actionsBot commented Sep 6, 2024

Uh oh!

github-actionsBot commented Sep 6, 2024

Uh oh!

xinyiZzz left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants