Skip to content

[fix](chmod) change chmod to filesystem::permission to avoid race condition - #31032

Merged
morningman merged 1 commit into
apache:masterfrom
morningman:chmod_bug
Feb 17, 2024
Merged

[fix](chmod) change chmod to filesystem::permission to avoid race condition#31032
morningman merged 1 commit into
apache:masterfrom
morningman:chmod_bug

Conversation

@morningman

@morningmanmorningman commented Feb 16, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot

doris-robot commented Feb 16, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

github-actions[bot]
github-actionsBot reviewed Feb 16, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/io/fs/local_file_system.cpp
FILESYSTEM_M(permission_impl(path, prms));
}

Status LocalFileSystem::permission_impl(const Path& file, std::filesystem::perms prms) {

@github-actionsgithub-actionsBotFeb 16, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'permission_impl' can be made static [readability-convert-member-functions-to-static]

be/src/io/fs/local_file_system.h:100:

- Status permission_impl(const Path& file, std::filesystem::perms prms);+ static Status permission_impl(const Path& file, std::filesystem::perms prms);

@morningmanmorningman mentioned this pull request Feb 16, 2024
Merged
@morningmanmorningman mentioned this pull request Feb 16, 2024
Merged
@morningman

morningman commented Feb 16, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Feb 16, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.83% (8554/23877)
Line Coverage: 27.78% (69466/250029)
Region Coverage: 26.92% (36045/133904)
Branch Coverage: 23.70% (18432/77764)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4c5b5bbc1431cf39ad6eb45a5747cc4b214d89a2_4c5b5bbc1431cf39ad6eb45a5747cc4b214d89a2/report/index.html

@doris-robot

doris-robot commented Feb 16, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41279 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4c5b5bbc1431cf39ad6eb45a5747cc4b214d89a2, data reload: false
------ Round 1 ----------------------------------
q1	17615	4995	4807	4807
q2	2042	139	132	132
q3	10575	1042	995	995
q4	4654	962	968	962
q5	7702	3216	3257	3216
q6	202	139	130	130
q7	1279	768	761	761
q8	9223	2079	2063	2063
q9	7570	6679	6686	6679
q10	8304	2637	2627	2627
q11	416	216	210	210
q12	700	330	327	327
q13	17986	3724	3694	3694
q14	300	262	253	253
q15	603	542	495	495
q16	488	409	408	408
q17	917	876	868	868
q18	7578	6694	6559	6559
q19	1557	1484	1484	1484
q20	618	321	355	321
q21	6583	3950	3960	3950
q22	878	353	338	338
Total cold run time: 107790 ms
Total hot run time: 41279 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4801	4785	4780	4780
q2	291	189	188	188
q3	3590	3590	3593	3590
q4	2518	2542	2487	2487
q5	5745	5722	5738	5722
q6	210	126	121	121
q7	2236	1668	1663	1663
q8	2986	3048	3080	3048
q9	8659	8679	8577	8577
q10	6759	4224	4228	4224
q11	526	392	418	392
q12	783	552	555	552
q13	5985	3366	3399	3366
q14	266	231	256	231
q15	631	494	493	493
q16	481	439	429	429
q17	1650	1615	1605	1605
q18	8213	7694	7547	7547
q19	1634	1638	1640	1638
q20	2121	1820	1813	1813
q21	6622	6172	6157	6157
q22	573	523	531	523
Total cold run time: 67280 ms
Total hot run time: 59146 ms

@doris-robot

doris-robot commented Feb 16, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 180636 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4c5b5bbc1431cf39ad6eb45a5747cc4b214d89a2, data reload: false
query1	927	356	353	353
query2	6525	1766	1852	1766
query3	6699	208	206	206
query4	24472	22618	22715	22618
query5	4258	370	393	370
query6	261	174	161	161
query7	4618	292	300	292
query8	252	199	207	199
query9	8436	2783	2799	2783
query10	419	226	217	217
query11	16040	15572	15595	15572
query12	135	85	80	80
query13	1698	412	410	410
query14	9138	7452	7458	7452
query15	222	190	187	187
query16	7623	255	243	243
query17	1408	558	538	538
query18	1950	274	263	263
query19	191	145	149	145
query20	85	84	82	82
query21	189	114	116	114
query22	4973	4940	4877	4877
query23	32503	31559	31718	31559
query24	12663	3401	3356	3356
query25	614	364	355	355
query26	1699	154	153	153
query27	3057	325	305	305
query28	6645	1855	1828	1828
query29	1095	610	604	604
query30	270	136	152	136
query31	946	748	766	748
query32	95	58	53	53
query33	714	232	234	232
query34	1062	492	493	492
query35	947	805	821	805
query36	993	873	913	873
query37	139	59	63	59
query38	3454	3286	3310	3286
query39	1357	1333	1288	1288
query40	279	105	106	105
query41	36	34	34	34
query42	103	98	97	97
query43	468	439	442	439
query44	1066	686	707	686
query45	203	183	181	181
query46	1052	794	763	763
query47	1667	1549	1628	1549
query48	415	344	339	339
query49	1156	302	292	292
query50	768	384	368	368
query51	5343	5171	5149	5149
query52	104	91	84	84
query53	390	292	287	287
query54	293	217	232	217
query55	83	80	79	79
query56	215	201	189	189
query57	1045	967	936	936
query58	214	189	192	189
query59	2282	2177	2183	2177
query60	244	224	219	219
query61	83	83	90	83
query62	599	373	368	368
query63	325	284	285	284
query64	6414	3098	3114	3098
query65	3259	3263	3213	3213
query66	1348	332	317	317
query67	14590	14693	14539	14539
query68	5129	541	545	541
query69	508	363	356	356
query70	1187	1244	1179	1179
query71	385	252	252	252
query72	6371	2796	2620	2620
query73	703	309	312	309
query74	7023	6684	6585	6585
query75	3233	2555	2552	2552
query76	3290	1136	1202	1136
query77	354	242	234	234
query78	9361	8875	8748	8748
query79	960	501	491	491
query80	515	347	336	336
query81	441	209	204	204
query82	160	80	80	80
query83	138	115	118	115
query84	225	81	78	78
query85	1034	347	328	328
query86	281	294	315	294
query87	3598	3416	3424	3416
query88	2717	2293	2278	2278
query89	437	361	342	342
query90	1953	164	166	164
query91	153	127	123	123
query92	55	48	50	48
query93	957	485	468	468
query94	1140	183	179	179
query95	490	383	8447	383
query96	581	260	261	260
query97	4469	4277	4318	4277
query98	222	213	199	199
query99	1055	727	720	720
Total cold run time: 271941 ms
Total hot run time: 180636 ms

@doris-robot

doris-robot commented Feb 16, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4c5b5bbc1431cf39ad6eb45a5747cc4b214d89a2, data reload: false
query1	0.03	0.02	0.03
query2	0.06	0.03	0.03
query3	0.23	0.08	0.07
query4	1.65	0.08	0.08
query5	0.49	0.47	0.48
query6	1.35	0.63	0.61
query7	0.02	0.01	0.02
query8	0.04	0.02	0.03
query9	0.51	0.46	0.45
query10	0.48	0.49	0.49
query11	0.13	0.09	0.09
query12	0.13	0.10	0.10
query13	0.59	0.59	0.58
query14	0.76	0.79	0.78
query15	0.81	0.80	0.80
query16	0.34	0.34	0.33
query17	0.92	0.89	0.91
query18	0.17	0.17	0.17
query19	1.82	1.69	1.67
query20	0.01	0.01	0.01
query21	15.40	0.61	0.61
query22	3.41	4.13	3.11
query23	17.55	1.03	1.02
query24	2.00	0.76	0.12
query25	0.52	0.07	0.05
query26	0.17	0.13	0.15
query27	0.06	0.06	0.05
query28	12.19	0.83	0.83
query29	12.67	3.40	3.35
query30	0.52	0.48	0.44
query31	2.77	0.36	0.37
query32	3.34	0.47	0.47
query33	3.12	3.13	3.15
query34	15.36	4.47	4.47
query35	4.50	4.51	4.46
query36	1.07	0.95	0.97
query37	0.07	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.01	0.02
query40	0.17	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 105.61 s
Total hot run time: 31.5 s

@doris-robot

doris-robot commented Feb 16, 2024

Copy link
Copy Markdown

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4c5b5bbc1431cf39ad6eb45a5747cc4b214d89a2 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.3 seconds inserted 10000000 Rows, about 751K ops/s

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 17, 2024
@github-actions

github-actionsBot commented Feb 17, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Feb 17, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

morningman added a commit that referenced this pull request Feb 17, 2024
@morningman
[fix](chmod) change chmod to filesystem::permission to avoid race con…
9718114
…dition #31032 (#31034)
bp #31032
morningman added a commit that referenced this pull request Feb 17, 2024
@morningman
[fix](chmod) change chmod to filesystem::permission to avoid race con…
df6a7a9
…dition #31032 (#31033)
bp #31032
@morningman
morningman merged commit 6f1f1bd into apache:masterFeb 17, 2024
yiguolei pushed a commit that referenced this pull request Feb 18, 2024
@morningman@Doris-Extras
[fix](chmod) change chmod to filesystem::permission to avoid race con…
fc53c72
…dition (#31032)
mymeiyi pushed a commit to mymeiyi/doris that referenced this pull request Feb 19, 2024
@morningman@mymeiyi
[fix](chmod) change chmod to filesystem::permission to avoid race con…
f20708e
…dition (apache#31032)
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@morningmancaiconghui1
[fix](chmod) change chmod to filesystem::permission to avoid race con…
01e7fcc
…dition apache#31032 (apache#31034)
bp apache#31032
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@morningman@HappenLee
[fix](chmod) change chmod to filesystem::permission to avoid race con…
c925bdf
…dition (apache#31032)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@CalvinKirsCalvinKirsCalvinKirs approved these changes
@zy-kkkzy-kkkzy-kkk approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/1.2.8-mergeddev/2.0.5-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@morningman@doris-robot@CalvinKirs@zy-kkk