Skip to content

[fix](filecache) fix crash when clear file cache - #56170

Merged
dataroaring merged 5 commits into
apache:masterfrom
freemandealer:fix-sycn-clear-cache-crash
Sep 28, 2025
Merged

[fix](filecache) fix crash when clear file cache#56170
dataroaring merged 5 commits into
apache:masterfrom
freemandealer:fix-sycn-clear-cache-crash

Conversation

@freemandealer

@freemandealerfreemandealer commented Sep 17, 2025

Copy link
Copy Markdown
Member

_need_update_lru_blocks holds the reference of the block which will be cleared by api/op=clear thus causing use-after-free

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@freemandealer
[fix](filecache) fix crash when clear file cache
1ea90bb
_need_update_lru_blocks holds the reference of the block
which will be cleared by api/op=clear thus causing
use-after-free
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@Thearas

Thearas commented Sep 17, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer

freemandealer commented Sep 17, 2025

Copy link
Copy Markdown
MemberAuthor

run buildall

dataroaring
dataroaring previously approved these changes Sep 18, 2025
View reviewed changes

@dataroaringdataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@freemandealer
minor fix
818254b
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@freemandealer

freemandealer commented Sep 18, 2025

Copy link
Copy Markdown
MemberAuthor

run buildall

@doris-robot

doris-robot commented Sep 18, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34774 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 818254b3e2f15d2cb10eba76ca4e49d1b039da03, data reload: false
------ Round 1 ----------------------------------
q1	17612	5168	5076	5076
q2	1985	328	204	204
q3	10256	1339	736	736
q4	10243	1048	531	531
q5	7544	2455	2401	2401
q6	188	174	139	139
q7	929	773	620	620
q8	9351	1364	1091	1091
q9	7253	5084	5113	5084
q10	6910	2428	1970	1970
q11	489	294	286	286
q12	355	370	231	231
q13	17806	3686	3074	3074
q14	245	243	222	222
q15	570	490	501	490
q16	1016	1006	921	921
q17	603	870	354	354
q18	7516	7200	7286	7200
q19	1225	961	559	559
q20	353	360	229	229
q21	3860	2604	2360	2360
q22	1075	1032	996	996
Total cold run time: 107384 ms
Total hot run time: 34774 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5114	5051	5207	5051
q2	249	335	236	236
q3	2216	2672	2327	2327
q4	1351	1801	1327	1327
q5	4215	4538	4649	4538
q6	229	177	129	129
q7	2026	1983	1855	1855
q8	2666	2684	2610	2610
q9	7503	7494	7140	7140
q10	3145	3284	3029	3029
q11	618	533	514	514
q12	685	780	634	634
q13	3655	4007	3326	3326
q14	282	348	297	297
q15	542	507	495	495
q16	1129	1142	1051	1051
q17	1195	1567	1412	1412
q18	7878	7651	7772	7651
q19	827	802	826	802
q20	2002	2027	1869	1869
q21	5169	4566	4368	4368
q22	1084	1035	1000	1000
Total cold run time: 53780 ms
Total hot run time: 51661 ms

@doris-robot

doris-robot commented Sep 18, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 188142 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 818254b3e2f15d2cb10eba76ca4e49d1b039da03, data reload: false
query1	1060	435	410	410
query2	6555	1715	1716	1715
query3	6763	221	221	221
query4	26006	23247	23199	23199
query5	4701	642	503	503
query6	358	241	244	241
query7	4676	517	296	296
query8	321	268	248	248
query9	8690	2628	2644	2628
query10	486	344	269	269
query11	15755	15018	14809	14809
query12	186	118	119	118
query13	1673	543	416	416
query14	11768	9058	9144	9058
query15	207	185	172	172
query16	7719	660	524	524
query17	1295	748	628	628
query18	2036	424	327	327
query19	219	201	175	175
query20	125	127	120	120
query21	210	132	119	119
query22	4142	4173	4123	4123
query23	33902	32885	32853	32853
query24	8294	2381	2402	2381
query25	573	513	437	437
query26	1247	283	162	162
query27	2731	514	363	363
query28	4416	2242	2208	2208
query29	791	635	511	511
query30	293	229	201	201
query31	934	798	735	735
query32	82	74	72	72
query33	607	383	341	341
query34	801	862	511	511
query35	792	839	760	760
query36	998	1039	911	911
query37	122	109	89	89
query38	3600	3535	3457	3457
query39	1522	1509	1449	1449
query40	221	132	120	120
query41	70	67	68	67
query42	129	174	111	111
query43	494	513	488	488
query44	1318	857	833	833
query45	185	176	175	175
query46	858	1003	657	657
query47	1832	1829	1749	1749
query48	379	413	317	317
query49	774	516	421	421
query50	646	697	400	400
query51	3956	3981	3863	3863
query52	110	110	98	98
query53	234	254	197	197
query54	601	584	519	519
query55	85	83	83	83
query56	311	320	299	299
query57	1216	1177	1118	1118
query58	280	270	261	261
query59	2585	2702	2506	2506
query60	339	352	328	328
query61	163	160	167	160
query62	813	721	664	664
query63	228	194	186	186
query64	4399	1166	836	836
query65	4031	4014	3973	3973
query66	1138	447	337	337
query67	15611	15269	15231	15231
query68	9225	919	622	622
query69	483	312	270	270
query70	1345	1264	1314	1264
query71	589	348	303	303
query72	5813	5056	5073	5056
query73	743	644	362	362
query74	8859	9045	8750	8750
query75	4438	3304	2824	2824
query76	3684	1150	736	736
query77	874	392	308	308
query78	9572	9760	8863	8863
query79	2151	796	601	601
query80	714	570	487	487
query81	477	271	222	222
query82	430	157	132	132
query83	289	266	251	251
query84	306	117	95	95
query85	966	466	437	437
query86	339	332	288	288
query87	3774	3733	3686	3686
query88	3279	2200	2184	2184
query89	392	331	291	291
query90	1954	209	213	209
query91	169	174	133	133
query92	80	64	65	64
query93	1645	1001	654	654
query94	692	456	335	335
query95	383	317	306	306
query96	479	576	277	277
query97	2950	2985	2871	2871
query98	240	218	218	218
query99	1426	1407	1286	1286
Total cold run time: 278422 ms
Total hot run time: 188142 ms

@doris-robot

doris-robot commented Sep 18, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.8 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 818254b3e2f15d2cb10eba76ca4e49d1b039da03, data reload: false
query1	0.06	0.05	0.05
query2	0.09	0.05	0.06
query3	0.25	0.08	0.08
query4	1.62	0.11	0.12
query5	0.28	0.26	0.26
query6	1.21	0.65	0.64
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.62	0.53	0.53
query10	0.58	0.58	0.57
query11	0.16	0.11	0.12
query12	0.16	0.15	0.11
query13	0.62	0.62	0.61
query14	1.03	1.01	1.02
query15	0.87	0.88	0.85
query16	0.42	0.40	0.39
query17	1.04	1.05	1.09
query18	0.20	0.20	0.20
query19	1.91	1.86	1.84
query20	0.02	0.01	0.01
query21	15.47	0.93	0.59
query22	0.74	1.24	0.99
query23	14.72	1.36	0.65
query24	7.60	1.01	0.40
query25	0.46	0.13	0.14
query26	0.50	0.16	0.14
query27	0.07	0.06	0.05
query28	9.38	0.89	0.43
query29	12.61	3.86	3.26
query30	0.27	0.13	0.12
query31	2.84	0.60	0.37
query32	3.31	0.56	0.47
query33	3.13	3.03	3.09
query34	16.11	5.50	4.83
query35	4.95	4.92	4.92
query36	0.70	0.51	0.50
query37	0.11	0.07	0.08
query38	0.07	0.05	0.05
query39	0.04	0.03	0.03
query40	0.19	0.15	0.15
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.03
Total cold run time: 104.67 s
Total hot run time: 29.8 s

@doris-robot

doris-robot commented Sep 18, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 77.78% (7/9) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage52.31% (17548/33544)
Line Coverage37.45% (159021/424626)
Region Coverage32.03% (121287/378708)
Branch Coverage33.37% (53197/159413)

gavinchou
gavinchou reviewed Sep 18, 2025
View reviewed changes
Comment threadbe/src/io/cache/block_file_cache.cpp Outdated
@freemandealer

freemandealer commented Sep 22, 2025

Copy link
Copy Markdown
MemberAuthor

run buildall

@doris-robot

doris-robot commented Sep 22, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 100.00% (7/7) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage52.40% (17623/33631)
Line Coverage37.62% (160014/425308)
Region Coverage32.15% (121875/379057)
Branch Coverage33.49% (53414/159486)

@hello-stephen

hello-stephen commented Sep 22, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 71.43% (5/7) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.14% (23389/32876)
Line Coverage57.53% (244174/424438)
Region Coverage52.91% (203340/384305)
Branch Coverage54.53% (87374/160243)

@freemandealer

freemandealer commented Sep 23, 2025

Copy link
Copy Markdown
MemberAuthor

run buildall

@hello-stephen

hello-stephen commented Sep 23, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 87.50% (7/8) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage52.40% (17627/33638)
Line Coverage37.62% (160007/425338)
Region Coverage32.12% (121776/379135)
Branch Coverage33.49% (53412/159508)

@hello-stephen

hello-stephen commented Sep 23, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 62.50% (5/8) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.14% (23390/32877)
Line Coverage57.54% (244208/424449)
Region Coverage52.89% (203249/384309)
Branch Coverage54.50% (87332/160245)

@github-actions

github-actionsBot commented Sep 26, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Sep 26, 2025
@github-actions

github-actionsBot commented Sep 26, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Sep 28, 2025
View reviewed changes

@dataroaringdataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring
dataroaring merged commit ceea052 into apache:masterSep 28, 2025
29 of 31 checks passed
github-actionsBot pushed a commit that referenced this pull request Sep 28, 2025
@freemandealerYour Name
[fix](filecache) fix crash when clear file cache (#56170)
5528b4a
_need_update_lru_blocks holds the reference of the block which will be
cleared by api/op=clear thus causing use-after-free
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@github-actionsgithub-actionsBot mentioned this pull request Sep 28, 2025
Merged
github-actionsBot pushed a commit that referenced this pull request Sep 28, 2025
@freemandealerYour Name
[fix](filecache) fix crash when clear file cache (#56170)
cbe748d
_need_update_lru_blocks holds the reference of the block which will be
cleared by api/op=clear thus causing use-after-free
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@github-actionsgithub-actionsBot mentioned this pull request Sep 28, 2025
Merged
yiguolei pushed a commit that referenced this pull request Sep 29, 2025
@github-actions@freemandealer
branch-4.0: [fix](filecache) fix crash when clear file cache#56170(#…
4a02d1d
…56585)
Cherry-picked from #56170
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
Co-authored-by: zhengyu <zhangzhengyu@selectdb.com>
freemandealer added a commit to freemandealer/doris that referenced this pull request Oct 9, 2025
@freemandealer
[fix](filecache) fix crash when clear file cache (apache#56170)
267054d
_need_update_lru_blocks holds the reference of the block which will be
cleared by api/op=clear thus causing use-after-free
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
morrySnow pushed a commit that referenced this pull request Oct 10, 2025
@github-actions@freemandealer
branch-3.1: [fix](filecache) fix crash when clear file cache#56170(#…
ed2ed5b
…56584)
Cherry-picked from #56170
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
Co-authored-by: zhengyu <zhangzhengyu@selectdb.com>
@morrySnowmorrySnow mentioned this pull request Oct 13, 2025
Merged
morrySnow pushed a commit that referenced this pull request Oct 13, 2025
@freemandealer
branch-3.1: [fix](filecache) fix crash when clear file cache#56170(#…
a29ce5d
…56761)
picked from #56170
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchougavinchougavinchou approved these changes
@dataroaringdataroaringdataroaring approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/3.1.2-mergeddev/4.0.0-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@freemandealer@Thearas@doris-robot@hello-stephen@gavinchou@dataroaring@yiguolei@morrySnow