[fix](packed-file) Fix packed file cache cleanup issue by liaoxin01 · Pull Request #59892 · apache/doris

liaoxin01 · 2026-01-14T14:09:27Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Previously, file cache used packed file path as cache key, which caused cache entries to be orphaned when stale rowsets were cleaned up (cleanup uses segment path as key).

This fix moves the cache layer from inner reader to PackedFileReader wrapper level, ensuring:

Cache key = hash(segment_path.filename()) - matches cleanup key
Cache size = segment size - correct boundary
Each segment has independent cache entry - no interference

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-01-14T14:09:54Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

liaoxin01 · 2026-01-14T14:11:17Z

run buildall

Copilot

Pull request overview

This PR fixes a packed file cache cleanup issue by moving the cache layer from the inner reader to the PackedFileReader wrapper level. Previously, file cache used the packed file path as the cache key, which caused cache entries to become orphaned when stale rowsets were cleaned up (cleanup uses segment path as key).

Changes:

Modified cache handling in PackedFileSystem::open_file_impl() to disable caching at the inner reader level and wrap PackedFileReader with CachedRemoteFileReader using segment path and size
Added asynchronous cache write functions (do_write_to_file_cache and write_small_file_to_cache_async) to write small files to cache during packed file creation
Changed packed file writer options to disable write_file_cache flag, as individual small files are now cached separately
Updated logging from LOG(INFO) to VLOG_DEBUG for less verbose output

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
be/src/io/fs/packed_file_system.cpp	Moved cache layer to wrap PackedFileReader instead of inner reader; ensures cache key uses segment path and cache size uses segment size
be/src/io/fs/packed_file_manager.cpp	Added async cache writing for small files during packed file creation; disabled write_file_cache for packed files themselves

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

liaoxin01 · 2026-01-14T14:15:16Z

run buildall

doris-robot · 2026-01-14T14:50:31Z

TPC-H: Total hot run time: 30806 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 96a95e40d598113244cacae6c3638e2822c1201b, data reload: false
------ Round 1 ----------------------------------
q1	17660	4166	4022	4022
q2	2173	344	240	240
q3	10190	1278	698	698
q4	10229	821	301	301
q5	7544	2066	1778	1778
q6	191	165	136	136
q7	908	789	663	663
q8	9280	1376	1041	1041
q9	4964	4538	4508	4508
q10	7015	1770	1380	1380
q11	582	296	263	263
q12	701	736	561	561
q13	17794	3754	3013	3013
q14	288	294	265	265
q15	582	512	490	490
q16	673	647	631	631
q17	666	790	474	474
q18	6763	6432	6304	6304
q19	1360	972	618	618
q20	389	351	245	245
q21	2964	2388	2196	2196
q22	1039	1016	979	979
Total cold run time: 103955 ms
Total hot run time: 30806 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4138	4045	4083	4045
q2	321	380	304	304
q3	2028	2549	2184	2184
q4	1301	1727	1315	1315
q5	4109	3957	4090	3957
q6	212	168	128	128
q7	1846	1831	1675	1675
q8	2893	2373	2413	2373
q9	7193	7166	7134	7134
q10	2459	2775	2403	2403
q11	539	482	482	482
q12	731	745	631	631
q13	3608	4036	3396	3396
q14	303	306	278	278
q15	564	544	510	510
q16	661	668	637	637
q17	1125	1411	1340	1340
q18	8002	7902	7717	7717
q19	898	817	799	799
q20	2014	2057	1886	1886
q21	4994	4570	4213	4213
q22	1115	1072	1018	1018
Total cold run time: 51054 ms
Total hot run time: 48425 ms

doris-robot · 2026-01-14T15:01:23Z

TPC-DS: Total hot run time: 172662 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 96a95e40d598113244cacae6c3638e2822c1201b, data reload: false
query5	4409	589	470	470
query6	323	222	204	204
query7	4220	449	246	246
query8	359	241	230	230
query9	8669	2855	2805	2805
query10	509	378	317	317
query11	15173	15131	14750	14750
query12	164	121	116	116
query13	1243	471	402	402
query14	6237	2958	2733	2733
query14_1	2642	2626	2688	2626
query15	199	188	171	171
query16	3167	508	464	464
query17	1090	666	582	582
query18	2197	466	319	319
query19	221	213	185	185
query20	117	113	108	108
query21	296	135	110	110
query22	3924	4032	3750	3750
query23	16109	15582	15324	15324
query23_1	15469	15654	15443	15443
query24	7178	1502	1135	1135
query24_1	1160	1163	1144	1144
query25	524	432	386	386
query26	1247	253	149	149
query27	2741	436	265	265
query28	4567	2102	2100	2100
query29	726	509	419	419
query30	351	231	205	205
query31	801	612	552	552
query32	80	72	71	71
query33	529	352	302	302
query34	873	856	519	519
query35	748	759	673	673
query36	864	908	825	825
query37	174	99	83	83
query38	2683	2669	2577	2577
query39	796	737	716	716
query39_1	714	721	697	697
query40	220	127	116	116
query41	65	63	60	60
query42	101	100	98	98
query43	431	422	409	409
query44	1297	724	722	722
query45	184	183	174	174
query46	830	935	573	573
query47	1383	1498	1409	1409
query48	304	327	246	246
query49	596	411	339	339
query50	601	277	204	204
query51	3728	3835	3790	3790
query52	101	104	94	94
query53	287	324	273	273
query54	286	270	255	255
query55	81	76	75	75
query56	301	307	305	305
query57	999	1004	949	949
query58	270	278	250	250
query59	1931	2027	2062	2027
query60	334	327	343	327
query61	149	153	147	147
query62	393	352	301	301
query63	291	263	267	263
query64	4566	1293	988	988
query65	3819	3821	3727	3727
query66	1358	398	317	317
query67	15224	15128	15198	15128
query68	2743	994	730	730
query69	448	359	315	315
query70	992	893	922	893
query71	331	311	280	280
query72	5511	3388	3546	3388
query73	600	714	309	309
query74	8722	8736	8448	8448
query75	2828	2789	2435	2435
query76	1954	1046	638	638
query77	354	365	302	302
query78	9808	9852	9204	9204
query79	946	905	566	566
query80	1126	560	477	477
query81	543	261	230	230
query82	414	145	107	107
query83	395	253	237	237
query84	262	114	97	97
query85	896	522	460	460
query86	397	320	291	291
query87	2875	2823	2807	2807
query88	3453	2567	2556	2556
query89	380	354	317	317
query90	1991	170	153	153
query91	166	155	139	139
query92	79	73	69	69
query93	960	921	520	520
query94	624	321	297	297
query95	571	320	311	311
query96	651	492	224	224
query97	2346	2377	2291	2291
query98	227	200	192	192
query99	612	590	590	590
Total cold run time: 247139 ms
Total hot run time: 172662 ms

doris-robot · 2026-01-14T15:06:27Z

ClickBench: Total hot run time: 26.64 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 96a95e40d598113244cacae6c3638e2822c1201b, data reload: false
query1	0.05	0.05	0.05
query2	0.36	0.05	0.04
query3	0.66	0.09	0.09
query4	1.65	0.12	0.11
query5	0.27	0.25	0.24
query6	1.19	0.67	0.65
query7	0.04	0.03	0.02
query8	0.05	0.03	0.04
query9	0.65	0.50	0.50
query10	0.56	0.55	0.55
query11	0.16	0.10	0.10
query12	0.15	0.10	0.10
query13	0.63	0.59	0.58
query14	0.94	0.94	0.95
query15	0.78	0.77	0.78
query16	0.40	0.38	0.39
query17	1.02	1.05	1.02
query18	0.23	0.21	0.21
query19	1.85	1.81	1.77
query20	0.03	0.01	0.01
query21	15.55	0.28	0.14
query22	5.22	0.05	0.05
query23	15.95	0.28	0.10
query24	3.23	1.03	0.36
query25	0.14	0.06	0.05
query26	0.14	0.14	0.13
query27	0.05	0.07	0.06
query28	3.94	1.07	0.89
query29	12.56	3.90	3.14
query30	0.28	0.14	0.11
query31	3.01	0.64	0.39
query32	3.24	0.56	0.46
query33	2.98	2.94	3.04
query34	15.78	5.10	4.39
query35	4.41	4.44	4.49
query36	0.66	0.50	0.48
query37	0.10	0.07	0.07
query38	0.08	0.04	0.03
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.09	0.03	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 99.38 s
Total hot run time: 26.64 s

dataroaring · 2026-01-14T22:12:22Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

Previously, file cache used packed file path as cache key, which caused cache entries to be orphaned when stale rowsets were cleaned up (cleanup uses segment path as key). This fix moves the cache layer from inner reader to PackedFileReader wrapper level, ensuring: 1. Cache key = hash(segment_path.filename()) - matches cleanup key 2. Cache size = segment size - correct boundary 3. Each segment has independent cache entry - no interference

dataroaring · 2026-01-15T05:56:33Z

run buildall

doris-robot · 2026-01-15T06:31:45Z

TPC-H: Total hot run time: 31129 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2cdcfe2698ad44928610af733152a0fc7df35695, data reload: false
------ Round 1 ----------------------------------
q1	17624	4193	4052	4052
q2	2004	349	238	238
q3	10179	1291	710	710
q4	10227	862	308	308
q5	7526	2072	1840	1840
q6	191	177	144	144
q7	939	780	663	663
q8	9290	1385	1112	1112
q9	4921	4626	4548	4548
q10	6769	1804	1382	1382
q11	533	292	289	289
q12	713	737	596	596
q13	17791	3830	3086	3086
q14	286	292	273	273
q15	596	510	502	502
q16	702	656	640	640
q17	650	823	450	450
q18	6876	6372	6292	6292
q19	1108	961	606	606
q20	388	363	248	248
q21	2982	2481	2201	2201
q22	1054	1015	949	949
Total cold run time: 103349 ms
Total hot run time: 31129 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4111	4060	4000	4000
q2	324	397	311	311
q3	2061	2606	2214	2214
q4	1304	1746	1311	1311
q5	4096	3964	4122	3964
q6	205	171	129	129
q7	1892	1832	1690	1690
q8	2841	2421	2403	2403
q9	7175	7186	7098	7098
q10	2623	2665	2256	2256
q11	577	551	468	468
q12	719	779	628	628
q13	3646	4154	3461	3461
q14	294	291	270	270
q15	550	499	487	487
q16	631	662	679	662
q17	1153	1283	1362	1283
q18	8318	7932	7843	7843
q19	884	835	875	835
q20	2023	2286	1956	1956
q21	4923	4405	4245	4245
q22	1181	1071	1011	1011
Total cold run time: 51531 ms
Total hot run time: 48525 ms

doris-robot · 2026-01-15T06:42:31Z

TPC-DS: Total hot run time: 174229 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2cdcfe2698ad44928610af733152a0fc7df35695, data reload: false
query5	4858	619	507	507
query6	324	224	215	215
query7	4242	460	251	251
query8	327	240	269	240
query9	8769	2872	2861	2861
query10	498	367	317	317
query11	15177	15344	14848	14848
query12	177	110	110	110
query13	1251	506	378	378
query14	6413	3056	2850	2850
query14_1	2692	2652	2684	2652
query15	203	195	175	175
query16	995	486	489	486
query17	1078	680	583	583
query18	2627	436	338	338
query19	219	226	232	226
query20	117	119	110	110
query21	216	134	112	112
query22	3841	3956	3790	3790
query23	15889	15620	15424	15424
query23_1	15447	15491	15380	15380
query24	7115	1536	1193	1193
query24_1	1180	1165	1183	1165
query25	519	425	408	408
query26	1241	273	157	157
query27	2747	441	271	271
query28	4556	2132	2116	2116
query29	745	518	429	429
query30	307	228	205	205
query31	812	650	569	569
query32	98	73	73	73
query33	518	344	313	313
query34	890	892	553	553
query35	724	751	682	682
query36	880	876	840	840
query37	133	99	88	88
query38	2720	2743	2690	2690
query39	770	773	736	736
query39_1	700	718	733	718
query40	219	136	123	123
query41	67	61	61	61
query42	108	104	103	103
query43	423	475	406	406
query44	1324	724	732	724
query45	187	189	182	182
query46	814	938	565	565
query47	1394	1479	1409	1409
query48	326	324	238	238
query49	599	422	337	337
query50	612	281	200	200
query51	3893	3794	3808	3794
query52	102	107	97	97
query53	282	316	273	273
query54	288	279	264	264
query55	87	82	80	80
query56	318	328	326	326
query57	1015	1034	957	957
query58	281	270	265	265
query59	2122	2191	1958	1958
query60	349	348	316	316
query61	191	186	184	184
query62	399	357	322	322
query63	298	270	275	270
query64	4994	1391	1105	1105
query65	3770	3711	3762	3711
query66	1420	410	320	320
query67	15564	15521	15441	15441
query68	2448	1082	752	752
query69	451	362	324	324
query70	1019	939	863	863
query71	329	307	296	296
query72	5443	3244	3483	3244
query73	606	714	308	308
query74	8713	8766	8504	8504
query75	2753	2823	2483	2483
query76	2275	1059	653	653
query77	363	376	291	291
query78	9756	9914	9156	9156
query79	1060	908	566	566
query80	1251	588	493	493
query81	532	271	234	234
query82	1352	150	108	108
query83	344	261	242	242
query84	258	114	95	95
query85	942	504	455	455
query86	381	290	286	286
query87	2893	2943	2821	2821
query88	3508	2620	2580	2580
query89	380	358	323	323
query90	1940	176	165	165
query91	175	165	142	142
query92	77	75	69	69
query93	909	883	535	535
query94	591	316	282	282
query95	566	394	328	328
query96	651	504	227	227
query97	2331	2398	2327	2327
query98	215	206	195	195
query99	596	601	513	513
Total cold run time: 247893 ms
Total hot run time: 174229 ms

doris-robot · 2026-01-15T06:47:33Z

ClickBench: Total hot run time: 26.97 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2cdcfe2698ad44928610af733152a0fc7df35695, data reload: false
query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.26	0.09	0.09
query4	1.60	0.11	0.11
query5	0.27	0.27	0.27
query6	1.14	0.65	0.66
query7	0.04	0.03	0.03
query8	0.06	0.04	0.04
query9	0.58	0.51	0.50
query10	0.55	0.56	0.54
query11	0.14	0.09	0.10
query12	0.14	0.11	0.11
query13	0.61	0.58	0.59
query14	0.96	0.94	0.96
query15	0.79	0.78	0.79
query16	0.40	0.39	0.38
query17	1.04	1.05	1.01
query18	0.23	0.22	0.22
query19	1.93	1.86	1.88
query20	0.02	0.01	0.01
query21	15.43	0.26	0.13
query22	5.26	0.05	0.04
query23	15.81	0.27	0.11
query24	1.43	0.72	0.41
query25	0.13	0.05	0.08
query26	0.13	0.13	0.14
query27	0.05	0.08	0.05
query28	4.71	1.09	0.88
query29	12.55	3.91	3.14
query30	0.27	0.13	0.12
query31	2.81	0.62	0.39
query32	3.24	0.55	0.46
query33	2.96	3.08	3.14
query34	16.14	5.15	4.38
query35	4.45	4.43	4.48
query36	0.66	0.49	0.50
query37	0.11	0.07	0.07
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.18	0.14	0.14
query41	0.09	0.03	0.04
query42	0.04	0.02	0.02
query43	0.05	0.04	0.04
Total cold run time: 97.52 s
Total hot run time: 26.97 s

liaoxin01 · 2026-01-16T07:55:44Z

run buildall

doris-robot · 2026-01-16T08:49:48Z

TPC-H: Total hot run time: 32236 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 26ccf1d8096ec03208de911a49b43ef310cf63c4, data reload: false
------ Round 1 ----------------------------------
q1	17613	4189	4094	4094
q2	2133	370	257	257
q3	10043	1328	707	707
q4	10245	895	318	318
q5	7507	2097	1836	1836
q6	197	172	140	140
q7	983	798	657	657
q8	9269	1451	1194	1194
q9	4980	4656	4639	4639
q10	6767	1805	1401	1401
q11	538	325	296	296
q12	722	755	590	590
q13	17769	3884	3173	3173
q14	303	313	272	272
q15	578	510	509	509
q16	681	722	646	646
q17	738	863	507	507
q18	6697	6422	7135	6422
q19	1145	1118	693	693
q20	470	427	285	285
q21	3398	2779	2552	2552
q22	1201	1134	1048	1048
Total cold run time: 103977 ms
Total hot run time: 32236 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4642	4269	4282	4269
q2	347	448	347	347
q3	2380	2872	2446	2446
q4	1479	1933	1502	1502
q5	4528	4294	4466	4294
q6	240	170	131	131
q7	1974	2067	1762	1762
q8	2766	2412	2405	2405
q9	7230	7273	7170	7170
q10	2596	2743	2372	2372
q11	547	470	458	458
q12	777	808	653	653
q13	3634	4027	3180	3180
q14	283	291	276	276
q15	519	479	481	479
q16	646	678	617	617
q17	1128	1265	1273	1265
q18	7656	7506	7431	7431
q19	912	847	852	847
q20	1898	1959	1846	1846
q21	4554	4371	4184	4184
q22	1109	1033	975	975
Total cold run time: 51845 ms
Total hot run time: 48909 ms

doris-robot · 2026-01-16T09:00:47Z

TPC-DS: Total hot run time: 178301 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 26ccf1d8096ec03208de911a49b43ef310cf63c4, data reload: false
query5	4448	646	520	520
query6	343	228	216	216
query7	4263	490	297	297
query8	370	277	271	271
query9	8743	3265	3232	3232
query10	528	423	358	358
query11	15245	15348	14890	14890
query12	183	119	119	119
query13	1271	540	428	428
query14	6395	3095	2890	2890
query14_1	2803	2718	2744	2718
query15	207	205	179	179
query16	1006	492	482	482
query17	1108	680	598	598
query18	2532	460	360	360
query19	231	231	201	201
query20	127	119	122	119
query21	225	145	128	128
query22	3973	4207	3996	3996
query23	16183	15631	15444	15444
query23_1	15487	15554	15587	15554
query24	7263	1589	1238	1238
query24_1	1212	1241	1253	1241
query25	555	472	443	443
query26	1245	280	164	164
query27	2744	461	311	311
query28	4505	2452	2438	2438
query29	782	543	466	466
query30	327	257	219	219
query31	806	660	598	598
query32	94	81	81	81
query33	552	375	327	327
query34	951	908	578	578
query35	728	789	689	689
query36	958	964	921	921
query37	145	102	91	91
query38	2837	2755	2655	2655
query39	797	765	770	765
query39_1	721	727	750	727
query40	228	146	128	128
query41	77	73	72	72
query42	112	107	111	107
query43	524	482	459	459
query44	1425	835	829	829
query45	188	189	180	180
query46	888	980	616	616
query47	1526	1546	1426	1426
query48	332	336	260	260
query49	621	440	349	349
query50	706	282	220	220
query51	3936	3860	3782	3782
query52	115	115	100	100
query53	308	329	278	278
query54	304	278	273	273
query55	90	84	89	84
query56	339	323	317	317
query57	1084	1079	978	978
query58	285	277	268	268
query59	2272	2328	2212	2212
query60	367	363	331	331
query61	152	161	159	159
query62	410	384	322	322
query63	299	270	278	270
query64	4842	1326	986	986
query65	3863	3732	3807	3732
query66	1448	431	323	323
query67	15803	15754	15586	15586
query68	2393	1179	865	865
query69	466	384	335	335
query70	1114	1033	1023	1023
query71	339	333	310	310
query72	5248	3221	3290	3221
query73	668	767	345	345
query74	8854	8820	8661	8661
query75	2756	2816	2445	2445
query76	2291	1109	701	701
query77	387	389	317	317
query78	9902	10173	9225	9225
query79	1403	950	634	634
query80	1336	614	510	510
query81	569	278	240	240
query82	1048	154	116	116
query83	364	286	263	263
query84	257	124	99	99
query85	1036	494	440	440
query86	441	319	320	319
query87	2914	2908	2814	2814
query88	3919	2913	2878	2878
query89	409	368	337	337
query90	1975	195	170	170
query91	176	174	150	150
query92	87	75	78	75
query93	1208	970	602	602
query94	659	319	294	294
query95	602	419	336	336
query96	717	531	258	258
query97	2375	2392	2325	2325
query98	237	212	212	212
query99	617	626	556	556
Total cold run time: 251541 ms
Total hot run time: 178301 ms

doris-robot · 2026-01-16T09:05:50Z

ClickBench: Total hot run time: 27.3 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 26ccf1d8096ec03208de911a49b43ef310cf63c4, data reload: false
query1	0.06	0.05	0.05
query2	0.11	0.05	0.05
query3	0.27	0.09	0.08
query4	1.61	0.12	0.11
query5	0.29	0.26	0.25
query6	1.15	0.68	0.66
query7	0.03	0.03	0.02
query8	0.06	0.04	0.04
query9	0.59	0.53	0.52
query10	0.56	0.56	0.56
query11	0.16	0.10	0.11
query12	0.15	0.11	0.12
query13	0.61	0.60	0.58
query14	0.97	0.95	0.96
query15	0.80	0.79	0.79
query16	0.40	0.40	0.43
query17	1.08	1.03	1.10
query18	0.25	0.22	0.21
query19	1.94	1.78	1.87
query20	0.02	0.01	0.01
query21	15.44	0.30	0.16
query22	5.18	0.06	0.05
query23	15.91	0.30	0.10
query24	1.52	0.30	0.29
query25	0.08	0.09	0.09
query26	0.15	0.14	0.14
query27	0.08	0.06	0.06
query28	3.31	1.10	0.90
query29	12.52	4.14	3.34
query30	0.28	0.14	0.13
query31	2.82	0.66	0.41
query32	3.25	0.57	0.47
query33	3.11	3.07	3.08
query34	16.21	5.12	4.43
query35	4.48	4.50	4.54
query36	0.66	0.51	0.50
query37	0.11	0.06	0.06
query38	0.08	0.04	0.04
query39	0.05	0.04	0.03
query40	0.17	0.15	0.14
query41	0.10	0.04	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 96.72 s
Total hot run time: 27.3 s

liaoxin01 · 2026-01-16T10:07:11Z

run buildall

liaoxin01 · 2026-01-16T10:15:32Z

run buildall

doris-robot · 2026-01-16T11:55:07Z

TPC-H: Total hot run time: 31795 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 11c2daefed300b79be10a46d2f18fa5bd51b04c0, data reload: false
------ Round 1 ----------------------------------
q1	17626	4266	4033	4033
q2	2049	368	236	236
q3	10152	1317	711	711
q4	10216	841	296	296
q5	7517	2083	1909	1909
q6	203	181	139	139
q7	910	789	654	654
q8	9251	1406	1182	1182
q9	5182	4656	4632	4632
q10	6766	1788	1385	1385
q11	552	293	281	281
q12	724	722	613	613
q13	17768	3855	3038	3038
q14	296	308	294	294
q15	592	515	509	509
q16	707	671	644	644
q17	643	733	589	589
q18	6758	6548	6533	6533
q19	1450	983	614	614
q20	389	373	249	249
q21	2994	2476	2264	2264
q22	1071	1010	990	990
Total cold run time: 103816 ms
Total hot run time: 31795 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4159	4064	4041	4041
q2	315	398	309	309
q3	2073	2573	2187	2187
q4	1335	1749	1293	1293
q5	4089	3999	4105	3999
q6	253	168	126	126
q7	1881	1787	1924	1787
q8	2641	2479	2483	2479
q9	7326	7306	7277	7277
q10	2612	2751	2263	2263
q11	583	471	466	466
q12	747	762	632	632
q13	3544	4154	3410	3410
q14	299	320	285	285
q15	535	505	493	493
q16	662	693	652	652
q17	1167	1341	1315	1315
q18	7893	8148	7872	7872
q19	901	855	822	822
q20	2013	2176	2048	2048
q21	4654	4435	4152	4152
q22	1097	1034	1007	1007
Total cold run time: 50779 ms
Total hot run time: 48915 ms

doris-robot · 2026-01-16T12:05:53Z

TPC-DS: Total hot run time: 174472 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 11c2daefed300b79be10a46d2f18fa5bd51b04c0, data reload: false
query5	4395	609	484	484
query6	351	243	202	202
query7	4223	464	259	259
query8	347	250	236	236
query9	8701	2873	2909	2873
query10	539	388	324	324
query11	15294	15095	14906	14906
query12	186	114	109	109
query13	1293	490	377	377
query14	6160	3020	2749	2749
query14_1	2662	2612	2623	2612
query15	214	188	171	171
query16	988	479	455	455
query17	1062	633	540	540
query18	2421	423	324	324
query19	210	215	187	187
query20	121	117	113	113
query21	221	139	118	118
query22	4020	4128	4131	4128
query23	16063	15710	15551	15551
query23_1	15516	15461	15504	15461
query24	7171	1547	1189	1189
query24_1	1151	1163	1186	1163
query25	535	457	407	407
query26	1231	269	162	162
query27	2760	440	273	273
query28	4612	2183	2171	2171
query29	795	550	432	432
query30	317	246	207	207
query31	817	653	589	589
query32	84	76	71	71
query33	551	361	318	318
query34	923	903	535	535
query35	717	745	666	666
query36	875	930	851	851
query37	144	100	91	91
query38	2740	2674	2714	2674
query39	766	775	739	739
query39_1	709	719	743	719
query40	218	136	116	116
query41	64	61	61	61
query42	111	100	103	100
query43	443	426	411	411
query44	1313	746	745	745
query45	183	194	178	178
query46	824	939	567	567
query47	1375	1531	1436	1436
query48	307	315	234	234
query49	595	421	336	336
query50	611	269	204	204
query51	3750	3840	3749	3749
query52	104	109	100	100
query53	284	325	271	271
query54	291	263	260	260
query55	80	79	78	78
query56	302	308	299	299
query57	1033	1038	945	945
query58	273	257	253	253
query59	2044	2141	2036	2036
query60	335	341	312	312
query61	144	141	150	141
query62	389	353	328	328
query63	299	264	259	259
query64	4991	1246	979	979
query65	3844	3795	3693	3693
query66	1452	431	307	307
query67	15691	15532	15380	15380
query68	2367	1106	777	777
query69	437	355	325	325
query70	1027	956	884	884
query71	315	320	290	290
query72	5339	3204	3268	3204
query73	620	733	325	325
query74	8732	8775	8613	8613
query75	2724	2825	2450	2450
query76	2285	1058	656	656
query77	352	376	306	306
query78	9670	9784	9204	9204
query79	1056	910	594	594
query80	1271	584	476	476
query81	548	262	230	230
query82	1013	140	109	109
query83	332	258	236	236
query84	255	118	92	92
query85	896	518	427	427
query86	405	303	286	286
query87	2856	2916	2724	2724
query88	3463	2591	2639	2591
query89	391	347	330	330
query90	1945	169	164	164
query91	169	161	136	136
query92	76	71	71	71
query93	979	892	538	538
query94	642	337	283	283
query95	575	390	324	324
query96	645	500	243	243
query97	2368	2384	2321	2321
query98	216	208	197	197
query99	658	602	512	512
Total cold run time: 246935 ms
Total hot run time: 174472 ms

doris-robot · 2026-01-16T12:10:48Z

ClickBench: Total hot run time: 26.67 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 11c2daefed300b79be10a46d2f18fa5bd51b04c0, data reload: false
query1	0.05	0.05	0.05
query2	0.10	0.04	0.05
query3	0.26	0.09	0.08
query4	1.61	0.11	0.11
query5	0.27	0.26	0.25
query6	1.15	0.67	0.65
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.56	0.51	0.51
query10	0.56	0.55	0.56
query11	0.14	0.09	0.10
query12	0.14	0.11	0.10
query13	0.62	0.59	0.59
query14	0.94	0.95	0.95
query15	0.80	0.76	0.77
query16	0.40	0.40	0.39
query17	1.00	1.02	1.01
query18	0.22	0.21	0.21
query19	1.93	1.84	1.88
query20	0.02	0.02	0.01
query21	15.43	0.28	0.14
query22	5.10	0.05	0.05
query23	15.98	0.28	0.09
query24	1.50	0.68	0.19
query25	0.08	0.06	0.06
query26	0.15	0.15	0.14
query27	0.07	0.06	0.04
query28	4.04	1.09	0.89
query29	12.54	3.89	3.11
query30	0.28	0.14	0.13
query31	2.83	0.62	0.40
query32	3.23	0.55	0.47
query33	3.06	3.08	3.03
query34	15.86	5.14	4.41
query35	4.47	4.48	4.44
query36	0.65	0.50	0.49
query37	0.10	0.06	0.07
query38	0.08	0.04	0.03
query39	0.04	0.02	0.02
query40	0.18	0.15	0.13
query41	0.08	0.03	0.03
query42	0.05	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.69 s
Total hot run time: 26.67 s

hello-stephen · 2026-01-16T13:21:09Z

BE UT Coverage Report

Increment line coverage 77.01% (67/87) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.39% (19029/36322)
Line Coverage	35.71% (176313/493800)
Region Coverage	32.21% (136418/423542)
Branch Coverage	33.13% (59009/178101)

hello-stephen · 2026-01-16T14:42:08Z

BE Regression && UT Coverage Report

Increment line coverage 79.31% (69/87) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.39% (25414/35600)
Line Coverage	53.94% (266062/493218)
Region Coverage	51.51% (220469/428044)
Branch Coverage	53.01% (94816/178862)

github-actions · 2026-01-17T09:01:21Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-01-17T09:01:24Z

PR approved by anyone and no changes requested.

Previously, file cache used packed file path as cache key, which caused cache entries to be orphaned when stale rowsets were cleaned up (cleanup uses segment path as key). This fix moves the cache layer from inner reader to PackedFileReader wrapper level, ensuring: 1. Cache key = hash(segment_path.filename()) - matches cleanup key 2. Cache size = segment size - correct boundary 3. Each segment has independent cache entry - no interference

…pache#59892) Previously, file cache used packed file path as cache key, which caused cache entries to be orphaned when stale rowsets were cleaned up (cleanup uses segment path as key). This fix moves the cache layer from inner reader to PackedFileReader wrapper level, ensuring: 1. Cache key = hash(segment_path.filename()) - matches cleanup key 2. Cache size = segment size - correct boundary 3. Each segment has independent cache entry - no interference

…59892) (#60052) Cherry-picked from #59892

…pache#59892) (apache#60052) Cherry-picked from apache#59892

…ting small file to file cache (#65701) ### What problem does this PR solve? Issue Number: DORIS-27148 When a packed small file is written to the file cache, `do_write_to_file_cache()` did not set `ctx.tablet_id`, so the cache block meta was persisted with `tablet_id=0`. As a result: 1. `information_schema.file_cache_info` shows these blocks under `tablet_id=0`, and they cannot be queried by the real tablet id. 2. TTL blocks are not registered to `BlockFileCacheTtlMgr` (registration is skipped when `context.tablet_id == 0`), so the TTL expiration check cannot evict them by tablet. This was introduced when picking #59892 to this branch: the pick dropped the `ctx.tablet_id = tablet_id;` line that exists on master. This PR restores it, aligning with master. Note: existing cache meta already persisted with `tablet_id=0` will not be rewritten automatically; it is corrected only when the block is evicted and cached again. ### Release note Fix that file cache blocks written via the packed-file path were recorded with tablet_id=0 in information_schema.file_cache_info. ### Check List (For Author) - Test - [x] Manual test (add detailed scripts or steps below) - Verified per DORIS-27148: write small segments that go through the packed-file path, then query `information_schema.file_cache_info` by the real tablet ids and confirm blocks are no longer recorded under tablet_id=0. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label

liaoxin01 requested a review from gavinchou as a code owner January 14, 2026 14:09

CopilotAI review requested due to automatic review settings January 14, 2026 14:09

liaoxin01 requested a review from dataroaring as a code owner January 14, 2026 14:09

Copilotstarted reviewing on behalf of liaoxin01 January 14, 2026 14:10 View session

CopilotAI reviewed Jan 14, 2026
View reviewed changes

Comment threadbe/src/io/fs/packed_file_manager.cpp Outdated

liaoxin01 force-pushed the fix/packed-file-cache-cleanup branch from 20954cd to 96a95e4Compare January 14, 2026 14:14

dataroaring force-pushed the fix/packed-file-cache-cleanup branch from 96a95e4 to 2cdcfe2Compare January 15, 2026 05:56

dataroaring added the dev/4.0.x label Jan 15, 2026

gavinchou reviewed Jan 15, 2026
View reviewed changes

Comment threadbe/src/io/fs/packed_file_manager.cpp

gavinchou reviewed Jan 15, 2026
View reviewed changes

Comment threadbe/src/io/fs/packed_file_manager.cpp Outdated

gavinchou reviewed Jan 15, 2026
View reviewed changes

Comment threadbe/src/io/fs/packed_file_manager.cpp

fix comment and ut
26ccf1d

fix ut
380d014

fix
11c2dae

luwei16 approved these changes Jan 17, 2026
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Jan 17, 2026

gavinchou approved these changes Jan 19, 2026
View reviewed changes

liaoxin01 merged commit d80e119 into apache:masterJan 19, 2026
27 of 29 checks passed

liaoxin01 deleted the fix/packed-file-cache-cleanup branch January 19, 2026 08:03

github-actionsBot mentioned this pull request Jan 19, 2026
branch-4.0: [fix](packed-file) Fix packed file cache cleanup issue #59892 #60024
Closed

liaoxin01 mentioned this pull request Jan 20, 2026
branch-4.0: [fix](packed-file) Fix packed file cache cleanup issue (#59892) #60052
Merged

yiguolei pushed a commit that referenced this pull request Jan 20, 2026
branch-4.0: [fix](packed-file) Fix packed file cache cleanup issue (#…
4c3a586
…59892) (#60052) Cherry-picked from #59892

yiguolei added dev/4.0.3-merged and removed dev/4.0.x labels Jan 20, 2026

ybtsdst pushed a commit to ybtsdst/doris that referenced this pull request Feb 27, 2026
branch-4.0: [fix](packed-file) Fix packed file cache cleanup issue (a…
850d6eb
…pache#59892) (apache#60052) Cherry-picked from apache#59892

liaoxin01 mentioned this pull request Jul 16, 2026
branch-4.1: [fix](packed-file) set tablet_id in CacheContext when writing small file to file cache #65701
Merged
5 tasks

Uh oh!

Conversation

liaoxin01 commented Jan 14, 2026

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jan 14, 2026

Uh oh!

liaoxin01 commented Jan 14, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

liaoxin01 commented Jan 14, 2026

Uh oh!

doris-robot commented Jan 14, 2026

Uh oh!

doris-robot commented Jan 14, 2026

Uh oh!

doris-robot commented Jan 14, 2026

Uh oh!

dataroaring commented Jan 14, 2026

Code review

Uh oh!

dataroaring commented Jan 15, 2026

Uh oh!

doris-robot commented Jan 15, 2026

Uh oh!

doris-robot commented Jan 15, 2026

Uh oh!

doris-robot commented Jan 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

liaoxin01 commented Jan 16, 2026

Uh oh!

doris-robot commented Jan 16, 2026

Uh oh!

doris-robot commented Jan 16, 2026

Uh oh!

doris-robot commented Jan 16, 2026

Uh oh!

liaoxin01 commented Jan 16, 2026

Uh oh!

liaoxin01 commented Jan 16, 2026

Uh oh!

doris-robot commented Jan 16, 2026

Uh oh!

doris-robot commented Jan 16, 2026

Uh oh!

doris-robot commented Jan 16, 2026

Uh oh!

hello-stephen commented Jan 16, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Jan 16, 2026

BE Regression && UT Coverage Report

Uh oh!

github-actionsBot commented Jan 17, 2026

Uh oh!

github-actionsBot commented Jan 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants