[opt](cloud) optimize load performance for inverted index when pack small files by liaoxin01 · Pull Request #59011 · apache/doris

liaoxin01 · 2025-12-12T15:00:53Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #57770

Problem Summary:

When merging small files with inverted indexes, the segment close operation was synchronously waiting for inverted index files to be uploaded to S3. This blocking behavior significantly impacted the memtable flush thread performance, causing bottlenecks in the data loading pipeline.

Solution:

The solution introduces a two-phase close mechanism for inverted index file writers:

Asynchronous Close Phase: During segment close, inverted index files are closed asynchronously and the S3 upload task is submitted immediately without waiting for completion.
Wait Phase: When the load channel closes, the system waits for all pending S3 upload tasks to complete, ensuring data consistency.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-12-12T15:00:59Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

liaoxin01 · 2025-12-12T15:01:07Z

run buildall

Copilot

Pull request overview

This PR optimizes the load performance for inverted indexes when packing small files by introducing non-blocking file writer close operations and adding explicit file writer cleanup logic.

Key Changes

Changed file writer close operation to non-blocking mode (close(true)) in FSIndexOutputV2 to improve performance
Added explicit cleanup loop in SegmentFlusher::close() to ensure all index file writers' underlying file writers are properly closed

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
be/src/olap/rowset/segment_v2/inverted_index_fs_directory.cpp	Modified FSIndexOutputV2::close() to use non-blocking close (close(true)) for the underlying file writer to improve performance
be/src/olap/rowset/segment_creator.cpp	Added explicit loop to close underlying file writers after closing the index file collection, ensuring proper resource cleanup even if errors occur during the close chain

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

doris-robot · 2025-12-12T15:40:52Z

TPC-H: Total hot run time: 36035 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1edde8328e46737c7c1f954eb6c82d6c0423ab3b, data reload: false
------ Round 1 ----------------------------------
q1	18277	4163	4243	4163
q2	2032	354	237	237
q3	10212	1321	747	747
q4	10224	879	313	313
q5	7533	2116	1919	1919
q6	184	165	134	134
q7	998	836	702	702
q8	9355	1398	1114	1114
q9	6904	5350	5500	5350
q10	6837	2389	1969	1969
q11	527	317	291	291
q12	657	730	562	562
q13	17811	3667	3001	3001
q14	290	288	276	276
q15	600	522	525	522
q16	695	681	619	619
q17	683	796	537	537
q18	7506	7559	7691	7559
q19	1188	1083	643	643
q20	409	357	254	254
q21	4544	4352	4074	4074
q22	1121	1062	1049	1049
Total cold run time: 108587 ms
Total hot run time: 36035 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4315	4271	4186	4186
q2	316	403	331	331
q3	2290	3014	2454	2454
q4	1421	1959	1399	1399
q5	4560	4462	4374	4374
q6	213	176	128	128
q7	2035	1972	1885	1885
q8	2638	2517	2526	2517
q9	7631	7374	7017	7017
q10	2906	3081	2660	2660
q11	562	502	479	479
q12	642	688	559	559
q13	3233	3661	2999	2999
q14	265	289	265	265
q15	535	488	482	482
q16	609	654	607	607
q17	1102	1408	1377	1377
q18	7279	7111	7114	7111
q19	829	780	817	780
q20	1852	1957	1817	1817
q21	4678	4255	4204	4204
q22	1113	1015	980	980
Total cold run time: 51024 ms
Total hot run time: 48611 ms

doris-robot · 2025-12-12T15:51:57Z

TPC-DS: Total hot run time: 178491 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1edde8328e46737c7c1f954eb6c82d6c0423ab3b, data reload: false
query5	5425	622	452	452
query6	339	231	218	218
query7	4227	454	266	266
query8	292	246	234	234
query9	8759	2507	2550	2507
query10	583	378	310	310
query11	15337	14910	14602	14602
query12	187	119	117	117
query13	1262	517	382	382
query14	6351	3199	2964	2964
query14_1	2865	2880	2910	2880
query15	211	203	184	184
query16	881	494	478	478
query17	1143	701	621	621
query18	2704	446	387	387
query19	229	226	206	206
query20	119	113	113	113
query21	220	137	128	128
query22	4013	3827	3861	3827
query23	16680	16227	16064	16064
query23_1	15998	15988	15919	15919
query24	7348	1629	1228	1228
query24_1	1253	1189	1230	1189
query25	543	478	428	428
query26	1232	263	151	151
query27	2768	468	317	317
query28	4455	2139	2119	2119
query29	837	549	437	437
query30	315	252	216	216
query31	815	696	612	612
query32	83	69	66	66
query33	537	334	292	292
query34	898	901	531	531
query35	799	816	726	726
query36	873	920	819	819
query37	128	94	76	76
query38	2869	2913	2856	2856
query39	750	735	713	713
query39_1	685	687	713	687
query40	225	138	122	122
query41	73	63	62	62
query42	114	105	106	105
query43	439	425	411	411
query44	1329	733	735	733
query45	201	191	183	183
query46	884	972	610	610
query47	1687	1693	1617	1617
query48	316	341	250	250
query49	625	435	353	353
query50	685	301	221	221
query51	3779	3846	3772	3772
query52	107	109	98	98
query53	315	360	287	287
query54	288	260	251	251
query55	79	82	72	72
query56	300	306	294	294
query57	1145	1169	1072	1072
query58	271	257	253	253
query59	2378	2412	2391	2391
query60	309	308	304	304
query61	172	166	160	160
query62	692	689	634	634
query63	326	293	301	293
query64	4969	1432	1121	1121
query65	4018	3948	3935	3935
query66	1411	455	336	336
query67	15140	15011	14909	14909
query68	4736	1030	751	751
query69	500	359	320	320
query70	1075	1028	968	968
query71	370	314	294	294
query72	6336	5108	5097	5097
query73	639	566	304	304
query74	8950	8910	8621	8621
query75	3152	3111	2755	2755
query76	3858	1136	740	740
query77	520	390	303	303
query78	9508	9749	8922	8922
query79	1848	853	623	623
query80	1416	642	563	563
query81	542	287	236	236
query82	402	131	106	106
query83	267	261	241	241
query84	259	122	105	105
query85	936	515	480	480
query86	331	294	259	259
query87	3052	3014	2965	2965
query88	3239	2277	2268	2268
query89	457	438	389	389
query90	1987	158	163	158
query91	176	166	155	155
query92	70	69	61	61
query93	1162	933	562	562
query94	538	297	281	281
query95	554	331	305	305
query96	596	466	218	218
query97	2297	2298	2210	2210
query98	216	193	201	193
query99	1277	1300	1200	1200
Total cold run time: 258596 ms
Total hot run time: 178491 ms

doris-robot · 2025-12-12T15:57:01Z

ClickBench: Total hot run time: 27.31 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1edde8328e46737c7c1f954eb6c82d6c0423ab3b, data reload: false
query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.25	0.09	0.08
query4	1.62	0.11	0.12
query5	0.29	0.28	0.27
query6	1.18	0.63	0.62
query7	0.03	0.02	0.03
query8	0.07	0.04	0.04
query9	0.58	0.51	0.51
query10	0.55	0.56	0.56
query11	0.15	0.12	0.12
query12	0.14	0.12	0.12
query13	0.61	0.60	0.60
query14	1.00	0.97	0.98
query15	0.80	0.80	0.82
query16	0.40	0.39	0.40
query17	1.04	0.98	1.07
query18	0.23	0.22	0.21
query19	1.92	1.78	1.79
query20	0.02	0.01	0.01
query21	15.47	0.28	0.13
query22	4.71	0.05	0.06
query23	16.08	0.28	0.10
query24	0.94	0.31	0.83
query25	0.06	0.09	0.06
query26	0.15	0.13	0.14
query27	0.08	0.05	0.05
query28	4.98	1.21	1.03
query29	12.61	4.05	3.34
query30	0.29	0.14	0.14
query31	2.81	0.63	0.39
query32	3.23	0.56	0.48
query33	2.92	2.97	2.97
query34	16.96	5.15	4.50
query35	4.59	4.59	4.51
query36	0.65	0.51	0.48
query37	0.11	0.07	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.10	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 98.12 s
Total hot run time: 27.31 s

hello-stephen · 2025-12-12T16:06:01Z

BE UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.42% (18823/35236)
Line Coverage	39.16% (174018/444378)
Region Coverage	33.79% (134706/398620)
Branch Coverage	34.74% (57983/166894)

hello-stephen · 2025-12-12T17:52:39Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	72.31% (24971/34533)
Line Coverage	59.08% (262252/443873)
Region Coverage	54.08% (218135/403336)
Branch Coverage	55.60% (93278/167767)

liaoxin01 · 2025-12-18T15:49:55Z

run buildall

doris-robot · 2025-12-18T17:45:37Z

TPC-H: Total hot run time: 36456 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
------ Round 1 ----------------------------------
q1	17668	4272	4039	4039
q2	2050	358	243	243
q3	10153	1325	735	735
q4	10223	853	321	321
q5	7518	2197	1940	1940
q6	237	170	136	136
q7	1017	860	720	720
q8	9365	1456	1200	1200
q9	7256	5298	5357	5298
q10	6885	2412	1979	1979
q11	537	330	311	311
q12	685	712	564	564
q13	17807	3742	3081	3081
q14	293	292	270	270
q15	595	516	519	516
q16	695	694	636	636
q17	696	863	500	500
q18	7920	7739	7841	7739
q19	1455	1001	619	619
q20	430	375	256	256
q21	4542	4316	4312	4312
q22	1137	1121	1041	1041
Total cold run time: 109164 ms
Total hot run time: 36456 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4373	4265	4240	4240
q2	347	388	426	388
q3	2532	2802	2459	2459
q4	1427	1983	1514	1514
q5	4609	4346	4523	4346
q6	208	165	128	128
q7	2026	1928	1746	1746
q8	2707	2626	2527	2527
q9	7548	7395	7056	7056
q10	2929	3132	2680	2680
q11	572	493	469	469
q12	619	711	574	574
q13	3303	3660	3009	3009
q14	260	286	253	253
q15	528	508	485	485
q16	618	654	600	600
q17	1098	1357	1300	1300
q18	7382	7084	7082	7082
q19	856	825	794	794
q20	1860	1949	1841	1841
q21	4694	4252	4253	4252
q22	1090	1056	991	991
Total cold run time: 51586 ms
Total hot run time: 48734 ms

doris-robot · 2025-12-18T17:56:44Z

TPC-DS: Total hot run time: 178672 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
query5	4648	605	442	442
query6	322	246	233	233
query7	4231	472	276	276
query8	335	272	257	257
query9	8788	2527	2543	2527
query10	550	388	333	333
query11	15377	15248	15007	15007
query12	189	115	114	114
query13	1275	500	402	402
query14	6412	3048	2778	2778
query14_1	2684	2638	2631	2631
query15	215	199	176	176
query16	837	472	457	457
query17	1158	723	610	610
query18	2641	459	357	357
query19	237	237	208	208
query20	126	114	114	114
query21	230	145	116	116
query22	4072	3900	3881	3881
query23	16617	16225	15900	15900
query23_1	16007	16177	16239	16177
query24	7407	1654	1257	1257
query24_1	1248	1226	1254	1226
query25	589	511	466	466
query26	1255	272	168	168
query27	2754	472	313	313
query28	4472	2134	2126	2126
query29	831	579	470	470
query30	313	251	213	213
query31	824	710	631	631
query32	85	79	69	69
query33	554	345	305	305
query34	919	902	532	532
query35	779	865	739	739
query36	845	923	828	828
query37	135	96	72	72
query38	2884	2907	2844	2844
query39	738	746	748	746
query39_1	697	693	690	690
query40	221	137	118	118
query41	65	60	59	59
query42	108	109	108	108
query43	436	431	404	404
query44	1328	740	739	739
query45	195	188	187	187
query46	882	994	604	604
query47	1661	1688	1628	1628
query48	332	327	251	251
query49	624	443	338	338
query50	655	287	221	221
query51	3803	3912	3801	3801
query52	103	110	97	97
query53	323	357	289	289
query54	312	254	243	243
query55	79	77	70	70
query56	293	308	293	293
query57	1122	1183	1071	1071
query58	271	262	259	259
query59	2363	2479	2356	2356
query60	310	309	293	293
query61	162	156	157	156
query62	714	658	645	645
query63	325	292	295	292
query64	4938	1303	993	993
query65	4052	3954	3953	3953
query66	1402	437	317	317
query67	15191	14853	14700	14700
query68	7941	1002	722	722
query69	486	348	308	308
query70	1075	982	989	982
query71	361	303	276	276
query72	6138	4972	4964	4964
query73	660	591	310	310
query74	8885	8698	8572	8572
query75	3199	3146	2812	2812
query76	3969	1125	747	747
query77	521	394	298	298
query78	9385	9563	8889	8889
query79	1675	897	611	611
query80	722	646	556	556
query81	527	276	244	244
query82	385	134	104	104
query83	252	252	245	245
query84	263	116	103	103
query85	913	494	458	458
query86	394	293	308	293
query87	3099	3110	2946	2946
query88	3232	2225	2239	2225
query89	464	422	395	395
query90	2158	159	150	150
query91	170	169	140	140
query92	85	68	65	65
query93	1804	901	556	556
query94	522	299	279	279
query95	585	319	355	319
query96	586	463	212	212
query97	2265	2311	2213	2213
query98	210	190	192	190
query99	1257	1303	1238	1238
Total cold run time: 260725 ms
Total hot run time: 178672 ms

doris-robot · 2025-12-18T18:01:48Z

ClickBench: Total hot run time: 27.3 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
query1	0.06	0.04	0.04
query2	0.10	0.05	0.05
query3	0.26	0.09	0.08
query4	1.61	0.12	0.11
query5	0.28	0.26	0.25
query6	1.15	0.65	0.62
query7	0.03	0.02	0.03
query8	0.06	0.05	0.04
query9	0.58	0.50	0.50
query10	0.56	0.55	0.56
query11	0.15	0.11	0.11
query12	0.15	0.11	0.12
query13	0.63	0.62	0.60
query14	0.99	0.97	0.99
query15	0.81	0.79	0.80
query16	0.42	0.42	0.39
query17	1.03	1.02	1.01
query18	0.24	0.21	0.21
query19	1.97	1.87	1.90
query20	0.02	0.02	0.02
query21	15.43	0.29	0.14
query22	4.88	0.05	0.05
query23	16.15	0.28	0.10
query24	1.18	0.29	0.43
query25	0.12	0.06	0.06
query26	0.14	0.14	0.14
query27	0.08	0.07	0.06
query28	4.35	1.22	1.02
query29	12.60	4.00	3.28
query30	0.29	0.14	0.12
query31	2.80	0.63	0.40
query32	3.22	0.55	0.46
query33	2.97	3.02	2.97
query34	16.45	5.14	4.49
query35	4.54	4.54	4.56
query36	0.67	0.50	0.48
query37	0.11	0.07	0.07
query38	0.07	0.04	0.03
query39	0.04	0.04	0.03
query40	0.18	0.15	0.14
query41	0.08	0.03	0.03
query42	0.05	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.54 s
Total hot run time: 27.3 s

liaoxin01 · 2025-12-19T07:07:06Z

run buildall

doris-robot · 2025-12-19T09:03:50Z

TPC-H: Total hot run time: 34973 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
------ Round 1 ----------------------------------
q1	17629	4196	4057	4057
q2	2033	370	239	239
q3	10150	1348	737	737
q4	10232	942	318	318
q5	7785	2198	1915	1915
q6	231	169	137	137
q7	1000	858	716	716
q8	9380	1492	1139	1139
q9	7454	5367	5366	5366
q10	6853	2401	1961	1961
q11	551	329	304	304
q12	696	733	597	597
q13	17789	3749	3050	3050
q14	290	297	272	272
q15	600	522	524	522
q16	695	679	629	629
q17	717	781	579	579
q18	7511	7152	7070	7070
q19	1103	970	590	590
q20	411	359	247	247
q21	4297	4014	3557	3557
q22	1063	1040	971	971
Total cold run time: 108470 ms
Total hot run time: 34973 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4119	4036	4071	4036
q2	320	405	317	317
q3	2123	2700	2325	2325
q4	1361	1768	1315	1315
q5	4278	4732	4619	4619
q6	247	184	135	135
q7	2065	1986	1883	1883
q8	2672	2570	2568	2568
q9	7661	7648	7495	7495
q10	3139	3240	2868	2868
q11	624	515	505	505
q12	802	755	622	622
q13	3618	3977	3287	3287
q14	297	334	297	297
q15	562	523	501	501
q16	671	707	658	658
q17	1276	1501	1445	1445
q18	8076	7704	7579	7579
q19	901	828	916	828
q20	2073	2031	1814	1814
q21	4698	4314	4185	4185
q22	1084	1032	1015	1015
Total cold run time: 52667 ms
Total hot run time: 50297 ms

doris-robot · 2025-12-19T09:14:53Z

TPC-DS: Total hot run time: 177439 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
query5	4453	601	454	454
query6	336	260	230	230
query7	4222	466	288	288
query8	308	251	240	240
query9	8758	2529	2509	2509
query10	524	399	345	345
query11	15641	15111	14554	14554
query12	178	123	115	115
query13	1267	527	407	407
query14	6227	3036	2781	2781
query14_1	2649	2659	2697	2659
query15	215	197	180	180
query16	845	484	448	448
query17	1149	717	600	600
query18	2547	445	353	353
query19	231	234	210	210
query20	126	116	113	113
query21	223	148	120	120
query22	3922	4084	3779	3779
query23	16474	16079	15913	15913
query23_1	16041	15964	15966	15964
query24	7356	1666	1276	1276
query24_1	1258	1258	1254	1254
query25	589	493	463	463
query26	1243	278	175	175
query27	2747	460	312	312
query28	4474	2111	2115	2111
query29	844	585	478	478
query30	325	247	222	222
query31	853	713	620	620
query32	83	76	72	72
query33	580	339	289	289
query34	908	888	548	548
query35	762	827	733	733
query36	851	907	810	810
query37	123	91	78	78
query38	2825	2834	2808	2808
query39	764	774	711	711
query39_1	703	694	710	694
query40	225	134	121	121
query41	64	62	61	61
query42	108	104	105	104
query43	423	439	411	411
query44	1314	743	735	735
query45	194	196	183	183
query46	874	976	598	598
query47	1637	1703	1592	1592
query48	312	348	248	248
query49	624	440	351	351
query50	657	299	224	224
query51	3845	3906	3823	3823
query52	106	110	99	99
query53	322	349	295	295
query54	293	266	248	248
query55	77	80	71	71
query56	305	294	301	294
query57	1146	1122	1078	1078
query58	270	261	251	251
query59	2421	2482	2368	2368
query60	310	306	295	295
query61	164	161	159	159
query62	711	671	623	623
query63	328	289	303	289
query64	4970	1300	1020	1020
query65	4022	3936	3911	3911
query66	1441	431	309	309
query67	15072	15020	14703	14703
query68	7875	1008	723	723
query69	504	351	314	314
query70	1056	986	1019	986
query71	367	314	283	283
query72	5712	4829	4942	4829
query73	658	566	309	309
query74	8740	8713	8691	8691
query75	3190	3164	2781	2781
query76	3872	1119	738	738
query77	522	395	291	291
query78	9455	9551	8824	8824
query79	1071	869	606	606
query80	716	661	559	559
query81	499	263	232	232
query82	233	134	104	104
query83	265	267	240	240
query84	258	125	104	104
query85	917	514	452	452
query86	306	318	272	272
query87	3093	3060	2972	2972
query88	3127	2274	2251	2251
query89	460	425	384	384
query90	1959	166	163	163
query91	182	169	154	154
query92	71	76	67	67
query93	1007	898	562	562
query94	486	312	283	283
query95	586	323	301	301
query96	597	456	212	212
query97	2268	2303	2206	2206
query98	208	198	191	191
query99	1237	1293	1198	1198
Total cold run time: 257503 ms
Total hot run time: 177439 ms

doris-robot · 2025-12-19T09:19:54Z

ClickBench: Total hot run time: 27.13 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
query1	0.06	0.05	0.04
query2	0.10	0.05	0.05
query3	0.27	0.08	0.08
query4	1.61	0.12	0.11
query5	0.28	0.26	0.28
query6	1.18	0.64	0.62
query7	0.03	0.03	0.03
query8	0.05	0.04	0.05
query9	0.56	0.50	0.52
query10	0.57	0.56	0.55
query11	0.15	0.10	0.12
query12	0.15	0.12	0.12
query13	0.63	0.60	0.60
query14	0.99	1.00	0.98
query15	0.81	0.81	0.79
query16	0.40	0.42	0.40
query17	0.99	1.05	0.99
query18	0.24	0.22	0.21
query19	1.96	1.82	1.78
query20	0.02	0.01	0.01
query21	15.45	0.29	0.14
query22	4.74	0.05	0.04
query23	16.07	0.29	0.11
query24	1.48	0.29	0.18
query25	0.09	0.07	0.06
query26	0.16	0.13	0.13
query27	0.07	0.06	0.06
query28	2.86	1.21	1.03
query29	12.60	4.00	3.32
query30	0.28	0.13	0.11
query31	2.82	0.62	0.39
query32	3.27	0.55	0.46
query33	3.01	2.98	3.16
query34	16.92	5.17	4.48
query35	4.56	4.54	4.56
query36	0.66	0.50	0.48
query37	0.10	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.02	0.02
query40	0.18	0.14	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.64 s
Total hot run time: 27.13 s

liaoxin01 · 2025-12-19T15:42:54Z

run buildall

doris-robot · 2025-12-19T17:41:43Z

TPC-H: Total hot run time: 35195 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
------ Round 1 ----------------------------------
q1	17656	4299	4112	4112
q2	1996	355	228	228
q3	10211	1352	795	795
q4	10215	846	315	315
q5	7542	2140	1976	1976
q6	196	168	138	138
q7	995	860	714	714
q8	9358	1447	1138	1138
q9	7149	5351	5341	5341
q10	6876	2405	1977	1977
q11	517	339	296	296
q12	717	737	589	589
q13	17787	3719	3035	3035
q14	310	294	284	284
q15	599	516	521	516
q16	683	685	621	621
q17	698	769	589	589
q18	8148	7208	7042	7042
q19	1086	987	599	599
q20	404	368	250	250
q21	4335	3983	3675	3675
q22	1060	994	965	965
Total cold run time: 108538 ms
Total hot run time: 35195 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4139	4082	4086	4082
q2	327	392	339	339
q3	2134	2692	2282	2282
q4	1368	1783	1318	1318
q5	4264	4733	4618	4618
q6	245	184	141	141
q7	2123	1958	1839	1839
q8	2719	2719	2504	2504
q9	7609	7567	7429	7429
q10	3125	3252	2845	2845
q11	603	513	486	486
q12	820	834	617	617
q13	3573	4168	3242	3242
q14	317	328	299	299
q15	547	517	517	517
q16	642	869	656	656
q17	1263	1500	1459	1459
q18	7800	7491	7486	7486
q19	915	877	897	877
q20	2013	2062	1838	1838
q21	4621	4311	4246	4246
q22	1088	1058	967	967
Total cold run time: 52255 ms
Total hot run time: 50087 ms

doris-robot · 2025-12-19T17:52:44Z

TPC-DS: Total hot run time: 178442 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
query5	4843	599	454	454
query6	330	224	243	224
query7	4230	487	281	281
query8	326	257	246	246
query9	8775	2551	2536	2536
query10	502	372	333	333
query11	15238	14826	14554	14554
query12	185	115	116	115
query13	1283	540	367	367
query14	5916	2990	2789	2789
query14_1	2654	2613	2633	2613
query15	210	202	180	180
query16	839	464	427	427
query17	1078	682	585	585
query18	2409	437	337	337
query19	246	237	222	222
query20	124	115	113	113
query21	220	136	120	120
query22	3924	3932	3881	3881
query23	16550	16093	16031	16031
query23_1	16074	16184	16091	16091
query24	7355	1639	1219	1219
query24_1	1241	1237	1226	1226
query25	540	462	422	422
query26	1242	268	162	162
query27	2768	449	311	311
query28	4480	2127	2117	2117
query29	819	536	441	441
query30	318	243	220	220
query31	815	689	598	598
query32	87	66	77	66
query33	581	340	293	293
query34	907	905	544	544
query35	786	823	724	724
query36	876	939	831	831
query37	125	94	77	77
query38	2854	2791	2901	2791
query39	768	908	734	734
query39_1	693	712	709	709
query40	227	136	119	119
query41	79	62	62	62
query42	106	105	103	103
query43	444	430	398	398
query44	1351	748	748	748
query45	196	192	183	183
query46	872	987	608	608
query47	1652	1697	1618	1618
query48	319	342	257	257
query49	623	432	359	359
query50	667	300	215	215
query51	3772	3886	3949	3886
query52	109	113	102	102
query53	333	359	291	291
query54	285	265	248	248
query55	82	78	73	73
query56	290	292	294	292
query57	1175	1125	1098	1098
query58	274	245	254	245
query59	2374	2433	2278	2278
query60	322	320	288	288
query61	190	194	187	187
query62	707	660	627	627
query63	339	297	306	297
query64	5121	1461	1147	1147
query65	4009	3929	3997	3929
query66	1451	434	314	314
query67	15210	14819	14674	14674
query68	3420	1048	735	735
query69	495	357	329	329
query70	1091	1013	945	945
query71	359	304	279	279
query72	6234	5082	4992	4992
query73	692	601	330	330
query74	8853	8965	8628	8628
query75	3169	3165	2810	2810
query76	3722	1143	728	728
query77	513	387	304	304
query78	9412	9666	8824	8824
query79	1062	863	639	639
query80	1154	657	569	569
query81	546	274	231	231
query82	452	136	107	107
query83	391	257	243	243
query84	252	118	107	107
query85	950	512	461	461
query86	389	311	285	285
query87	3055	3018	2964	2964
query88	3261	2278	2253	2253
query89	462	424	393	393
query90	1997	168	160	160
query91	184	182	146	146
query92	71	70	66	66
query93	1041	906	562	562
query94	531	303	284	284
query95	598	325	313	313
query96	597	474	213	213
query97	2248	2351	2244	2244
query98	207	196	191	191
query99	1265	1300	1196	1196
Total cold run time: 254827 ms
Total hot run time: 178442 ms

doris-robot · 2025-12-19T17:57:48Z

ClickBench: Total hot run time: 27.95 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2e5391aefe6de1ae4abb8dcacfcb67c6a0c8535a, data reload: false
query1	0.06	0.05	0.05
query2	0.09	0.06	0.05
query3	0.25	0.09	0.09
query4	1.61	0.11	0.11
query5	0.27	0.24	0.26
query6	1.20	0.64	0.63
query7	0.04	0.03	0.03
query8	0.05	0.04	0.05
query9	0.57	0.52	0.51
query10	0.54	0.57	0.56
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.61	0.61
query14	1.00	0.98	0.98
query15	0.82	0.80	0.82
query16	0.39	0.40	0.43
query17	0.96	1.02	1.04
query18	0.23	0.22	0.21
query19	1.83	1.86	1.90
query20	0.02	0.02	0.01
query21	15.43	0.30	0.14
query22	4.93	0.06	0.05
query23	16.05	0.28	0.11
query24	0.95	0.75	0.59
query25	0.10	0.12	0.14
query26	0.15	0.14	0.13
query27	0.05	0.06	0.05
query28	4.96	1.22	1.03
query29	12.61	3.97	3.35
query30	0.28	0.14	0.11
query31	2.81	0.62	0.39
query32	3.22	0.55	0.46
query33	2.96	3.08	3.00
query34	16.99	5.22	4.55
query35	4.62	4.61	4.59
query36	0.66	0.50	0.50
query37	0.10	0.07	0.06
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.17	0.15	0.13
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 98.18 s
Total hot run time: 27.95 s

liaoxin01 · 2025-12-20T02:23:44Z

run buildall

doris-robot · 2025-12-20T02:59:38Z

TPC-H: Total hot run time: 36514 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 05424bc2881ae39135c065dbd8e20dc2f2998e0d, data reload: false
------ Round 1 ----------------------------------
q1	18019	4185	4040	4040
q2	2245	338	229	229
q3	10403	1317	728	728
q4	10365	836	311	311
q5	7526	2101	2049	2049
q6	181	171	136	136
q7	1031	849	739	739
q8	9389	1513	1234	1234
q9	7460	5299	5373	5299
q10	7082	2374	1972	1972
q11	596	334	295	295
q12	840	732	582	582
q13	18003	3661	2973	2973
q14	282	303	265	265
q15	591	511	512	511
q16	693	691	625	625
q17	727	839	537	537
q18	8016	7838	7805	7805
q19	1198	993	673	673
q20	402	388	272	272
q21	4543	4216	4224	4216
q22	1139	1023	1085	1023
Total cold run time: 110731 ms
Total hot run time: 36514 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4308	4222	4317	4222
q2	326	392	309	309
q3	2270	2859	2469	2469
q4	1458	1916	1415	1415
q5	4581	4526	4432	4432
q6	213	160	124	124
q7	2009	2018	1749	1749
q8	2634	2546	2543	2543
q9	7518	7292	6916	6916
q10	2873	3098	2655	2655
q11	570	489	468	468
q12	626	705	566	566
q13	3328	3618	3037	3037
q14	295	302	273	273
q15	531	501	480	480
q16	596	642	620	620
q17	1098	1303	1409	1303
q18	7507	7222	7061	7061
q19	847	811	843	811
q20	1882	1972	1786	1786
q21	4640	4175	4113	4113
q22	1087	1011	984	984
Total cold run time: 51197 ms
Total hot run time: 48336 ms

doris-robot · 2025-12-20T03:11:02Z

TPC-DS: Total hot run time: 178401 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 05424bc2881ae39135c065dbd8e20dc2f2998e0d, data reload: false
query5	4983	583	447	447
query6	342	223	221	221
query7	4233	463	277	277
query8	304	252	242	242
query9	8749	2560	2572	2560
query10	503	373	325	325
query11	15703	15038	14599	14599
query12	269	121	118	118
query13	1278	515	414	414
query14	6499	3025	2791	2791
query14_1	2639	2670	2634	2634
query15	212	196	181	181
query16	1097	481	433	433
query17	1289	739	604	604
query18	2771	452	351	351
query19	237	232	207	207
query20	122	115	115	115
query21	551	144	114	114
query22	4160	3990	4035	3990
query23	16642	16147	15982	15982
query23_1	16354	16096	16107	16096
query24	7568	1645	1218	1218
query24_1	1218	1241	1242	1241
query25	636	499	449	449
query26	1266	276	167	167
query27	2711	451	310	310
query28	4479	2137	2138	2137
query29	908	591	479	479
query30	359	242	219	219
query31	867	717	597	597
query32	84	70	78	70
query33	551	338	308	308
query34	899	901	543	543
query35	769	803	744	744
query36	860	967	819	819
query37	204	88	76	76
query38	2840	2854	2790	2790
query39	780	718	711	711
query39_1	695	688	687	687
query40	225	138	116	116
query41	64	76	66	66
query42	110	108	111	108
query43	434	439	396	396
query44	1332	739	740	739
query45	195	199	179	179
query46	865	962	609	609
query47	1708	1703	1619	1619
query48	312	323	249	249
query49	857	433	342	342
query50	657	292	216	216
query51	3783	3813	3828	3813
query52	103	110	98	98
query53	322	349	291	291
query54	282	256	247	247
query55	76	75	75	75
query56	283	304	283	283
query57	1159	1118	1065	1065
query58	267	259	247	247
query59	2378	2485	2375	2375
query60	331	311	292	292
query61	199	152	157	152
query62	730	654	633	633
query63	320	291	307	291
query64	4957	1304	980	980
query65	4035	3984	4017	3984
query66	1384	424	324	324
query67	15034	15064	14804	14804
query68	8331	1004	746	746
query69	567	341	311	311
query70	1075	985	988	985
query71	372	303	275	275
query72	6067	4891	4895	4891
query73	670	570	312	312
query74	8831	8855	8716	8716
query75	3227	3189	2804	2804
query76	4076	1115	733	733
query77	586	394	290	290
query78	9477	9614	8803	8803
query79	1421	965	603	603
query80	714	649	545	545
query81	518	267	230	230
query82	201	136	109	109
query83	262	261	241	241
query84	254	117	103	103
query85	930	519	454	454
query86	382	286	278	278
query87	3012	3061	2857	2857
query88	4405	2299	2315	2299
query89	464	421	383	383
query90	2191	156	154	154
query91	180	162	146	146
query92	79	71	65	65
query93	1654	918	561	561
query94	478	309	270	270
query95	565	323	302	302
query96	588	482	210	210
query97	2297	2276	2220	2220
query98	203	195	192	192
query99	1301	1317	1247	1247
Total cold run time: 265591 ms
Total hot run time: 178401 ms

doris-robot · 2025-12-20T03:16:05Z

ClickBench: Total hot run time: 27.32 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 05424bc2881ae39135c065dbd8e20dc2f2998e0d, data reload: false
query1	0.05	0.05	0.05
query2	0.16	0.04	0.04
query3	0.27	0.09	0.09
query4	1.62	0.11	0.12
query5	0.27	0.26	0.28
query6	1.16	0.63	0.65
query7	0.04	0.03	0.03
query8	0.06	0.04	0.04
query9	0.59	0.48	0.50
query10	0.56	0.57	0.55
query11	0.21	0.11	0.11
query12	0.15	0.11	0.12
query13	0.62	0.60	0.59
query14	0.99	1.00	0.98
query15	0.81	0.79	0.82
query16	0.38	0.39	0.42
query17	1.07	1.05	1.04
query18	0.23	0.22	0.21
query19	1.97	1.86	1.86
query20	0.02	0.01	0.02
query21	15.49	0.31	0.14
query22	4.85	0.05	0.04
query23	16.21	0.29	0.11
query24	1.99	0.71	0.29
query25	0.10	0.11	0.06
query26	0.15	0.15	0.13
query27	0.07	0.05	0.06
query28	3.19	1.23	1.03
query29	12.58	3.90	3.17
query30	0.27	0.14	0.12
query31	2.81	0.61	0.39
query32	3.23	0.54	0.46
query33	2.99	3.08	3.04
query34	16.74	5.16	4.53
query35	4.59	4.56	4.55
query36	0.67	0.50	0.49
query37	0.13	0.07	0.06
query38	0.08	0.04	0.04
query39	0.06	0.03	0.03
query40	0.17	0.14	0.14
query41	0.11	0.03	0.03
query42	0.06	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 97.81 s
Total hot run time: 27.32 s

hello-stephen · 2025-12-21T19:29:28Z

BE Regression && UT Coverage Report

Increment line coverage 87.04% (47/54) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	72.29% (25005/34592)
Line Coverage	59.04% (262614/444797)
Region Coverage	53.86% (218138/404996)
Branch Coverage	55.43% (93572/168802)

hello-stephen · 2025-12-22T03:39:48Z

BE Regression && UT Coverage Report

Increment line coverage 87.04% (47/54) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	72.30% (25010/34592)
Line Coverage	59.06% (262684/444797)
Region Coverage	53.91% (218323/404996)
Branch Coverage	55.46% (93612/168802)

liaoxin01 · 2025-12-23T02:27:28Z

run buildall

liaoxin01 · 2025-12-23T03:13:05Z

run buildall

doris-robot · 2025-12-23T05:09:13Z

TPC-H: Total hot run time: 36545 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2023b08866d9c3b448f7eb0d81089be8a26713dc, data reload: false
------ Round 1 ----------------------------------
q1	17664	4250	4133	4133
q2	2037	360	245	245
q3	10168	1348	751	751
q4	10230	880	320	320
q5	7884	2178	1939	1939
q6	231	164	132	132
q7	1033	872	713	713
q8	9343	1481	1107	1107
q9	6976	5394	5379	5379
q10	6870	2400	1990	1990
q11	537	341	299	299
q12	728	746	572	572
q13	17783	3676	3042	3042
q14	305	292	277	277
q15	594	517	508	508
q16	694	687	616	616
q17	675	752	594	594
q18	7573	7741	7922	7741
q19	1217	1103	667	667
q20	411	368	268	268
q21	4563	4268	4229	4229
q22	1162	1102	1023	1023
Total cold run time: 108678 ms
Total hot run time: 36545 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4369	4292	4297	4292
q2	308	410	429	410
q3	2398	2870	2500	2500
q4	1453	1883	1451	1451
q5	4504	4469	4573	4469
q6	208	169	126	126
q7	2042	1856	1850	1850
q8	2679	2530	2448	2448
q9	7599	7373	7005	7005
q10	2902	3091	2696	2696
q11	603	522	505	505
q12	629	685	551	551
q13	3330	3630	3015	3015
q14	264	280	255	255
q15	527	496	485	485
q16	606	661	617	617
q17	1131	1374	1367	1367
q18	7195	7225	7054	7054
q19	844	789	854	789
q20	1893	1955	1779	1779
q21	4760	4208	4172	4172
q22	1076	1030	1002	1002
Total cold run time: 51320 ms
Total hot run time: 48838 ms

doris-robot · 2025-12-23T05:20:20Z

TPC-DS: Total hot run time: 178672 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2023b08866d9c3b448f7eb0d81089be8a26713dc, data reload: false
query5	5063	607	437	437
query6	353	233	241	233
query7	4233	469	286	286
query8	312	258	246	246
query9	8763	2522	2539	2522
query10	533	372	328	328
query11	15484	14890	15060	14890
query12	185	120	117	117
query13	1268	504	389	389
query14	6323	2977	2751	2751
query14_1	2657	2641	2657	2641
query15	207	193	177	177
query16	844	462	454	454
query17	1127	720	605	605
query18	2509	443	351	351
query19	240	237	204	204
query20	123	115	119	115
query21	224	141	119	119
query22	3885	4243	3877	3877
query23	16617	16062	15942	15942
query23_1	16015	16153	16028	16028
query24	7335	1647	1223	1223
query24_1	1230	1255	1264	1255
query25	596	511	461	461
query26	1243	277	167	167
query27	2740	473	308	308
query28	4465	2141	2120	2120
query29	849	537	429	429
query30	319	241	210	210
query31	830	720	611	611
query32	78	71	70	70
query33	538	332	280	280
query34	907	899	546	546
query35	762	807	704	704
query36	878	883	835	835
query37	127	93	76	76
query38	2935	3027	2943	2943
query39	751	747	718	718
query39_1	690	710	714	710
query40	225	140	122	122
query41	74	62	63	62
query42	107	108	110	108
query43	427	430	398	398
query44	1353	776	743	743
query45	188	189	182	182
query46	894	983	621	621
query47	1694	1712	1634	1634
query48	312	335	251	251
query49	657	425	345	345
query50	673	293	212	212
query51	3844	3845	3864	3845
query52	103	106	95	95
query53	317	348	283	283
query54	278	268	257	257
query55	80	73	72	72
query56	300	289	301	289
query57	1153	1153	1075	1075
query58	281	257	254	254
query59	2383	2510	2383	2383
query60	320	308	287	287
query61	165	157	163	157
query62	753	753	653	653
query63	325	293	306	293
query64	4927	1312	1011	1011
query65	4053	3950	3934	3934
query66	1445	439	314	314
query67	15280	15143	14876	14876
query68	7495	1002	720	720
query69	496	355	316	316
query70	1077	1006	975	975
query71	359	305	276	276
query72	6084	4971	5011	4971
query73	710	599	307	307
query74	8896	8832	8690	8690
query75	3168	3144	2789	2789
query76	3907	1149	749	749
query77	529	396	296	296
query78	9371	9663	8814	8814
query79	1272	861	633	633
query80	686	634	548	548
query81	531	267	231	231
query82	202	131	113	113
query83	266	256	244	244
query84	260	134	109	109
query85	901	507	459	459
query86	392	310	281	281
query87	3219	3272	3059	3059
query88	3168	2273	2259	2259
query89	474	428	401	401
query90	2172	165	154	154
query91	183	165	152	152
query92	75	66	65	65
query93	1178	911	559	559
query94	478	309	269	269
query95	577	380	308	308
query96	598	483	207	207
query97	2226	2283	2223	2223
query98	217	191	195	191
query99	1325	1391	1254	1254
Total cold run time: 259536 ms
Total hot run time: 178672 ms

doris-robot · 2025-12-23T05:25:23Z

ClickBench: Total hot run time: 27.11 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2023b08866d9c3b448f7eb0d81089be8a26713dc, data reload: false
query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.27	0.09	0.10
query4	1.60	0.12	0.11
query5	0.26	0.26	0.27
query6	1.16	0.64	0.63
query7	0.04	0.03	0.02
query8	0.05	0.04	0.05
query9	0.56	0.50	0.51
query10	0.55	0.55	0.55
query11	0.16	0.11	0.10
query12	0.15	0.11	0.11
query13	0.62	0.60	0.60
query14	0.99	0.98	0.98
query15	0.81	0.79	0.80
query16	0.39	0.41	0.42
query17	1.06	1.02	1.04
query18	0.23	0.21	0.20
query19	1.90	1.79	1.80
query20	0.02	0.01	0.02
query21	15.44	0.30	0.14
query22	4.67	0.05	0.05
query23	15.97	0.29	0.09
query24	1.09	0.63	0.27
query25	0.11	0.08	0.05
query26	0.14	0.14	0.14
query27	0.07	0.06	0.04
query28	4.54	1.21	1.02
query29	12.59	4.10	3.20
query30	0.28	0.14	0.12
query31	2.82	0.62	0.39
query32	3.23	0.54	0.46
query33	2.98	3.08	2.97
query34	16.84	5.23	4.49
query35	4.53	4.63	4.56
query36	0.64	0.50	0.48
query37	0.10	0.06	0.07
query38	0.07	0.04	0.03
query39	0.05	0.03	0.03
query40	0.16	0.16	0.13
query41	0.08	0.03	0.03
query42	0.05	0.03	0.02
query43	0.05	0.03	0.03
Total cold run time: 97.46 s
Total hot run time: 27.11 s

hello-stephen · 2025-12-23T06:54:56Z

BE UT Coverage Report

Increment line coverage 84.91% (45/53) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.42% (18931/35436)
Line Coverage	39.31% (175530/446525)
Region Coverage	33.87% (135900/401241)
Branch Coverage	34.78% (58567/168407)

liaoxin01 · 2025-12-23T07:16:24Z

run buildall

airborne12

LGTM

github-actions · 2025-12-23T08:45:11Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-12-23T08:45:14Z

PR approved by anyone and no changes requested.

doris-robot · 2025-12-23T09:17:09Z

TPC-H: Total hot run time: 35195 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2fe3592d139faf1a158398116f5d9fb401452acd, data reload: false
------ Round 1 ----------------------------------
q1	17626	4181	4044	4044
q2	2000	349	249	249
q3	10175	1334	738	738
q4	10208	817	322	322
q5	7729	2214	1901	1901
q6	226	171	136	136
q7	1011	858	703	703
q8	9337	1484	1170	1170
q9	7376	5313	5309	5309
q10	6870	2395	1992	1992
q11	534	332	301	301
q12	705	743	556	556
q13	17823	3684	3034	3034
q14	295	299	268	268
q15	596	511	506	506
q16	697	693	625	625
q17	694	858	519	519
q18	8378	7273	7089	7089
q19	1123	975	593	593
q20	397	363	252	252
q21	4313	4021	3923	3923
q22	1016	998	965	965
Total cold run time: 109129 ms
Total hot run time: 35195 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4127	4012	4032	4012
q2	341	412	311	311
q3	2161	2667	2311	2311
q4	1321	1740	1290	1290
q5	4255	5009	4643	4643
q6	239	187	141	141
q7	2069	1961	1815	1815
q8	2680	2528	2684	2528
q9	7695	7674	7497	7497
q10	3059	3269	2843	2843
q11	617	605	554	554
q12	704	955	582	582
q13	3478	3860	3333	3333
q14	286	318	294	294
q15	565	506	548	506
q16	701	718	624	624
q17	1216	1472	1466	1466
q18	7807	7747	7481	7481
q19	868	856	876	856
q20	2030	2000	1842	1842
q21	4686	4379	4217	4217
q22	1070	1039	995	995
Total cold run time: 51975 ms
Total hot run time: 50141 ms

doris-robot · 2025-12-23T09:28:13Z

TPC-DS: Total hot run time: 179329 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2fe3592d139faf1a158398116f5d9fb401452acd, data reload: false
query5	4386	597	456	456
query6	327	224	215	215
query7	4223	459	276	276
query8	305	248	251	248
query9	8785	2502	2548	2502
query10	495	390	333	333
query11	15543	14935	15107	14935
query12	180	115	112	112
query13	1267	493	399	399
query14	5750	3022	2735	2735
query14_1	2656	2638	2634	2634
query15	216	198	176	176
query16	908	483	450	450
query17	1136	714	612	612
query18	2436	448	345	345
query19	237	245	208	208
query20	125	121	117	117
query21	233	140	131	131
query22	4144	4012	3897	3897
query23	16618	16238	15914	15914
query23_1	16178	16199	16335	16199
query24	7419	1694	1233	1233
query24_1	1255	1220	1255	1220
query25	561	469	407	407
query26	1241	267	162	162
query27	2752	466	307	307
query28	4439	2129	2127	2127
query29	805	533	457	457
query30	309	242	208	208
query31	826	686	625	625
query32	80	73	69	69
query33	534	334	278	278
query34	923	911	537	537
query35	825	810	707	707
query36	873	888	840	840
query37	132	94	78	78
query38	2952	2974	2961	2961
query39	756	726	742	726
query39_1	718	720	694	694
query40	223	132	120	120
query41	71	66	62	62
query42	118	112	104	104
query43	434	426	387	387
query44	1314	734	735	734
query45	194	188	182	182
query46	900	970	606	606
query47	1669	1733	1653	1653
query48	320	320	268	268
query49	629	428	348	348
query50	662	302	218	218
query51	3808	3743	3864	3743
query52	103	108	95	95
query53	316	349	286	286
query54	283	254	243	243
query55	78	75	71	71
query56	292	300	300	300
query57	1165	1158	1075	1075
query58	276	252	242	242
query59	2436	2490	2389	2389
query60	309	305	291	291
query61	160	155	155	155
query62	712	733	675	675
query63	329	293	303	293
query64	5080	1301	985	985
query65	4057	3976	3981	3976
query66	1502	444	315	315
query67	15631	15010	15183	15010
query68	3382	1055	731	731
query69	496	342	307	307
query70	1081	996	954	954
query71	361	298	273	273
query72	6093	4898	5021	4898
query73	695	605	311	311
query74	8917	8934	8753	8753
query75	3194	3186	2776	2776
query76	3744	1135	730	730
query77	526	401	294	294
query78	9531	9595	8930	8930
query79	1012	865	599	599
query80	754	651	542	542
query81	490	268	233	233
query82	369	133	102	102
query83	260	255	235	235
query84	257	116	104	104
query85	899	503	481	481
query86	320	288	277	277
query87	3170	3223	3120	3120
query88	3186	2315	2299	2299
query89	470	419	395	395
query90	1984	163	147	147
query91	193	185	163	163
query92	74	67	67	67
query93	985	873	567	567
query94	549	323	288	288
query95	599	351	313	313
query96	600	471	210	210
query97	2267	2279	2225	2225
query98	207	201	193	193
query99	1390	1353	1312	1312
Total cold run time: 254777 ms
Total hot run time: 179329 ms

doris-robot · 2025-12-23T09:33:18Z

ClickBench: Total hot run time: 27.45 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2fe3592d139faf1a158398116f5d9fb401452acd, data reload: false
query1	0.05	0.05	0.04
query2	0.09	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.11	0.11
query5	0.30	0.28	0.26
query6	1.16	0.64	0.65
query7	0.03	0.02	0.03
query8	0.05	0.04	0.04
query9	0.57	0.49	0.49
query10	0.54	0.56	0.55
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.62	0.60	0.61
query14	0.98	0.98	0.98
query15	0.81	0.81	0.80
query16	0.42	0.39	0.40
query17	1.07	1.04	1.05
query18	0.24	0.21	0.21
query19	1.93	1.89	1.85
query20	0.02	0.01	0.02
query21	15.43	0.31	0.14
query22	4.98	0.05	0.05
query23	16.03	0.27	0.10
query24	1.12	1.02	0.34
query25	0.12	0.13	0.11
query26	0.14	0.14	0.14
query27	0.08	0.06	0.06
query28	5.24	1.21	1.02
query29	12.67	4.08	3.23
query30	0.28	0.16	0.11
query31	2.81	0.62	0.39
query32	3.24	0.55	0.46
query33	2.98	3.01	3.04
query34	16.96	5.17	4.54
query35	4.57	4.54	4.59
query36	0.65	0.49	0.48
query37	0.10	0.06	0.07
query38	0.08	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 98.92 s
Total hot run time: 27.45 s

hello-stephen · 2025-12-23T11:00:05Z

BE UT Coverage Report

Increment line coverage 83.33% (45/54) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.39% (18947/35489)
Line Coverage	39.26% (175716/447603)
Region Coverage	33.82% (135921/401868)
Branch Coverage	34.73% (58662/168897)

…mall files (apache#59011) Related PR: apache#57770 Problem Summary: When merging small files with inverted indexes, the segment close operation was synchronously waiting for inverted index files to be uploaded to S3. This blocking behavior significantly impacted the memtable flush thread performance, causing bottlenecks in the data loading pipeline. Solution: The solution introduces a two-phase close mechanism for inverted index file writers: 1. **Asynchronous Close Phase**: During segment close, inverted index files are closed asynchronously and the S3 upload task is submitted immediately without waiting for completion. 2. **Wait Phase**: When the load channel closes, the system waits for all pending S3 upload tasks to complete, ensuring data consistency.

CopilotAI review requested due to automatic review settings December 12, 2025 15:00

liaoxin01 marked this pull request as draft December 12, 2025 15:00

Copilotstarted reviewing on behalf of liaoxin01 December 12, 2025 15:01 View session

CopilotAI reviewed Dec 12, 2025
View reviewed changes

Comment threadbe/src/olap/rowset/segment_creator.cpp Outdated

liaoxin01 force-pushed the opt_pack_for_idx_load branch from 1edde83 to 2e5391aCompare December 18, 2025 15:49

liaoxin01 force-pushed the opt_pack_for_idx_load branch from 2e5391a to 05424bcCompare December 20, 2025 02:23

liaoxin01 marked this pull request as ready for review December 20, 2025 02:28

liaoxin01 added the packed-file label Dec 22, 2025

airborne12 reviewed Dec 23, 2025
View reviewed changes

Comment threadbe/src/olap/rowset/segment_v2/index_file_writer.cpp Outdated

fix comment
2023b08

liaoxin01 force-pushed the opt_pack_for_idx_load branch from fa17d90 to 2023b08Compare December 23, 2025 03:12

airborne12 reviewed Dec 23, 2025
View reviewed changes

Comment threadbe/src/olap/rowset/beta_rowset_writer.cpp Outdated

rename and fix comment
2fe3592

airborne12 approved these changes Dec 23, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 23, 2025

github-actionsBot added the reviewed label Dec 23, 2025

luwei16 approved these changes Dec 25, 2025
View reviewed changes

liaoxin01 merged commit 3e2ed4a into apache:masterDec 25, 2025
26 of 28 checks passed

liaoxin01 deleted the opt_pack_for_idx_load branch December 25, 2025 03:45

yiguolei added the dev/4.0.3-merged label Jan 12, 2026

Uh oh!

Conversation

liaoxin01 commented Dec 12, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Dec 12, 2025

Uh oh!

liaoxin01 commented Dec 12, 2025

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

doris-robot commented Dec 12, 2025

Uh oh!

doris-robot commented Dec 12, 2025

Uh oh!

doris-robot commented Dec 12, 2025

Uh oh!

hello-stephen commented Dec 12, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Dec 12, 2025

BE Regression && UT Coverage Report

Uh oh!

liaoxin01 commented Dec 18, 2025

Uh oh!

doris-robot commented Dec 18, 2025

Uh oh!

doris-robot commented Dec 18, 2025

Uh oh!

doris-robot commented Dec 18, 2025

Uh oh!

liaoxin01 commented Dec 19, 2025

Uh oh!

doris-robot commented Dec 19, 2025

Uh oh!

doris-robot commented Dec 19, 2025

Uh oh!

doris-robot commented Dec 19, 2025

Uh oh!

liaoxin01 commented Dec 19, 2025

Uh oh!

doris-robot commented Dec 19, 2025

Uh oh!

doris-robot commented Dec 19, 2025

Uh oh!

doris-robot commented Dec 19, 2025

Uh oh!

liaoxin01 commented Dec 20, 2025

Uh oh!

doris-robot commented Dec 20, 2025

Uh oh!

doris-robot commented Dec 20, 2025

Uh oh!

doris-robot commented Dec 20, 2025

Uh oh!

hello-stephen commented Dec 21, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Dec 22, 2025

BE Regression && UT Coverage Report

Uh oh!

Uh oh!

liaoxin01 commented Dec 23, 2025

Uh oh!

liaoxin01 commented Dec 23, 2025

Uh oh!

Uh oh!

doris-robot commented Dec 23, 2025

Uh oh!

doris-robot commented Dec 23, 2025

Uh oh!

liaoxin01 commented Dec 12, 2025•
edited
Loading