[fix](load_stream) close brpc stream after load stream is closed by dataroaring · Pull Request #56120 · apache/doris

dataroaring · 2025-09-16T15:38:47Z

Otherwise, auto partition on multi bes may lead to segment num mismatch problem.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Otherwise, auto partition on multi bes may lead to segment num mismatch problem.

Thearas · 2025-09-16T15:38:52Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

dataroaring · 2025-09-16T15:38:56Z

run buildall

kaijchen

LGTM

github-actions · 2025-09-16T15:58:44Z

PR approved by anyone and no changes requested.

doris-robot · 2025-09-16T16:13:05Z

TPC-H: Total hot run time: 34484 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e666fa3eba92991c048c36150e5d86fc0e7cdee2, data reload: false
------ Round 1 ----------------------------------
q1	17608	5205	5057	5057
q2	2028	315	204	204
q3	10293	1272	699	699
q4	10238	1001	520	520
q5	7563	2425	2360	2360
q6	180	171	136	136
q7	924	774	641	641
q8	9349	1468	1089	1089
q9	6868	5134	5164	5134
q10	6968	2381	1968	1968
q11	489	314	284	284
q12	347	372	236	236
q13	17780	3609	3049	3049
q14	239	233	213	213
q15	563	508	484	484
q16	993	979	943	943
q17	618	858	355	355
q18	7429	7157	6973	6973
q19	1380	954	560	560
q20	348	331	224	224
q21	3635	2651	2374	2374
q22	1063	1040	981	981
Total cold run time: 106903 ms
Total hot run time: 34484 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5239	5088	5128	5088
q2	274	329	231	231
q3	2153	2672	2299	2299
q4	1328	1767	1310	1310
q5	4218	4392	4566	4392
q6	222	169	136	136
q7	1986	1989	1893	1893
q8	2660	2572	2594	2572
q9	7268	7332	7316	7316
q10	3075	3309	2878	2878
q11	600	542	520	520
q12	738	793	614	614
q13	3484	3915	3355	3355
q14	278	305	282	282
q15	530	490	477	477
q16	1067	1138	1085	1085
q17	1207	1510	1373	1373
q18	7919	7718	7481	7481
q19	771	754	791	754
q20	1892	1942	1799	1799
q21	4675	4390	4285	4285
q22	1060	1013	1032	1013
Total cold run time: 52644 ms
Total hot run time: 51153 ms

doris-robot · 2025-09-16T16:24:51Z

TPC-DS: Total hot run time: 188712 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e666fa3eba92991c048c36150e5d86fc0e7cdee2, data reload: false
query1	1086	439	408	408
query2	6578	1684	1696	1684
query3	6751	226	217	217
query4	25979	23829	23583	23583
query5	4481	672	483	483
query6	360	254	244	244
query7	5402	522	304	304
query8	429	264	263	263
query9	8843	2694	2695	2694
query10	491	362	304	304
query11	15408	15006	14808	14808
query12	174	117	110	110
query13	1684	583	435	435
query14	11015	9192	9200	9192
query15	215	195	188	188
query16	7749	682	490	490
query17	1665	775	652	652
query18	2062	490	340	340
query19	197	194	168	168
query20	128	121	120	120
query21	211	131	109	109
query22	4095	4268	4044	4044
query23	33820	33200	32916	32916
query24	8534	2402	2407	2402
query25	581	513	445	445
query26	1107	272	156	156
query27	2677	511	360	360
query28	4337	2244	2199	2199
query29	782	595	482	482
query30	300	227	197	197
query31	909	780	717	717
query32	84	69	77	69
query33	590	368	349	349
query34	836	864	517	517
query35	816	871	754	754
query36	981	1034	919	919
query37	122	107	81	81
query38	3514	3512	3467	3467
query39	1502	1431	1460	1431
query40	223	134	124	124
query41	65	61	66	61
query42	128	122	115	115
query43	512	477	459	459
query44	1323	859	846	846
query45	184	175	171	171
query46	862	1011	641	641
query47	1763	1827	1728	1728
query48	407	419	312	312
query49	783	506	405	405
query50	669	674	406	406
query51	3897	3951	3901	3901
query52	110	107	102	102
query53	234	254	200	200
query54	595	591	529	529
query55	97	89	81	81
query56	306	314	335	314
query57	1217	1195	1126	1126
query58	272	267	264	264
query59	2572	2658	2554	2554
query60	337	337	331	331
query61	183	161	159	159
query62	815	738	652	652
query63	233	189	194	189
query64	4328	1154	821	821
query65	4022	3946	3941	3941
query66	1028	434	335	335
query67	15516	15420	15245	15245
query68	8921	919	584	584
query69	496	321	287	287
query70	1349	1324	1280	1280
query71	562	338	308	308
query72	6130	5021	5131	5021
query73	734	659	367	367
query74	8910	9098	8673	8673
query75	4283	3298	2864	2864
query76	3715	1175	728	728
query77	820	415	328	328
query78	9536	9824	8911	8911
query79	2847	819	605	605
query80	711	644	496	496
query81	490	273	223	223
query82	480	160	134	134
query83	298	272	254	254
query84	313	109	98	98
query85	881	458	420	420
query86	339	304	320	304
query87	3731	3762	3605	3605
query88	2894	2234	2248	2234
query89	443	337	299	299
query90	1967	213	217	213
query91	165	169	135	135
query92	82	67	62	62
query93	2028	986	651	651
query94	700	442	334	334
query95	402	312	307	307
query96	488	595	282	282
query97	2945	2963	2874	2874
query98	239	215	212	212
query99	1440	1429	1306	1306
Total cold run time: 278515 ms
Total hot run time: 188712 ms

doris-robot · 2025-09-16T16:30:20Z

ClickBench: Total hot run time: 29.67 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e666fa3eba92991c048c36150e5d86fc0e7cdee2, data reload: false
query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.25	0.08	0.08
query4	1.61	0.11	0.12
query5	0.27	0.28	0.26
query6	1.16	0.66	0.65
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.62	0.53	0.52
query10	0.58	0.57	0.59
query11	0.16	0.12	0.11
query12	0.16	0.12	0.12
query13	0.62	0.62	0.63
query14	1.07	1.01	1.06
query15	0.89	0.83	0.87
query16	0.43	0.39	0.40
query17	1.09	1.04	1.07
query18	0.22	0.20	0.20
query19	1.97	1.88	1.90
query20	0.02	0.01	0.02
query21	15.42	0.93	0.58
query22	0.77	1.06	0.63
query23	15.11	1.42	0.68
query24	7.37	0.98	0.39
query25	0.47	0.21	0.10
query26	0.61	0.17	0.14
query27	0.07	0.06	0.06
query28	9.09	0.95	0.46
query29	12.55	4.04	3.32
query30	0.28	0.13	0.11
query31	2.85	0.59	0.39
query32	3.25	0.57	0.47
query33	3.09	3.12	3.13
query34	16.05	5.51	4.86
query35	4.90	4.94	4.88
query36	0.70	0.52	0.53
query37	0.10	0.07	0.08
query38	0.06	0.05	0.04
query39	0.03	0.03	0.03
query40	0.18	0.16	0.16
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 104.46 s
Total hot run time: 29.67 s

dataroaring · 2025-09-17T01:07:45Z

run buildall

doris-robot · 2025-09-17T01:40:02Z

TPC-H: Total hot run time: 34733 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a0610c16c2e2e10216777391b8417de4132f305a, data reload: false
------ Round 1 ----------------------------------
q1	17637	5184	5104	5104
q2	2037	333	225	225
q3	10250	1334	712	712
q4	10228	1019	538	538
q5	7542	2335	2343	2335
q6	184	167	137	137
q7	933	798	646	646
q8	9355	1329	1088	1088
q9	6868	5229	5164	5164
q10	6960	2401	1971	1971
q11	515	314	283	283
q12	363	365	231	231
q13	17772	3684	3051	3051
q14	237	246	215	215
q15	579	498	491	491
q16	989	992	942	942
q17	633	882	366	366
q18	7324	7109	7055	7055
q19	1401	963	598	598
q20	362	333	224	224
q21	3764	3218	2380	2380
q22	1061	1031	977	977
Total cold run time: 106994 ms
Total hot run time: 34733 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5226	5132	5057	5057
q2	261	331	243	243
q3	2188	2656	2278	2278
q4	1326	1756	1327	1327
q5	4215	4440	4464	4440
q6	213	188	148	148
q7	2070	2013	1915	1915
q8	2632	2573	2566	2566
q9	7485	7228	7214	7214
q10	3210	3372	2877	2877
q11	610	546	507	507
q12	690	855	655	655
q13	3449	3989	3318	3318
q14	301	315	295	295
q15	540	494	485	485
q16	1051	1149	1101	1101
q17	1192	1677	1324	1324
q18	7905	7750	7651	7651
q19	806	882	1092	882
q20	1990	1953	1843	1843
q21	4815	4415	4294	4294
q22	1097	1066	981	981
Total cold run time: 53272 ms
Total hot run time: 51401 ms

doris-robot · 2025-09-17T01:51:47Z

TPC-DS: Total hot run time: 188343 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a0610c16c2e2e10216777391b8417de4132f305a, data reload: false
query1	1100	441	412	412
query2	6557	1752	1669	1669
query3	6753	221	225	221
query4	26338	23663	22921	22921
query5	4413	646	447	447
query6	326	247	222	222
query7	4650	521	296	296
query8	289	250	238	238
query9	8663	2610	2601	2601
query10	488	326	281	281
query11	15631	15097	14826	14826
query12	161	117	113	113
query13	1684	565	433	433
query14	10423	9242	9237	9237
query15	201	196	171	171
query16	7378	671	525	525
query17	1255	737	634	634
query18	1999	437	337	337
query19	217	217	170	170
query20	135	127	127	127
query21	220	140	119	119
query22	4299	4281	4000	4000
query23	33747	32788	32869	32788
query24	8553	2450	2424	2424
query25	587	539	477	477
query26	1248	282	165	165
query27	2726	506	363	363
query28	4429	2224	2196	2196
query29	838	621	505	505
query30	293	234	203	203
query31	938	826	749	749
query32	82	76	73	73
query33	587	397	344	344
query34	793	869	541	541
query35	821	843	754	754
query36	986	1024	936	936
query37	127	111	89	89
query38	3523	3460	3528	3460
query39	1706	1451	1412	1412
query40	228	141	128	128
query41	71	131	65	65
query42	127	117	108	108
query43	501	496	480	480
query44	1319	858	835	835
query45	185	179	171	171
query46	847	1030	667	667
query47	1803	1811	1745	1745
query48	396	410	313	313
query49	795	508	418	418
query50	650	686	417	417
query51	3984	3911	4079	3911
query52	116	111	98	98
query53	244	263	196	196
query54	601	584	533	533
query55	85	81	82	81
query56	314	305	312	305
query57	1209	1213	1147	1147
query58	293	273	266	266
query59	2651	2701	2507	2507
query60	346	337	329	329
query61	186	161	161	161
query62	808	762	671	671
query63	235	193	195	193
query64	4529	1223	841	841
query65	4049	4010	3969	3969
query66	1168	433	339	339
query67	15741	15368	15236	15236
query68	8996	923	587	587
query69	492	313	281	281
query70	1406	1326	1282	1282
query71	556	337	306	306
query72	5893	5010	5149	5010
query73	739	636	369	369
query74	8864	9127	8915	8915
query75	4421	3320	2936	2936
query76	3733	1182	745	745
query77	823	387	319	319
query78	9648	9819	8867	8867
query79	2462	834	589	589
query80	693	585	507	507
query81	483	262	231	231
query82	470	166	130	130
query83	301	261	242	242
query84	311	117	103	103
query85	887	463	425	425
query86	343	324	312	312
query87	3743	3782	3672	3672
query88	3096	2214	2189	2189
query89	420	337	306	306
query90	1932	216	211	211
query91	166	168	181	168
query92	81	69	63	63
query93	1322	1004	641	641
query94	700	435	306	306
query95	397	321	314	314
query96	476	573	271	271
query97	2981	2962	2883	2883
query98	255	222	215	215
query99	1453	1424	1339	1339
Total cold run time: 277188 ms
Total hot run time: 188343 ms

doris-robot · 2025-09-17T01:57:19Z

ClickBench: Total hot run time: 30.2 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a0610c16c2e2e10216777391b8417de4132f305a, data reload: false
query1	0.05	0.05	0.05
query2	0.10	0.05	0.06
query3	0.26	0.09	0.09
query4	1.60	0.11	0.11
query5	0.28	0.26	0.26
query6	1.16	0.64	0.65
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.62	0.52	0.53
query10	0.57	0.58	0.58
query11	0.16	0.12	0.11
query12	0.15	0.11	0.11
query13	0.62	0.63	0.62
query14	1.01	1.03	1.06
query15	0.86	0.85	0.90
query16	0.40	0.39	0.40
query17	1.04	1.06	1.08
query18	0.22	0.20	0.20
query19	1.93	1.89	1.86
query20	0.02	0.01	0.02
query21	15.42	0.96	0.62
query22	0.76	1.25	0.65
query23	14.98	1.40	0.63
query24	6.77	1.60	0.90
query25	0.54	0.12	0.06
query26	0.56	0.16	0.15
query27	0.06	0.06	0.05
query28	10.08	0.88	0.43
query29	12.56	3.95	3.25
query30	0.28	0.13	0.12
query31	2.83	0.63	0.38
query32	3.25	0.58	0.49
query33	3.16	3.05	3.14
query34	16.14	5.49	4.90
query35	5.01	5.01	5.03
query36	0.73	0.53	0.52
query37	0.11	0.08	0.07
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.14	0.15
query41	0.09	0.04	0.03
query42	0.04	0.03	0.04
query43	0.05	0.04	0.04
Total cold run time: 104.84 s
Total hot run time: 30.2 s

dataroaring · 2025-09-17T01:58:24Z

run buildall

hello-stephen · 2025-09-17T02:17:14Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	84.01% (1534/1826)
Line Coverage	68.10% (27551/40458)
Region Coverage	68.52% (13556/19783)
Branch Coverage	58.82% (7246/12318)

hello-stephen · 2025-09-17T03:03:45Z

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

dataroaring · 2025-09-17T03:53:04Z

run buildall

hello-stephen · 2025-09-17T04:15:19Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	84.02% (1541/1834)
Line Coverage	68.03% (27588/40553)
Region Coverage	68.41% (13567/19832)
Branch Coverage	58.67% (7245/12348)

hello-stephen · 2025-09-17T04:57:10Z

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

dataroaring · 2025-09-17T06:52:15Z

run buildall

hello-stephen · 2025-09-17T07:19:38Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	84.02% (1541/1834)
Line Coverage	68.06% (27600/40553)
Region Coverage	68.48% (13581/19832)
Branch Coverage	58.71% (7249/12348)

doris-robot · 2025-09-17T07:46:58Z

TPC-H: Total hot run time: 35360 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 93e129b97be1a3c5e7c66aa8406596a7036a97af, data reload: false
------ Round 1 ----------------------------------
q1	16788	5280	5151	5151
q2	2008	321	214	214
q3	9977	1343	749	749
q4	10119	1040	557	557
q5	7755	2449	2427	2427
q6	192	169	145	145
q7	936	783	646	646
q8	9366	1394	1164	1164
q9	7028	5226	5225	5225
q10	6901	2436	1995	1995
q11	486	307	290	290
q12	362	371	236	236
q13	17770	3681	3082	3082
q14	245	257	217	217
q15	564	496	498	496
q16	1023	1011	950	950
q17	600	882	378	378
q18	7575	7248	7187	7187
q19	1108	967	566	566
q20	356	361	235	235
q21	3801	3261	2482	2482
q22	1075	1039	968	968
Total cold run time: 106035 ms
Total hot run time: 35360 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5232	5164	5246	5164
q2	258	343	228	228
q3	2179	2697	2304	2304
q4	1364	1773	1346	1346
q5	4254	4590	4631	4590
q6	225	182	143	143
q7	2104	1974	1829	1829
q8	2669	2595	2583	2583
q9	7592	7472	7436	7436
q10	3173	3303	2896	2896
q11	603	522	527	522
q12	698	774	637	637
q13	3596	4033	3395	3395
q14	305	309	308	308
q15	533	487	476	476
q16	1083	1118	1187	1118
q17	1187	1629	1389	1389
q18	7842	7797	7762	7762
q19	822	868	937	868
q20	2050	2088	1966	1966
q21	4795	4483	4331	4331
q22	1083	1011	994	994
Total cold run time: 53647 ms
Total hot run time: 52285 ms

doris-robot · 2025-09-17T07:58:44Z

TPC-DS: Total hot run time: 189408 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 93e129b97be1a3c5e7c66aa8406596a7036a97af, data reload: false
query1	1087	428	404	404
query2	6574	1731	1726	1726
query3	6755	222	221	221
query4	26469	23797	22979	22979
query5	4380	657	509	509
query6	345	251	243	243
query7	4655	506	302	302
query8	315	274	262	262
query9	8670	2646	2648	2646
query10	506	345	328	328
query11	15231	15174	14826	14826
query12	179	127	121	121
query13	1704	572	449	449
query14	10412	9542	9420	9420
query15	222	201	195	195
query16	7836	678	552	552
query17	1251	775	675	675
query18	2063	480	330	330
query19	219	205	195	195
query20	129	127	126	126
query21	221	149	127	127
query22	4120	4098	4093	4093
query23	33981	33002	33061	33002
query24	8316	2450	2435	2435
query25	562	515	459	459
query26	1236	276	162	162
query27	2732	523	361	361
query28	4245	2275	2238	2238
query29	782	620	494	494
query30	293	239	211	211
query31	909	823	720	720
query32	84	73	80	73
query33	586	392	342	342
query34	827	883	526	526
query35	798	843	767	767
query36	987	1035	901	901
query37	117	110	87	87
query38	3560	3517	3487	3487
query39	1476	1437	1418	1418
query40	220	130	128	128
query41	69	63	63	63
query42	130	119	114	114
query43	519	513	486	486
query44	1383	863	851	851
query45	191	175	172	172
query46	887	1024	668	668
query47	1769	1807	1748	1748
query48	396	431	322	322
query49	765	521	417	417
query50	691	705	423	423
query51	3915	4019	4018	4018
query52	114	114	113	113
query53	244	277	201	201
query54	607	602	536	536
query55	95	92	87	87
query56	337	336	319	319
query57	1209	1204	1117	1117
query58	273	282	275	275
query59	2605	2703	2535	2535
query60	344	347	315	315
query61	203	163	158	158
query62	823	761	667	667
query63	231	196	206	196
query64	4294	1153	818	818
query65	4044	3985	4023	3985
query66	1082	435	344	344
query67	15560	15398	15219	15219
query68	8719	950	588	588
query69	494	335	295	295
query70	1368	1321	1320	1320
query71	597	344	322	322
query72	6139	5016	5326	5016
query73	782	704	365	365
query74	9156	8867	9041	8867
query75	4090	3307	2788	2788
query76	3776	1187	775	775
query77	813	432	324	324
query78	9582	9725	8913	8913
query79	2199	857	614	614
query80	650	566	498	498
query81	479	276	233	233
query82	384	160	143	143
query83	309	259	249	249
query84	305	109	92	92
query85	887	525	422	422
query86	337	322	298	298
query87	3729	3796	3658	3658
query88	3114	2224	2241	2224
query89	395	332	308	308
query90	1932	225	227	225
query91	179	167	136	136
query92	87	68	65	65
query93	1415	1012	654	654
query94	690	430	346	346
query95	401	324	314	314
query96	494	567	281	281
query97	2943	2955	2876	2876
query98	256	225	208	208
query99	1434	1420	1284	1284
Total cold run time: 275969 ms
Total hot run time: 189408 ms

doris-robot · 2025-09-17T08:04:15Z

ClickBench: Total hot run time: 30.37 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 93e129b97be1a3c5e7c66aa8406596a7036a97af, data reload: false
query1	0.07	0.05	0.06
query2	0.10	0.06	0.06
query3	0.25	0.09	0.09
query4	1.61	0.13	0.13
query5	0.29	0.27	0.27
query6	1.19	0.65	0.66
query7	0.03	0.03	0.04
query8	0.07	0.05	0.05
query9	0.60	0.53	0.52
query10	0.58	0.58	0.59
query11	0.17	0.13	0.12
query12	0.19	0.13	0.12
query13	0.63	0.63	0.62
query14	1.02	1.03	1.04
query15	0.88	0.85	0.87
query16	0.44	0.41	0.41
query17	1.06	1.05	1.07
query18	0.22	0.21	0.21
query19	2.02	1.83	1.81
query20	0.02	0.02	0.02
query21	15.42	0.94	0.58
query22	0.76	1.10	0.61
query23	15.10	1.41	0.66
query24	6.88	1.10	1.41
query25	0.51	0.13	0.19
query26	0.57	0.16	0.14
query27	0.07	0.06	0.06
query28	10.04	0.88	0.45
query29	12.58	3.88	3.24
query30	0.32	0.17	0.14
query31	2.83	0.60	0.39
query32	3.23	0.57	0.48
query33	3.03	3.09	3.26
query34	16.11	5.46	4.84
query35	4.90	4.88	4.93
query36	0.68	0.52	0.51
query37	0.11	0.08	0.07
query38	0.07	0.06	0.05
query39	0.04	0.03	0.04
query40	0.18	0.17	0.15
query41	0.08	0.04	0.03
query42	0.04	0.04	0.03
query43	0.05	0.04	0.04
Total cold run time: 105.04 s
Total hot run time: 30.37 s

dataroaring · 2025-09-17T08:15:42Z

run buildall

Co-authored-by: Xin Liao <liaoxinbit@126.com>

dataroaring · 2025-09-18T01:03:41Z

run buildall

doris-robot · 2025-09-18T01:38:46Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	84.02% (1541/1834)
Line Coverage	68.05% (27598/40553)
Region Coverage	68.51% (13588/19835)
Branch Coverage	58.74% (7255/12350)

liaoxin01

LGTM

dataroaring · 2025-09-18T02:11:44Z

run buildall

doris-robot · 2025-09-18T02:37:39Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	84.02% (1541/1834)
Line Coverage	68.02% (27584/40553)
Region Coverage	68.46% (13580/19835)
Branch Coverage	58.68% (7247/12350)

hello-stephen · 2025-09-18T03:17:29Z

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-09-18T03:54:10Z

TPC-H: Total hot run time: 35157 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 66060fb14401c36633d91de6e55cbc82a9f7c191, data reload: false
------ Round 1 ----------------------------------
q1	17123	5342	5140	5140
q2	2018	326	214	214
q3	10084	1317	737	737
q4	10236	1037	553	553
q5	7657	2432	2413	2413
q6	187	172	146	146
q7	971	786	649	649
q8	9369	1375	1093	1093
q9	7127	5191	5264	5191
q10	6983	2401	1981	1981
q11	506	318	288	288
q12	368	370	246	246
q13	17801	3771	3089	3089
q14	252	246	227	227
q15	583	530	481	481
q16	1015	1006	963	963
q17	622	883	359	359
q18	7923	7112	7089	7089
q19	1096	968	597	597
q20	349	349	239	239
q21	3962	2627	2463	2463
q22	1094	1034	999	999
Total cold run time: 107326 ms
Total hot run time: 35157 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5288	5163	5132	5132
q2	254	336	233	233
q3	2179	2688	2346	2346
q4	1368	1778	1321	1321
q5	4291	4741	4480	4480
q6	222	192	142	142
q7	2079	2110	1760	1760
q8	2725	2636	2633	2633
q9	7502	7303	7343	7303
q10	3245	3324	2863	2863
q11	634	555	532	532
q12	705	815	656	656
q13	3560	3906	3369	3369
q14	295	509	323	323
q15	526	494	482	482
q16	1060	1116	1082	1082
q17	1155	1638	1478	1478
q18	7882	7813	7683	7683
q19	825	821	909	821
q20	2066	2049	1873	1873
q21	4866	4345	4394	4345
q22	1114	1049	1010	1010
Total cold run time: 53841 ms
Total hot run time: 51867 ms

doris-robot · 2025-09-18T04:06:11Z

TPC-DS: Total hot run time: 189346 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 66060fb14401c36633d91de6e55cbc82a9f7c191, data reload: false
query1	1120	450	412	412
query2	6544	1738	1717	1717
query3	6768	221	222	221
query4	26623	24253	23312	23312
query5	4479	631	484	484
query6	347	270	250	250
query7	4681	549	305	305
query8	336	262	255	255
query9	8699	2632	2644	2632
query10	508	360	272	272
query11	15402	15087	14902	14902
query12	179	119	122	119
query13	1686	613	477	477
query14	10402	9383	9469	9383
query15	228	192	187	187
query16	7682	689	520	520
query17	1262	779	678	678
query18	2046	432	391	391
query19	204	199	171	171
query20	130	122	130	122
query21	222	138	118	118
query22	4073	4121	3956	3956
query23	34025	33074	32965	32965
query24	8452	2448	2456	2448
query25	611	511	465	465
query26	1235	291	163	163
query27	2697	521	364	364
query28	4358	2230	2191	2191
query29	800	612	503	503
query30	299	235	213	213
query31	908	852	780	780
query32	86	83	73	73
query33	579	388	332	332
query34	823	909	552	552
query35	819	853	768	768
query36	986	1012	915	915
query37	121	112	89	89
query38	3538	3515	3494	3494
query39	1510	1484	1417	1417
query40	220	139	125	125
query41	66	63	66	63
query42	133	115	124	115
query43	508	516	469	469
query44	1397	859	846	846
query45	186	175	170	170
query46	917	1103	680	680
query47	1787	1789	1715	1715
query48	394	441	320	320
query49	781	525	407	407
query50	689	768	415	415
query51	3942	3936	3960	3936
query52	117	114	106	106
query53	257	286	199	199
query54	617	589	535	535
query55	94	87	87	87
query56	330	331	320	320
query57	1218	1175	1137	1137
query58	289	271	284	271
query59	2548	2664	2510	2510
query60	355	336	328	328
query61	174	170	159	159
query62	818	708	678	678
query63	236	202	200	200
query64	4468	1159	831	831
query65	4084	4022	4008	4008
query66	1128	449	336	336
query67	15509	15265	15085	15085
query68	8832	970	601	601
query69	573	322	283	283
query70	1400	1275	1305	1275
query71	565	351	317	317
query72	6069	5041	4952	4952
query73	744	631	359	359
query74	8929	9129	8794	8794
query75	4242	3315	2845	2845
query76	3680	1196	774	774
query77	822	426	335	335
query78	9501	9725	8913	8913
query79	5066	799	582	582
query80	736	565	500	500
query81	488	262	233	233
query82	681	165	133	133
query83	299	256	253	253
query84	308	119	93	93
query85	899	533	436	436
query86	343	308	306	306
query87	3731	3753	3761	3753
query88	3017	2248	2249	2248
query89	459	326	310	310
query90	1924	287	227	227
query91	162	170	135	135
query92	88	70	65	65
query93	2656	1027	639	639
query94	711	424	344	344
query95	405	318	316	316
query96	498	566	274	274
query97	2937	3010	2870	2870
query98	239	210	217	210
query99	1446	1423	1304	1304
Total cold run time: 281180 ms
Total hot run time: 189346 ms

doris-robot · 2025-09-18T04:11:45Z

ClickBench: Total hot run time: 30.64 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 66060fb14401c36633d91de6e55cbc82a9f7c191, data reload: false
query1	0.05	0.05	0.05
query2	0.12	0.06	0.06
query3	0.31	0.08	0.07
query4	1.61	0.09	0.09
query5	0.26	0.24	0.25
query6	1.18	0.64	0.65
query7	0.04	0.03	0.03
query8	0.07	0.06	0.06
query9	0.68	0.54	0.53
query10	0.59	0.60	0.58
query11	0.25	0.13	0.13
query12	0.26	0.13	0.14
query13	0.66	0.64	0.63
query14	1.05	1.05	1.06
query15	0.96	0.87	0.86
query16	0.40	0.39	0.40
query17	1.08	1.06	1.10
query18	0.25	0.22	0.22
query19	1.99	1.80	1.78
query20	0.02	0.02	0.02
query21	15.39	1.01	0.71
query22	0.94	1.09	0.84
query23	14.69	1.58	0.79
query24	5.53	0.60	0.31
query25	0.18	0.09	0.10
query26	0.55	0.22	0.18
query27	0.10	0.09	0.09
query28	11.03	1.18	0.57
query29	12.55	3.97	3.33
query30	0.32	0.12	0.10
query31	2.84	0.64	0.44
query32	3.24	0.60	0.51
query33	3.15	3.14	3.19
query34	16.86	5.50	4.79
query35	4.94	4.87	4.85
query36	0.65	0.53	0.52
query37	0.22	0.20	0.19
query38	0.20	0.18	0.18
query39	0.06	0.05	0.05
query40	0.21	0.20	0.18
query41	0.11	0.06	0.06
query42	0.08	0.05	0.06
query43	0.06	0.06	0.05
Total cold run time: 105.73 s
Total hot run time: 30.64 s

hello-stephen · 2025-09-18T05:12:03Z

BE UT Coverage Report

Increment line coverage 57.69% (15/26) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.31% (17548/33543)
Line Coverage	37.45% (159034/424635)
Region Coverage	32.04% (121328/378716)
Branch Coverage	33.37% (53203/159421)

hello-stephen · 2025-09-18T06:07:56Z

BE Regression && UT Coverage Report

Increment line coverage 80.77% (21/26) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.11% (23312/32783)
Line Coverage	57.44% (243299/423554)
Region Coverage	52.91% (203000/383635)
Branch Coverage	54.54% (87330/160110)

liaoxin01

LGTM

github-actions · 2025-09-18T07:42:37Z

PR approved by at least one committer and no changes requested.

…che#56120) pick apache#56120. Otherwise, auto partition on multi bes may lead to segment num mismatch problem. Co-authored-by: Yongqiang YANG <yangyogqiang@selectdb.com> Co-authored-by: Xin Liao <liaoxinbit@126.com>

…losed#56120 (#56287) picked from #56120 Co-authored-by: Yongqiang YANG <yangyogqiang@selectdb.com> Co-authored-by: Xin Liao <liaoxinbit@126.com>

…65584) Loads that open incremental destinations at runtime, including `INSERT OVERWRITE ... PARTITION(*)`, auto-partition creation, and auto-detect overwrite, could hang forever. On the receiver, `LoadStream` counted `CLOSE_LOAD` messages in `close()` and registered stream IDs for deferred close in a later lock scope, with `_report_result()` network I/O between them. Under concurrent `CLOSE_LOAD`s, the thread that observed `all_closed=true` could drain and clear `_closing_stream_ids` before another already-counted stream registered its ID. The late stream was then never `StreamClose`d: its `on_closed()` callback never fired, the sink remained in `close_wait`, the fragment never reported completion, and the FE latch was never released. This PR introduces `mark_eos_sent_and_collect()`. Deferred registration, the all-received check, and collection of streams to close now run under the same lock. The function is called after the current stream's EOS response write returns, while `brpc::StreamClose()` remains outside the lock. A write-once `_all_close_load_received` latch ensures that a counted-but-late thread observes that all `CLOSE_LOAD`s have arrived and drains its own deferred stream instead of leaving it orphaned. The existing close fence for loads whose sender opened incremental destinations is preserved. The PR adds a BE unit test and a docker regression suite. The inert debug point `LoadStream.close_load.delay_incremental_register` deterministically opens the count-before-register race window: the buggy implementation leaks streams and hangs, while the fixed implementation closes every stream and completes. ### What problem does this PR solve? Issue Number: close#65582 Related PR: #56120 Problem Summary: `INSERT OVERWRITE ... PARTITION(*)` and other loads that open incremental destinations at runtime could hang until `insert_load_default_timeout_second`. The receiver split `CLOSE_LOAD` counting and deferred-close registration across two lock scopes. A thread could therefore contribute to `_close_load_cnt`, pause before registering its stream ID, and resume only after the last-counting thread had already drained and cleared `_closing_stream_ids`. Because no later event would drain the newly registered ID, that BRPC stream remained open permanently and prevented the load from completing. `_closing_stream_ids` was introduced in #56120 to preserve the close fence required by dynamic destinations. This PR keeps that fence but makes registration and the all-received decision atomic under `_lock`, and performs the resulting `StreamClose()` calls outside the critical section. ### Release note Fix a permanent hang of `INSERT OVERWRITE ... PARTITION(*)` and other loads that open incremental destinations at runtime, caused by a receiver-side load-stream close race. --------- Co-authored-by: lizhuoyu5 <lizhuoyu5@jd.com>

[fix](load_stream) close brpc stream after load stream is closed
e666fa3
Otherwise, auto partition on multi bes may lead to segment num mismatch problem.

fix
a0610c1

kaijchen approved these changes Sep 16, 2025
View reviewed changes

github-actionsBot added the reviewed label Sep 16, 2025

fix
875e4df

fix
904862f

fix
93e129b

fix
085d670

Update be/src/runtime/load_stream.cpp
2c23e8d
Co-authored-by: Xin Liao <liaoxinbit@126.com>

liaoxin01 reviewed Sep 18, 2025
View reviewed changes

Fix formatting in load_stream.cpp
66060fb

liaoxin01 approved these changes Sep 18, 2025
View reviewed changes

dataroaring added the dev/3.1.x label Sep 18, 2025

dataroaring merged commit 7f16e31 into apache:masterSep 18, 2025
25 of 28 checks passed

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 18, 2025

github-actionsBot added the dev/3.1.x-conflict label Sep 18, 2025

dataroaring mentioned this pull request Sep 22, 2025
branch-3.1: [fix](load_stream) close brpc stream after load stream is closed #56120 #56287
Merged

morrySnow added dev/3.1.2-merged and removed dev/3.1.x dev/3.1.x-conflict labels Sep 25, 2025

ojalberts-mydata mentioned this pull request Jun 22, 2026
[Bug] BE write path wedges: a BE-to-BE load-stream brpc socket (8060) goes "Broken" and is never revived (4.0.6, coupled mode) #64708
Open
3 tasks

This was referenced Jul 14, 2026
[Bug] INSERT OVERWRITE ... PARTITION(*) hangs forever due to a receiver-side load-stream close race #65582
Closed
[fix](load_stream) fix receiver-side stream leak race on close_load #65584
Merged

Uh oh!

Conversation

dataroaring commented Sep 16, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Sep 16, 2025

Uh oh!

dataroaring commented Sep 16, 2025

Uh oh!

kaijchen left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Sep 16, 2025

Uh oh!

doris-robot commented Sep 16, 2025

Uh oh!

doris-robot commented Sep 16, 2025

Uh oh!

doris-robot commented Sep 16, 2025

Uh oh!

dataroaring commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

dataroaring commented Sep 17, 2025

Uh oh!

hello-stephen commented Sep 17, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Sep 17, 2025

FE UT Coverage Report

Uh oh!

dataroaring commented Sep 17, 2025

Uh oh!

hello-stephen commented Sep 17, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Sep 17, 2025

FE UT Coverage Report

Uh oh!

dataroaring commented Sep 17, 2025

Uh oh!

hello-stephen commented Sep 17, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

dataroaring commented Sep 17, 2025

Uh oh!

dataroaring commented Sep 18, 2025

Uh oh!

doris-robot commented Sep 18, 2025

Cloud UT Coverage Report

Uh oh!

liaoxin01 left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaring commented Sep 18, 2025

Uh oh!

doris-robot commented Sep 18, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Sep 18, 2025

FE UT Coverage Report

Uh oh!

doris-robot commented Sep 18, 2025

Uh oh!

doris-robot commented Sep 18, 2025

Uh oh!