[Improvement](cloud) Hold tablets in another RPC thread by Gabriel39 · Pull Request #52879 · apache/doris

Gabriel39 · 2025-07-07T08:18:08Z

What problem does this PR solve?

With cloud mode on, remote tablets will be loaded into local storage before execution. This will block execution thread now. If it's too slow, pipeline execution thread may hang.
This PR makes remote tablets loading asynchronously.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Gabriel39 · 2025-07-07T08:18:18Z

run buildall

Thearas · 2025-07-07T08:18:21Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Gabriel39 · 2025-07-07T09:59:57Z

run buildall

doris-robot · 2025-07-07T10:40:12Z

TPC-H: Total hot run time: 33223 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2021a3c92292e983707281c16b7a3828c743a752, data reload: false
------ Round 1 ----------------------------------
q1	17576	5208	5104	5104
q2	1931	286	192	192
q3	10486	1320	720	720
q4	10314	1024	505	505
q5	8956	2398	2373	2373
q6	213	156	126	126
q7	895	753	599	599
q8	9303	1271	1086	1086
q9	6868	5094	5152	5094
q10	6949	2382	1949	1949
q11	513	296	276	276
q12	337	361	209	209
q13	17773	3708	3075	3075
q14	225	224	223	223
q15	563	482	468	468
q16	450	426	390	390
q17	619	863	365	365
q18	7683	7079	7004	7004
q19	1569	1006	562	562
q20	327	341	216	216
q21	3630	3221	2382	2382
q22	357	330	305	305
Total cold run time: 107537 ms
Total hot run time: 33223 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5170	5100	5115	5100
q2	243	328	220	220
q3	2214	2640	2297	2297
q4	1391	1774	1320	1320
q5	4226	4487	4490	4487
q6	223	170	153	153
q7	2052	1992	1801	1801
q8	2680	2511	2506	2506
q9	7245	7224	7266	7224
q10	3125	3305	2900	2900
q11	571	519	489	489
q12	739	801	714	714
q13	3568	3817	3406	3406
q14	271	303	277	277
q15	699	515	477	477
q16	426	479	440	440
q17	1189	1539	1415	1415
q18	7839	7771	7632	7632
q19	843	828	947	828
q20	2060	2079	2001	2001
q21	4848	4320	4282	4282
q22	647	591	543	543
Total cold run time: 52269 ms
Total hot run time: 50512 ms

doris-robot · 2025-07-07T10:51:42Z

TPC-DS: Total hot run time: 185577 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2021a3c92292e983707281c16b7a3828c743a752, data reload: false
query1	1018	387	397	387
query2	6528	1690	1703	1690
query3	6746	217	215	215
query4	26054	23641	23254	23254
query5	4309	559	439	439
query6	293	202	189	189
query7	4626	476	285	285
query8	260	215	203	203
query9	8622	2654	2648	2648
query10	454	346	285	285
query11	15202	15208	14783	14783
query12	160	107	104	104
query13	1656	531	404	404
query14	8544	5815	5847	5815
query15	203	182	176	176
query16	7546	429	260	260
query17	1335	687	560	560
query18	1991	394	301	301
query19	183	214	153	153
query20	120	117	119	117
query21	214	125	102	102
query22	3988	4094	4054	4054
query23	33952	33003	33131	33003
query24	8483	2362	2385	2362
query25	546	486	391	391
query26	1235	267	150	150
query27	2786	513	343	343
query28	4277	2142	2130	2130
query29	764	553	425	425
query30	287	222	197	197
query31	922	843	740	740
query32	73	68	60	60
query33	548	323	303	303
query34	793	851	541	541
query35	608	661	555	555
query36	917	965	873	873
query37	119	104	78	78
query38	4180	4118	4086	4086
query39	1474	1415	1406	1406
query40	204	113	110	110
query41	55	57	51	51
query42	119	107	105	105
query43	512	486	458	458
query44	1338	850	825	825
query45	173	171	163	163
query46	837	1032	630	630
query47	1742	1800	1724	1724
query48	406	434	313	313
query49	741	460	390	390
query50	638	693	416	416
query51	4124	4218	4100	4100
query52	113	106	92	92
query53	232	261	183	183
query54	575	562	490	490
query55	83	78	85	78
query56	287	311	288	288
query57	1147	1208	1118	1118
query58	270	248	254	248
query59	2546	2648	2507	2507
query60	317	309	305	305
query61	123	120	122	120
query62	790	717	673	673
query63	227	185	188	185
query64	4492	1163	803	803
query65	4254	4172	4167	4167
query66	1133	394	329	329
query67	15749	15724	15237	15237
query68	7525	877	539	539
query69	537	312	261	261
query70	1190	1110	1119	1110
query71	386	333	295	295
query72	5774	4765	4890	4765
query73	673	660	357	357
query74	9141	8977	8933	8933
query75	3191	3222	2744	2744
query76	3155	1134	724	724
query77	492	390	282	282
query78	11077	11111	10282	10282
query79	1393	822	589	589
query80	631	530	448	448
query81	477	262	227	227
query82	349	126	94	94
query83	250	305	239	239
query84	248	109	92	92
query85	750	352	314	314
query86	310	304	305	304
query87	4420	4526	4275	4275
query88	2926	2317	2336	2317
query89	386	313	283	283
query90	1780	216	215	215
query91	136	143	113	113
query92	63	59	69	59
query93	1099	961	597	597
query94	633	317	201	201
query95	374	299	285	285
query96	502	554	282	282
query97	2705	2772	2687	2687
query98	226	212	220	212
query99	1332	1415	1277	1277
Total cold run time: 269374 ms
Total hot run time: 185577 ms

doris-robot · 2025-07-07T10:56:59Z

ClickBench: Total hot run time: 29.84 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2021a3c92292e983707281c16b7a3828c743a752, data reload: false
query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.07	0.08
query4	1.62	0.12	0.11
query5	0.42	0.42	0.42
query6	1.18	0.68	0.65
query7	0.03	0.02	0.02
query8	0.05	0.03	0.03
query9	0.59	0.50	0.50
query10	0.56	0.57	0.58
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.63	0.61	0.62
query14	0.80	0.84	0.81
query15	0.89	0.87	0.88
query16	0.39	0.40	0.40
query17	1.10	1.04	1.04
query18	0.23	0.22	0.21
query19	2.02	1.85	1.83
query20	0.02	0.01	0.02
query21	15.38	0.89	0.54
query22	0.75	1.26	0.75
query23	14.76	1.34	0.67
query24	6.67	1.64	0.91
query25	0.50	0.19	0.16
query26	0.57	0.17	0.13
query27	0.06	0.06	0.05
query28	9.58	0.92	0.44
query29	12.55	3.95	3.30
query30	0.25	0.10	0.07
query31	2.83	0.61	0.38
query32	3.23	0.56	0.48
query33	3.06	3.07	3.30
query34	16.10	5.42	4.79
query35	4.77	4.88	4.83
query36	0.69	0.52	0.49
query37	0.10	0.07	0.07
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.18	0.15	0.14
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.47 s
Total hot run time: 29.84 s

doris-robot · 2025-07-07T11:48:59Z

BE UT Coverage Report

Increment line coverage 42.37% (25/59) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.31% (15498/27043)
Line Coverage	46.29% (140853/304278)
Region Coverage	45.55% (71220/156372)
Branch Coverage	40.26% (37526/93208)

Gabriel39 · 2025-07-08T02:18:21Z

run buildall

doris-robot · 2025-07-08T04:03:07Z

TPC-H: Total hot run time: 33231 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e936d8fd51eaeaa6d523611f9846c05e6b102836, data reload: false
------ Round 1 ----------------------------------
q1	17711	5143	5031	5031
q2	1929	278	199	199
q3	10568	1280	718	718
q4	10250	1022	507	507
q5	8311	2460	2356	2356
q6	186	163	126	126
q7	940	736	595	595
q8	9307	1295	1119	1119
q9	7543	5257	5081	5081
q10	6904	2392	1965	1965
q11	483	296	268	268
q12	347	355	217	217
q13	17761	3661	3087	3087
q14	233	235	221	221
q15	548	479	477	477
q16	421	423	380	380
q17	599	875	356	356
q18	7800	7164	7318	7164
q19	1674	942	547	547
q20	322	341	221	221
q21	3720	2556	2310	2310
q22	376	318	286	286
Total cold run time: 107933 ms
Total hot run time: 33231 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5133	5061	5061	5061
q2	247	330	219	219
q3	2176	2695	2304	2304
q4	1456	1779	1350	1350
q5	4217	4569	4416	4416
q6	224	175	126	126
q7	2088	1979	1791	1791
q8	2694	2524	2528	2524
q9	7380	7279	7389	7279
q10	3131	3403	2940	2940
q11	565	502	480	480
q12	677	743	672	672
q13	3651	3919	3574	3574
q14	276	287	274	274
q15	517	477	485	477
q16	464	482	431	431
q17	1218	1779	1374	1374
q18	7801	7698	7669	7669
q19	805	811	983	811
q20	1975	2076	1830	1830
q21	4802	4349	4287	4287
q22	638	592	563	563
Total cold run time: 52135 ms
Total hot run time: 50452 ms

doris-robot · 2025-07-08T04:14:41Z

TPC-DS: Total hot run time: 185766 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e936d8fd51eaeaa6d523611f9846c05e6b102836, data reload: false
query1	1033	390	401	390
query2	6536	1688	1704	1688
query3	6743	230	220	220
query4	26370	23663	23359	23359
query5	4371	598	455	455
query6	303	214	214	214
query7	4629	509	296	296
query8	270	222	217	217
query9	8631	2661	2649	2649
query10	474	328	280	280
query11	15541	15105	14760	14760
query12	158	104	100	100
query13	1672	527	431	431
query14	8509	5957	5932	5932
query15	209	194	172	172
query16	7750	442	261	261
query17	1332	742	585	585
query18	2020	406	333	333
query19	198	191	178	178
query20	120	123	116	116
query21	219	126	120	120
query22	4151	4310	3985	3985
query23	34059	33107	32855	32855
query24	8538	2375	2393	2375
query25	530	482	382	382
query26	815	270	151	151
query27	2730	514	343	343
query28	4367	2181	2149	2149
query29	682	568	431	431
query30	288	215	199	199
query31	938	845	771	771
query32	70	63	59	59
query33	542	328	279	279
query34	816	854	515	515
query35	622	665	551	551
query36	952	1003	879	879
query37	114	101	71	71
query38	4165	4120	4073	4073
query39	1472	1399	1408	1399
query40	208	114	99	99
query41	58	55	53	53
query42	126	113	107	107
query43	516	508	488	488
query44	1309	842	842	842
query45	174	175	168	168
query46	825	1014	621	621
query47	1775	1761	1750	1750
query48	387	417	309	309
query49	708	476	386	386
query50	644	702	421	421
query51	4253	4164	4291	4164
query52	109	109	92	92
query53	224	255	195	195
query54	582	567	501	501
query55	80	81	81	81
query56	307	307	304	304
query57	1162	1212	1121	1121
query58	261	253	264	253
query59	2672	2766	2591	2591
query60	332	325	306	306
query61	155	123	126	123
query62	787	724	655	655
query63	231	186	189	186
query64	3534	1201	892	892
query65	4253	4183	4219	4183
query66	963	430	324	324
query67	15854	15580	15379	15379
query68	8526	869	532	532
query69	510	310	270	270
query70	1199	1089	1044	1044
query71	463	320	298	298
query72	5772	4745	4844	4745
query73	729	636	351	351
query74	9263	9086	8671	8671
query75	3891	3272	2706	2706
query76	3591	1152	710	710
query77	788	369	292	292
query78	10947	11367	10321	10321
query79	1143	821	588	588
query80	618	518	453	453
query81	470	320	219	219
query82	376	127	98	98
query83	257	256	234	234
query84	247	107	82	82
query85	771	364	315	315
query86	332	292	296	292
query87	4348	4422	4298	4298
query88	2976	2319	2317	2317
query89	375	309	289	289
query90	1965	217	205	205
query91	138	158	110	110
query92	70	59	56	56
query93	1070	981	600	600
query94	689	318	196	196
query95	381	301	289	289
query96	499	568	286	286
query97	2775	2788	2638	2638
query98	228	207	204	204
query99	1496	1401	1240	1240
Total cold run time: 272331 ms
Total hot run time: 185766 ms

doris-robot · 2025-07-08T04:19:58Z

ClickBench: Total hot run time: 29.72 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e936d8fd51eaeaa6d523611f9846c05e6b102836, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.08	0.07
query4	1.62	0.11	0.12
query5	0.42	0.43	0.44
query6	1.17	0.68	0.67
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.60	0.50	0.52
query10	0.57	0.57	0.58
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.63	0.61	0.61
query14	0.82	0.81	0.85
query15	0.90	0.90	0.89
query16	0.39	0.39	0.38
query17	1.05	1.04	1.07
query18	0.23	0.22	0.22
query19	1.93	1.90	1.92
query20	0.01	0.01	0.02
query21	15.38	0.90	0.54
query22	0.77	1.12	0.67
query23	14.97	1.39	0.59
query24	6.88	1.53	0.83
query25	0.48	0.15	0.07
query26	0.62	0.17	0.13
query27	0.06	0.05	0.06
query28	9.69	0.91	0.44
query29	12.56	4.02	3.37
query30	0.25	0.09	0.06
query31	2.84	0.59	0.39
query32	3.25	0.56	0.46
query33	3.15	3.16	3.14
query34	16.09	5.45	4.77
query35	4.87	4.84	4.86
query36	0.70	0.51	0.49
query37	0.10	0.07	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.15	0.15
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.15 s
Total hot run time: 29.72 s

Gabriel39 · 2025-07-08T04:44:20Z

run buildall

doris-robot · 2025-07-08T05:58:32Z

TPC-H: Total hot run time: 33335 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 088a6678ed1c3311734942b3a7371bf95786258c, data reload: false
------ Round 1 ----------------------------------
q1	17603	5202	4996	4996
q2	1934	281	186	186
q3	10321	1286	720	720
q4	10226	1052	521	521
q5	7537	2373	2344	2344
q6	185	157	128	128
q7	884	736	594	594
q8	9299	1288	1056	1056
q9	6827	5154	5132	5132
q10	6874	2386	1936	1936
q11	500	288	280	280
q12	342	354	220	220
q13	17776	3682	3118	3118
q14	221	234	219	219
q15	549	477	480	477
q16	419	456	377	377
q17	605	880	359	359
q18	7623	7452	7247	7247
q19	1207	949	564	564
q20	327	333	219	219
q21	3893	2521	2339	2339
q22	352	318	303	303
Total cold run time: 105504 ms
Total hot run time: 33335 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5096	5096	5059	5059
q2	240	325	229	229
q3	2190	2710	2276	2276
q4	1407	1803	1415	1415
q5	4218	4201	4632	4201
q6	220	172	131	131
q7	2026	2008	1785	1785
q8	2682	2765	2662	2662
q9	7393	7352	7636	7352
q10	3079	3259	2813	2813
q11	563	524	527	524
q12	691	763	654	654
q13	3553	3886	3334	3334
q14	316	319	311	311
q15	550	491	475	475
q16	450	495	451	451
q17	1235	1611	1362	1362
q18	7954	7726	7629	7629
q19	825	817	857	817
q20	2065	2194	1891	1891
q21	5082	4655	4462	4462
q22	700	629	540	540
Total cold run time: 52535 ms
Total hot run time: 50373 ms

doris-robot · 2025-07-08T06:10:06Z

TPC-DS: Total hot run time: 184087 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 088a6678ed1c3311734942b3a7371bf95786258c, data reload: false
query1	1002	384	393	384
query2	6525	1742	1689	1689
query3	6747	211	214	211
query4	26191	23619	23236	23236
query5	4374	585	429	429
query6	303	213	230	213
query7	4631	496	289	289
query8	284	210	234	210
query9	8607	2626	2645	2626
query10	489	311	266	266
query11	15710	15499	14844	14844
query12	153	111	104	104
query13	1646	520	390	390
query14	9084	5627	5532	5532
query15	197	185	167	167
query16	7242	634	468	468
query17	1184	707	566	566
query18	1975	390	297	297
query19	185	183	199	183
query20	124	117	117	117
query21	214	117	105	105
query22	4225	4206	4476	4206
query23	34808	33810	32990	32990
query24	8373	2350	2440	2350
query25	524	452	420	420
query26	1227	274	148	148
query27	2748	515	361	361
query28	4350	2130	2112	2112
query29	767	546	425	425
query30	277	223	179	179
query31	914	828	742	742
query32	73	57	58	57
query33	549	348	298	298
query34	796	829	506	506
query35	777	794	745	745
query36	952	1001	886	886
query37	99	91	68	68
query38	4197	4074	4042	4042
query39	1449	1409	1405	1405
query40	220	126	99	99
query41	53	54	48	48
query42	121	102	103	102
query43	494	507	461	461
query44	1313	820	830	820
query45	203	164	159	159
query46	865	987	621	621
query47	1797	1779	1736	1736
query48	376	409	299	299
query49	769	471	391	391
query50	642	673	405	405
query51	4203	4122	4231	4122
query52	113	106	98	98
query53	234	261	182	182
query54	558	566	504	504
query55	82	79	81	79
query56	324	325	272	272
query57	1186	1184	1105	1105
query58	265	248	239	239
query59	2629	2697	2676	2676
query60	320	320	311	311
query61	128	126	125	125
query62	805	719	652	652
query63	226	202	181	181
query64	4352	1003	670	670
query65	4260	4151	4168	4151
query66	1164	417	336	336
query67	15955	15424	15472	15424
query68	7847	875	524	524
query69	488	311	268	268
query70	1142	1102	1084	1084
query71	424	318	312	312
query72	5810	4647	4507	4507
query73	635	556	354	354
query74	8861	9132	8666	8666
query75	3471	3153	2639	2639
query76	3257	1134	691	691
query77	528	402	286	286
query78	10185	10310	9313	9313
query79	2413	781	571	571
query80	622	507	420	420
query81	476	267	233	233
query82	437	122	91	91
query83	281	249	230	230
query84	297	112	80	80
query85	773	404	315	315
query86	374	313	272	272
query87	4411	4431	4298	4298
query88	3656	2262	2281	2262
query89	380	306	286	286
query90	1910	204	205	204
query91	142	143	110	110
query92	75	57	57	57
query93	1903	935	586	586
query94	696	417	297	297
query95	370	290	278	278
query96	490	580	277	277
query97	2726	2771	2600	2600
query98	229	203	197	197
query99	1460	1410	1245	1245
Total cold run time: 274398 ms
Total hot run time: 184087 ms

doris-robot · 2025-07-08T06:15:22Z

ClickBench: Total hot run time: 29.78 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 088a6678ed1c3311734942b3a7371bf95786258c, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.07	0.08
query4	1.61	0.11	0.10
query5	0.45	0.43	0.41
query6	1.16	0.66	0.65
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.62	0.51	0.51
query10	0.57	0.57	0.56
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.62	0.61	0.62
query14	0.84	0.80	0.81
query15	0.89	0.86	0.85
query16	0.41	0.37	0.39
query17	1.04	1.02	1.06
query18	0.22	0.21	0.21
query19	1.94	1.84	1.90
query20	0.02	0.01	0.01
query21	15.42	0.89	0.54
query22	0.76	1.19	0.69
query23	14.90	1.33	0.59
query24	6.92	1.14	1.06
query25	0.52	0.15	0.13
query26	0.58	0.16	0.14
query27	0.06	0.04	0.05
query28	9.74	0.91	0.44
query29	12.56	4.02	3.33
query30	0.26	0.09	0.07
query31	2.85	0.60	0.38
query32	3.23	0.55	0.47
query33	3.14	3.07	3.08
query34	16.32	5.44	4.80
query35	4.90	4.89	4.84
query36	0.69	0.50	0.49
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 104.46 s
Total hot run time: 29.78 s

Gabriel39 · 2025-07-08T07:18:22Z

run buildall

doris-robot · 2025-07-08T10:05:20Z

BE UT Coverage Report

Increment line coverage 27.27% (24/88) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.31% (15502/27048)
Line Coverage	46.30% (140886/304290)
Region Coverage	45.56% (71240/156376)
Branch Coverage	40.27% (37535/93206)

Gabriel39 · 2025-07-08T14:28:52Z

run buildall

doris-robot · 2025-07-08T16:28:56Z

BE UT Coverage Report

Increment line coverage 27.59% (24/87) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.32% (15508/27056)
Line Coverage	46.30% (140927/304358)
Region Coverage	45.55% (71253/156412)
Branch Coverage	40.27% (37545/93222)

Gabriel39 · 2025-07-09T02:08:49Z

run buildall

Gabriel39 · 2025-07-09T03:57:06Z

run buildall

doris-robot · 2025-07-09T04:24:16Z

TPC-H: Total hot run time: 33354 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9645b47164d8cde7fa6492b07b4b5eb0f5264b5, data reload: false
------ Round 1 ----------------------------------
q1	17580	5216	5114	5114
q2	1930	291	185	185
q3	10582	1338	736	736
q4	10290	1032	506	506
q5	8595	2365	2389	2365
q6	191	165	128	128
q7	877	746	602	602
q8	9299	1249	1064	1064
q9	6849	5080	5092	5080
q10	6886	2369	1979	1979
q11	470	296	286	286
q12	340	340	222	222
q13	17766	3729	3085	3085
q14	222	236	231	231
q15	535	489	476	476
q16	424	410	377	377
q17	596	861	376	376
q18	7441	7183	7133	7133
q19	1201	955	557	557
q20	335	344	226	226
q21	3673	3176	2334	2334
q22	355	327	292	292
Total cold run time: 106437 ms
Total hot run time: 33354 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5108	5107	5075	5075
q2	236	332	227	227
q3	2160	2697	2260	2260
q4	1372	1773	1311	1311
q5	4195	4424	4579	4424
q6	217	169	129	129
q7	2019	1962	1815	1815
q8	2704	2712	2626	2626
q9	7365	7218	7215	7215
q10	3110	3297	2872	2872
q11	576	509	610	509
q12	688	773	637	637
q13	3494	4021	3447	3447
q14	283	311	300	300
q15	541	507	490	490
q16	452	511	445	445
q17	1192	1634	1397	1397
q18	7859	7802	7574	7574
q19	759	846	1102	846
q20	2014	2070	1951	1951
q21	5045	4549	4626	4549
q22	668	645	572	572
Total cold run time: 52057 ms
Total hot run time: 50671 ms

doris-robot · 2025-07-10T11:12:06Z

ClickBench: Total hot run time: 29.88 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 26262626b0b87532253da053283f6bc364a62c23, data reload: false
query1	0.04	0.04	0.04
query2	0.08	0.05	0.04
query3	0.24	0.07	0.07
query4	1.63	0.11	0.11
query5	0.41	0.42	0.40
query6	1.18	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.60	0.52	0.53
query10	0.57	0.57	0.58
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.61	0.62
query14	0.79	0.80	0.82
query15	0.90	0.87	0.89
query16	0.38	0.38	0.40
query17	1.07	1.12	1.04
query18	0.22	0.22	0.21
query19	1.91	1.84	1.87
query20	0.01	0.01	0.02
query21	15.39	0.91	0.54
query22	0.75	1.02	0.58
query23	15.21	1.39	0.61
query24	7.09	1.72	1.19
query25	0.56	0.19	0.11
query26	0.66	0.17	0.14
query27	0.06	0.06	0.05
query28	10.18	0.83	0.44
query29	12.58	4.06	3.32
query30	0.25	0.09	0.07
query31	2.83	0.62	0.39
query32	3.25	0.55	0.47
query33	3.13	3.12	3.16
query34	16.08	5.42	4.73
query35	4.84	4.87	4.92
query36	0.68	0.50	0.49
query37	0.09	0.08	0.06
query38	0.06	0.05	0.04
query39	0.03	0.02	0.03
query40	0.17	0.16	0.14
query41	0.08	0.03	0.02
query42	0.03	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.05 s
Total hot run time: 29.88 s

doris-robot · 2025-07-10T12:36:18Z

BE UT Coverage Report

Increment line coverage 33.33% (33/99) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.05% (15509/27185)
Line Coverage	45.89% (138431/301683)
Region Coverage	35.34% (103656/293286)
Branch Coverage	37.77% (45582/120690)

yiguolei · 2025-07-16T04:26:16Z


+Status OlapScanLocalState::init(RuntimeState* state, LocalStateInfo& info) {
+ RETURN_IF_ERROR(Base::init(state, info));
+ RETURN_IF_ERROR(_sync_cloud_tablets(state));


这个_sync_cloud_tablets 实际是一个执行逻辑，但是我们把他放到了一个初始化的逻辑里，可能会有一些不可预期的问题

yiguolei · 2025-07-16T07:27:56Z

+ _tablets.resize(_scan_ranges.size());
+ _tasks.reserve(_scan_ranges.size());
+ _sync_statistics.resize(_scan_ranges.size());
+ SCOPED_RAW_TIMER(&_duration_ns);


这个_duration_ns 似乎没用，因为底下这个调用是异步的。
可能我们需要跟踪一下那个dependency的时间，比如他set ready的时候，记录一个时间，这样才是准确的。

Gabriel39 · 2025-07-16T11:07:26Z

run buildall

github-actions · 2025-07-16T12:02:54Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-07-16T12:02:56Z

PR approved by anyone and no changes requested.

doris-robot · 2025-07-16T12:52:14Z

TPC-H: Total hot run time: 34017 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 36f2eb433af77dbd0208bb18c7a3bc3ced58ced9, data reload: false
------ Round 1 ----------------------------------
q1	17567	5274	5112	5112
q2	1920	286	181	181
q3	10329	1285	717	717
q4	10225	1025	530	530
q5	7462	2426	2351	2351
q6	181	166	133	133
q7	899	732	624	624
q8	9296	1317	1048	1048
q9	6924	5184	5147	5147
q10	6877	2395	1977	1977
q11	466	296	276	276
q12	352	349	215	215
q13	17767	3708	3088	3088
q14	223	241	219	219
q15	554	480	468	468
q16	427	438	380	380
q17	586	865	349	349
q18	7533	7174	7127	7127
q19	1217	962	570	570
q20	344	336	225	225
q21	3670	3195	2335	2335
q22	1063	992	945	945
Total cold run time: 105882 ms
Total hot run time: 34017 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5226	5140	5179	5140
q2	242	330	226	226
q3	2212	2692	2286	2286
q4	1309	1765	1323	1323
q5	4272	4541	4412	4412
q6	215	170	131	131
q7	2057	1914	1801	1801
q8	2651	2602	2575	2575
q9	7282	7259	7310	7259
q10	3141	3345	2951	2951
q11	604	550	503	503
q12	705	902	595	595
q13	3657	3956	3336	3336
q14	280	316	283	283
q15	521	479	496	479
q16	455	528	442	442
q17	1191	1559	1452	1452
q18	7795	7673	7672	7672
q19	795	807	819	807
q20	2042	2035	1805	1805
q21	4821	4299	4389	4299
q22	1041	1013	991	991
Total cold run time: 52514 ms
Total hot run time: 50768 ms

doris-robot · 2025-07-16T13:03:54Z

TPC-DS: Total hot run time: 186876 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 36f2eb433af77dbd0208bb18c7a3bc3ced58ced9, data reload: false
query1	1001	392	441	392
query2	6517	1725	1641	1641
query3	6742	209	226	209
query4	26362	23867	23504	23504
query5	4355	592	433	433
query6	308	216	206	206
query7	4623	490	289	289
query8	269	225	220	220
query9	8614	2626	2629	2626
query10	506	326	282	282
query11	15765	15037	14744	14744
query12	158	103	105	103
query13	1633	514	395	395
query14	9165	5853	5832	5832
query15	201	203	169	169
query16	7342	617	483	483
query17	1174	705	583	583
query18	1989	394	296	296
query19	193	187	172	172
query20	122	116	114	114
query21	217	126	111	111
query22	4182	4149	4063	4063
query23	34041	32976	32780	32780
query24	8440	2372	2357	2357
query25	549	471	422	422
query26	1228	270	152	152
query27	2773	503	350	350
query28	4351	2166	2112	2112
query29	783	581	470	470
query30	282	224	189	189
query31	955	846	773	773
query32	74	65	63	63
query33	576	360	332	332
query34	808	844	514	514
query35	801	816	738	738
query36	976	996	910	910
query37	111	102	89	89
query38	4088	4112	4140	4112
query39	1526	1444	1421	1421
query40	219	129	110	110
query41	66	74	58	58
query42	124	107	107	107
query43	510	480	492	480
query44	1328	823	817	817
query45	179	175	168	168
query46	880	1020	622	622
query47	1790	1790	1726	1726
query48	384	420	307	307
query49	721	447	390	390
query50	616	689	416	416
query51	5410	5592	5538	5538
query52	106	114	100	100
query53	224	248	185	185
query54	577	577	508	508
query55	86	79	81	79
query56	310	366	303	303
query57	1170	1193	1105	1105
query58	264	256	259	256
query59	2522	2591	2517	2517
query60	332	325	306	306
query61	129	126	127	126
query62	801	721	681	681
query63	222	185	180	180
query64	4356	1017	667	667
query65	4287	4238	4200	4200
query66	1153	421	323	323
query67	15866	15704	15597	15597
query68	8903	898	532	532
query69	479	328	271	271
query70	1240	1149	1090	1090
query71	470	326	306	306
query72	5311	4703	4656	4656
query73	756	569	349	349
query74	8891	8937	9051	8937
query75	4173	3213	2684	2684
query76	3613	1155	719	719
query77	794	375	295	295
query78	9884	10100	9380	9380
query79	3078	816	585	585
query80	655	507	450	450
query81	474	252	219	219
query82	262	136	97	97
query83	284	248	243	243
query84	287	106	90	90
query85	757	370	325	325
query86	350	324	284	284
query87	4385	4425	4282	4282
query88	2858	2253	2262	2253
query89	431	321	293	293
query90	2065	210	208	208
query91	140	137	120	120
query92	81	62	55	55
query93	2091	932	585	585
query94	680	438	318	318
query95	382	293	291	291
query96	478	579	288	288
query97	2734	2763	2642	2642
query98	226	212	201	201
query99	1413	1405	1305	1305
Total cold run time: 277057 ms
Total hot run time: 186876 ms

doris-robot · 2025-07-16T13:09:22Z

ClickBench: Total hot run time: 31.95 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 36f2eb433af77dbd0208bb18c7a3bc3ced58ced9, data reload: false
query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.24	0.07	0.07
query4	1.62	0.10	0.11
query5	0.44	0.43	0.41
query6	1.15	0.67	0.68
query7	0.02	0.02	0.02
query8	0.05	0.03	0.04
query9	0.59	0.51	0.50
query10	0.58	0.57	0.56
query11	0.16	0.10	0.11
query12	0.15	0.12	0.12
query13	0.63	0.61	0.61
query14	0.81	0.82	0.80
query15	0.88	0.87	0.91
query16	0.40	0.39	0.39
query17	1.07	1.07	1.07
query18	0.22	0.21	0.21
query19	1.91	1.81	1.89
query20	0.01	0.01	0.01
query21	15.38	0.90	0.55
query22	0.76	1.05	0.65
query23	15.09	1.39	0.58
query24	6.56	1.42	0.46
query25	0.53	0.18	0.07
query26	0.58	0.16	0.15
query27	0.07	0.05	0.06
query28	9.86	0.95	0.44
query29	12.51	4.09	3.34
query30	3.01	2.98	2.88
query31	2.82	0.62	0.38
query32	3.23	0.55	0.48
query33	3.17	3.06	3.14
query34	16.01	5.43	4.82
query35	4.87	4.85	4.81
query36	0.70	0.51	0.48
query37	0.09	0.07	0.06
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.15	0.14
query41	0.07	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.67 s
Total hot run time: 31.95 s

doris-robot · 2025-07-16T14:14:27Z

BE UT Coverage Report

Increment line coverage 32.67% (33/101) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.50% (15723/27346)
Line Coverage	46.13% (140209/303942)
Region Coverage	35.41% (104495/295102)
Branch Coverage	38.03% (46080/121161)

hello-stephen · 2025-07-16T14:36:36Z

BE Regression && UT Coverage Report

Increment line coverage 97.96% (96/98) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	81.10% (21753/26822)
Line Coverage	73.70% (223551/303331)
Region Coverage	61.31% (185622/302760)
Branch Coverage	65.23% (79905/122505)

hello-stephen · 2025-07-17T01:29:33Z

BE Regression && UT Coverage Report

Increment line coverage 97.96% (96/98) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	81.10% (21753/26822)
Line Coverage	73.70% (223551/303331)
Region Coverage	61.31% (185622/302760)
Branch Coverage	65.23% (79905/122505)

With cloud mode on, remote tablets will be loaded into local storage before execution. This will block execution thread now. If it's too slow, pipeline execution thread may hang. This PR makes remote tablets loading asynchronously.

…52879 #53605 (#53416) pick #52879 #53605 With cloud mode on, remote tablets will be loaded into local storage before execution. This will block execution thread now. If it's too slow, pipeline execution thread may hang. This PR makes remote tablets loading asynchronously.

[Improvement](cloud) Hold tablets in another RPC thread
585a6ba

update
2021a3c

update
e936d8f

update
088a667

update
b5f1f09

update
4eebf9c

update
206c79a

update
c9645b4

yiguolei added dev/3.1.x usercase Important user case type label labels Jul 16, 2025

yiguolei reviewed Jul 16, 2025
View reviewed changes

update
36f2eb4

yiguolei approved these changes Jul 16, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 16, 2025

github-actionsBot added the reviewed label Jul 16, 2025

Mryange approved these changes Jul 16, 2025
View reviewed changes

Gabriel39 merged commit 53b096d into apache:masterJul 17, 2025

github-actionsBot added the dev/3.1.x-conflict label Jul 17, 2025

Gabriel39 mentioned this pull request Jul 22, 2025
branch-3.1: [Improvement](cloud) Hold tablets in another RPC thread #52879 #53605 #53416
Merged
16 tasks

morrySnow added dev/3.1.0-merged and removed dev/3.1.x dev/3.1.x-conflict labels Jul 23, 2025

Gabriel39 deleted the dev_0707 branch July 30, 2026 06:26

Uh oh!

Conversation

Gabriel39 commented Jul 7, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Gabriel39 commented Jul 7, 2025

Uh oh!

Thearas commented Jul 7, 2025

Uh oh!

Gabriel39 commented Jul 7, 2025

Uh oh!

doris-robot commented Jul 7, 2025

Uh oh!

doris-robot commented Jul 7, 2025

Uh oh!

doris-robot commented Jul 7, 2025

Uh oh!

doris-robot commented Jul 7, 2025

BE UT Coverage Report

Uh oh!

Gabriel39 commented Jul 8, 2025

Uh oh!

doris-robot commented Jul 8, 2025

Uh oh!

doris-robot commented Jul 8, 2025

Uh oh!

doris-robot commented Jul 8, 2025

Uh oh!

Gabriel39 commented Jul 8, 2025

Uh oh!

doris-robot commented Jul 8, 2025

Uh oh!

doris-robot commented Jul 8, 2025

Uh oh!

doris-robot commented Jul 8, 2025

Uh oh!

Gabriel39 commented Jul 8, 2025

Uh oh!

doris-robot commented Jul 8, 2025

BE UT Coverage Report

Uh oh!

Gabriel39 commented Jul 8, 2025

Uh oh!

doris-robot commented Jul 8, 2025

BE UT Coverage Report

Uh oh!

Gabriel39 commented Jul 9, 2025

Uh oh!

Gabriel39 commented Jul 9, 2025

Uh oh!

doris-robot commented Jul 9, 2025

Uh oh!

doris-robot commented Jul 10, 2025

Uh oh!

doris-robot commented Jul 10, 2025

BE UT Coverage Report

Uh oh!

yiguoleiJul 16, 2025

Choose a reason for hiding this comment

Uh oh!

yiguoleiJul 16, 2025

Choose a reason for hiding this comment

Uh oh!

Gabriel39 commented Jul 16, 2025

Uh oh!

github-actionsBot commented Jul 16, 2025

Uh oh!

github-actionsBot commented Jul 16, 2025

Uh oh!

doris-robot commented Jul 16, 2025

Uh oh!

doris-robot commented Jul 16, 2025

Uh oh!

doris-robot commented Jul 16, 2025

Uh oh!

doris-robot commented Jul 16, 2025

BE UT Coverage Report

Gabriel39 commented Jul 7, 2025•
edited
Loading