[refactor](compaction) submit manual full compaction task to thread pool instead of detached thread by Yukang-Lian · Pull Request #61222 · apache/doris

Yukang-Lian · 2026-03-11T08:14:34Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Previously, manually triggering full compaction on a single tablet via HTTP API would create a detached thread for each request, which lacks concurrency control, deduplication, and metrics tracking.

This PR changes the single-tablet full compaction path to use submit_compaction_task(), submitting to the base compaction thread pool — consistent with the table-level path and the cloud engine. Base and cumulative compaction remain unchanged.

Added force HTTP parameter to skip permit limiter when submitting full compaction, allowing compaction to proceed even when resources are constrained.

Usage

# single tablet, default (with permit limiter)
curl -X POST "http://be_host:http_port/api/compaction/run?tablet_id=12345&compact_type=full"# single tablet, force (skip permit limiter)
curl -X POST "http://be_host:http_port/api/compaction/run?tablet_id=12345&compact_type=full&force=true"# table level
curl -X POST "http://be_host:http_port/api/compaction/run?table_id=67890&compact_type=full"# table level, force
curl -X POST "http://be_host:http_port/api/compaction/run?table_id=67890&compact_type=full&force=true"

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test.
Behavior changed:
- No.
- Yes. Single-tablet manual full compaction now submits to thread pool instead of detached thread. Added force parameter to skip permit limiter.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Yukang-Lian · 2026-03-11T08:14:44Z

run buildall

Thearas · 2026-03-11T08:14:55Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Yukang-Lian · 2026-03-11T08:34:36Z

run buildall

Yukang-Lian · 2026-03-11T08:38:33Z

run buildall

doris-robot · 2026-03-11T09:01:38Z

TPC-H: Total hot run time: 27717 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4420035c2c3afc07f46a1e0473141101f9fffae5, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17641	4571	4315	4315
q2	q3	10643	807	513	513
q4	4679	368	260	260
q5	7553	1198	1019	1019
q6	180	179	148	148
q7	802	845	681	681
q8	9282	1477	1380	1380
q9	4945	4736	4642	4642
q10	6248	1904	1652	1652
q11	447	263	241	241
q12	674	564	468	468
q13	18070	2928	2169	2169
q14	233	231	215	215
q15	886	793	793	793
q16	734	722	667	667
q17	712	865	438	438
q18	6020	5373	5293	5293
q19	1112	1011	615	615
q20	503	507	389	389
q21	4342	2082	1550	1550
q22	426	309	269	269
Total cold run time: 96132 ms
Total hot run time: 27717 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4804	4549	4527	4527
q2	q3	3863	4331	3819	3819
q4	882	1219	790	790
q5	4055	4439	4340	4340
q6	186	181	145	145
q7	1757	1699	1499	1499
q8	2482	2713	2734	2713
q9	7445	7154	7427	7154
q10	3782	3945	3640	3640
q11	509	479	486	479
q12	539	603	456	456
q13	2675	3143	2365	2365
q14	289	301	294	294
q15	890	827	796	796
q16	722	754	751	751
q17	1141	1425	1353	1353
q18	7051	6721	6578	6578
q19	958	878	893	878
q20	2064	2198	2034	2034
q21	4029	3498	3413	3413
q22	528	462	393	393
Total cold run time: 50651 ms
Total hot run time: 48417 ms

doris-robot · 2026-03-11T09:11:38Z

TPC-DS: Total hot run time: 153336 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4420035c2c3afc07f46a1e0473141101f9fffae5, data reload: false
query5	4345	629	508	508
query6	330	226	222	222
query7	4219	474	263	263
query8	342	250	259	250
query9	8719	2766	2745	2745
query10	557	389	366	366
query11	7391	5811	5575	5575
query12	180	135	127	127
query13	1284	456	351	351
query14	6362	3808	3527	3527
query14_1	2831	2849	2833	2833
query15	208	194	180	180
query16	1001	490	477	477
query17	1111	721	621	621
query18	2654	448	353	353
query19	210	209	180	180
query20	137	136	127	127
query21	227	151	125	125
query22	4917	5003	4828	4828
query23	16556	16232	15738	15738
query23_1	15772	16003	15953	15953
query24	7991	1685	1254	1254
query24_1	1260	1212	1231	1212
query25	572	505	490	490
query26	1243	261	148	148
query27	2782	468	286	286
query28	4478	1849	1849	1849
query29	888	561	469	469
query30	305	246	209	209
query31	1355	1287	1219	1219
query32	100	75	72	72
query33	504	329	274	274
query34	958	914	579	579
query35	635	677	609	609
query36	1040	1172	920	920
query37	131	96	81	81
query38	2969	2897	2894	2894
query39	886	877	851	851
query39_1	820	840	823	823
query40	230	153	134	134
query41	65	62	61	61
query42	342	293	300	293
query43	245	258	226	226
query44 query45	201	190	186	186
query46	856	983	604	604
query47	2123	2136	2067	2067
query48	318	306	234	234
query49	643	478	379	379
query50	674	275	215	215
query51	4102	4107	4065	4065
query52	284	286	280	280
query53	293	339	284	284
query54	293	275	258	258
query55	94	89	86	86
query56	307	320	302	302
query57	1367	1341	1259	1259
query58	285	279	275	275
query59	1302	1468	1325	1325
query60	336	321	318	318
query61	143	143	144	143
query62	611	584	542	542
query63	313	278	269	269
query64	5135	1288	1011	1011
query65 query66	1399	462	351	351
query67	16353	16431	16379	16379
query68 query69	383	297	280	280
query70	1024	989	982	982
query71	340	311	303	303
query72	2921	2426	2492	2426
query73	539	551	317	317
query74	9984	9933	9746	9746
query75	2864	2765	2460	2460
query76	2301	1031	671	671
query77	360	400	311	311
query78	11149	11537	10678	10678
query79	1134	800	600	600
query80	1512	625	520	520
query81	576	279	243	243
query82	1015	155	118	118
query83	347	265	241	241
query84	241	115	104	104
query85	1072	489	431	431
query86	419	313	301	301
query87	3133	3122	3007	3007
query88	3559	2663	2670	2663
query89	426	367	349	349
query90	1958	177	173	173
query91	168	157	135	135
query92	80	86	71	71
query93	961	818	505	505
query94	640	328	299	299
query95	590	410	332	332
query96	652	512	231	231
query97	2441	2516	2481	2481
query98	233	221	221	221
query99	943	996	925	925
Total cold run time: 235540 ms
Total hot run time: 153336 ms

doris-robot · 2026-03-11T09:56:17Z

BE UT Coverage Report

Increment line coverage 0.00% (0/23) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.57% (19653/37384)
Line Coverage	36.21% (183434/506642)
Region Coverage	32.33% (141631/438061)
Branch Coverage	33.48% (61774/184503)

hello-stephen · 2026-03-11T11:22:28Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/23) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.62% (26219/36606)
Line Coverage	54.40% (274792/505101)
Region Coverage	51.64% (228349/442209)
Branch Coverage	53.11% (98292/185079)

Yukang-Lian · 2026-03-12T12:24:11Z

run buildall

Yukang-Lian · 2026-03-12T12:40:23Z

run buildall

doris-robot · 2026-03-12T14:12:42Z

TPC-H: Total hot run time: 27766 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7ff50ae2caf139d0337bed64698ba211c5ec102d, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17629	4539	4326	4326
q2	q3	10665	800	519	519
q4	4735	350	257	257
q5	8002	1217	1012	1012
q6	212	174	147	147
q7	814	831	667	667
q8	10345	1540	1376	1376
q9	6629	4752	4757	4752
q10	6437	1922	1648	1648
q11	492	266	262	262
q12	749	566	473	473
q13	18098	2934	2187	2187
q14	239	231	215	215
q15	928	822	797	797
q16	789	731	686	686
q17	723	883	419	419
q18	6032	5495	5215	5215
q19	1149	1004	616	616
q20	505	500	391	391
q21	4924	1979	1531	1531
q22	393	311	270	270
Total cold run time: 100489 ms
Total hot run time: 27766 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4737	4526	4526	4526
q2	q3	3865	4328	3782	3782
q4	909	1189	803	803
q5	4083	4372	4338	4338
q6	207	187	154	154
q7	1833	1698	1551	1551
q8	2501	2808	2598	2598
q9	7436	7334	7371	7334
q10	3775	4006	3685	3685
q11	503	457	410	410
q12	492	614	433	433
q13	2727	3120	2387	2387
q14	292	310	287	287
q15	828	785	787	785
q16	720	755	722	722
q17	1152	1500	1462	1462
q18	7234	6979	6583	6583
q19	911	927	944	927
q20	2084	2244	2029	2029
q21	4042	3482	3340	3340
q22	469	451	395	395
Total cold run time: 50800 ms
Total hot run time: 48531 ms

doris-robot · 2026-03-12T14:22:41Z

TPC-DS: Total hot run time: 153332 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7ff50ae2caf139d0337bed64698ba211c5ec102d, data reload: false
query5	4327	638	527	527
query6	336	227	209	209
query7	4206	471	265	265
query8	339	240	231	231
query9	8718	2734	2706	2706
query10	508	391	333	333
query11	7312	5772	5644	5644
query12	182	127	127	127
query13	1252	459	337	337
query14	5664	3825	3573	3573
query14_1	2773	2794	2785	2785
query15	197	189	176	176
query16	975	475	457	457
query17	1083	692	594	594
query18	2421	436	340	340
query19	242	205	179	179
query20	133	126	133	126
query21	222	144	122	122
query22	4894	4935	4861	4861
query23	15976	15463	15339	15339
query23_1	15441	16217	15919	15919
query24	7271	1696	1295	1295
query24_1	1322	1284	1250	1250
query25	647	516	463	463
query26	1448	315	169	169
query27	2951	554	306	306
query28	4735	1934	1921	1921
query29	942	676	478	478
query30	310	254	218	218
query31	1347	1293	1252	1252
query32	81	69	72	69
query33	505	332	267	267
query34	932	934	568	568
query35	647	693	599	599
query36	1076	1130	998	998
query37	132	90	82	82
query38	2917	2928	2881	2881
query39	993	886	848	848
query39_1	838	809	830	809
query40	233	151	138	138
query41	62	60	61	60
query42	291	293	303	293
query43	236	263	214	214
query44 query45	202	191	191	191
query46	880	982	626	626
query47	2102	2130	2043	2043
query48	311	312	225	225
query49	633	454	380	380
query50	674	275	214	214
query51	4134	4143	4051	4051
query52	286	293	275	275
query53	283	335	280	280
query54	307	269	260	260
query55	88	94	84	84
query56	321	319	317	317
query57	1375	1365	1297	1297
query58	301	277	316	277
query59	1287	1447	1319	1319
query60	343	338	322	322
query61	157	151	149	149
query62	612	599	543	543
query63	308	285	273	273
query64	5141	1260	1043	1043
query65 query66	1460	472	351	351
query67	16368	16584	16177	16177
query68 query69	391	308	292	292
query70	1006	987	940	940
query71	337	313	301	301
query72	2888	2882	2727	2727
query73	534	548	328	328
query74	9985	9916	9740	9740
query75	2873	2766	2477	2477
query76	2302	1033	685	685
query77	363	381	318	318
query78	11312	11464	10710	10710
query79	1155	781	604	604
query80	1459	635	580	580
query81	580	282	250	250
query82	1004	158	122	122
query83	378	271	250	250
query84	252	125	99	99
query85	1285	565	442	442
query86	418	311	295	295
query87	3210	3071	2998	2998
query88	3601	2691	2670	2670
query89	432	369	357	357
query90	1885	176	177	176
query91	178	161	136	136
query92	81	73	74	73
query93	1135	838	515	515
query94	641	327	311	311
query95	595	399	319	319
query96	632	530	229	229
query97	2513	2518	2413	2413
query98	240	221	220	220
query99	1015	1002	917	917
Total cold run time: 233711 ms
Total hot run time: 153332 ms

doris-robot · 2026-03-12T16:37:26Z

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.70% (19722/37426)
Line Coverage	36.24% (184131/508095)
Region Coverage	32.34% (142037/439177)
Branch Coverage	33.50% (62021/185115)

hello-stephen · 2026-03-12T17:07:49Z

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.35% (26869/36631)
Line Coverage	56.69% (287063/506373)
Region Coverage	54.08% (239708/443225)
Branch Coverage	55.71% (103395/185597)

Yukang-Lian · 2026-03-13T07:07:12Z

run p0

Yukang-Lian · 2026-03-13T07:09:03Z

run nonConcurrent

Yukang-Lian · 2026-03-13T08:46:25Z

run p0

Yukang-Lian · 2026-03-13T08:46:30Z

run nonConcurrent

hello-stephen · 2026-03-13T10:53:54Z

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.35% (26868/36631)
Line Coverage	56.69% (287080/506373)
Region Coverage	54.09% (239759/443225)
Branch Coverage	55.71% (103393/185597)

hello-stephen · 2026-03-13T11:00:08Z

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.35% (26870/36631)
Line Coverage	56.70% (287103/506373)
Region Coverage	54.12% (239861/443225)
Branch Coverage	55.72% (103415/185597)

Yukang-Lian · 2026-03-16T07:20:00Z

run buildall

doris-robot · 2026-03-16T08:59:34Z

TPC-H: Total hot run time: 27084 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 827471cc17d36d9670cc0d37b3131658451c19c8, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17614	4486	4350	4350
q2	q3	10648	846	529	529
q4	4672	353	252	252
q5	7648	1228	1031	1031
q6	174	172	145	145
q7	820	850	672	672
q8	9804	1475	1345	1345
q9	5566	4788	4728	4728
q10	6328	1966	1686	1686
q11	470	247	236	236
q12	751	590	464	464
q13	18050	2936	2193	2193
q14	227	233	216	216
q15	q16	741	750	667	667
q17	728	877	401	401
q18	6039	5335	5255	5255
q19	1286	996	610	610
q20	537	507	373	373
q21	4569	1890	1642	1642
q22	398	357	289	289
Total cold run time: 97070 ms
Total hot run time: 27084 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4721	4540	4646	4540
q2	q3	3909	4338	3859	3859
q4	880	1206	803	803
q5	4074	4467	4343	4343
q6	183	174	142	142
q7	1746	1739	1585	1585
q8	2612	2694	2581	2581
q9	7533	7537	7199	7199
q10	3760	3959	3613	3613
q11	532	443	424	424
q12	510	600	452	452
q13	2665	3665	2379	2379
q14	288	309	283	283
q15	q16	744	878	812	812
q17	1189	1399	1347	1347
q18	7340	7137	6695	6695
q19	898	882	865	865
q20	2026	2141	1992	1992
q21	3988	3476	3291	3291
q22	461	421	410	410
Total cold run time: 50059 ms
Total hot run time: 47615 ms

doris-robot · 2026-03-16T09:10:19Z

TPC-DS: Total hot run time: 168774 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 827471cc17d36d9670cc0d37b3131658451c19c8, data reload: false
query5	4321	635	511	511
query6	326	224	200	200
query7	4216	464	268	268
query8	372	249	239	239
query9	8696	2731	2760	2731
query10	537	387	359	359
query11	6946	5079	4895	4895
query12	185	129	124	124
query13	1271	454	348	348
query14	5731	3717	3453	3453
query14_1	2873	2799	2803	2799
query15	208	199	177	177
query16	986	463	446	446
query17	912	728	617	617
query18	2439	455	352	352
query19	218	211	190	190
query20	135	132	134	132
query21	216	136	114	114
query22	13220	13882	14926	13882
query23	16214	15968	15599	15599
query23_1	15762	15997	15877	15877
query24	7226	1615	1222	1222
query24_1	1225	1261	1196	1196
query25	554	447	397	397
query26	1240	254	149	149
query27	2800	469	291	291
query28	4459	1871	1926	1871
query29	830	556	469	469
query30	296	225	189	189
query31	1015	944	863	863
query32	77	67	65	65
query33	505	324	278	278
query34	884	879	531	531
query35	631	693	603	603
query36	1083	1141	975	975
query37	130	93	80	80
query38	2945	2982	2898	2898
query39	850	834	831	831
query39_1	804	811	802	802
query40	232	157	134	134
query41	61	61	62	61
query42	266	260	254	254
query43	251	240	230	230
query44 query45	198	186	181	181
query46	871	971	594	594
query47	2094	2536	2522	2522
query48	323	316	231	231
query49	629	455	386	386
query50	681	272	210	210
query51	4027	4085	4030	4030
query52	264	265	256	256
query53	295	332	277	277
query54	307	274	256	256
query55	93	89	89	89
query56	319	335	299	299
query57	1915	1893	1637	1637
query58	282	278	269	269
query59	2776	2947	2749	2749
query60	348	344	323	323
query61	152	140	147	140
query62	617	585	540	540
query63	315	284	273	273
query64	5038	1272	987	987
query65 query66	1471	451	349	349
query67	24232	24288	24248	24248
query68 query69	422	318	294	294
query70	965	957	936	936
query71	334	334	298	298
query72	2810	2833	2395	2395
query73	559	546	320	320
query74	9632	9564	9419	9419
query75	2833	2754	2456	2456
query76	2289	1018	678	678
query77	356	371	307	307
query78	10965	11139	10478	10478
query79	1112	847	571	571
query80	1308	617	569	569
query81	561	265	227	227
query82	1023	149	119	119
query83	349	277	260	260
query84	296	122	111	111
query85	886	490	436	436
query86	423	322	296	296
query87	3170	3115	3038	3038
query88	3534	2667	2665	2665
query89	422	377	343	343
query90	2022	176	175	175
query91	168	159	136	136
query92	81	68	67	67
query93	924	869	494	494
query94	647	322	278	278
query95	588	421	316	316
query96	648	506	236	236
query97	2459	2457	2391	2391
query98	231	231	224	224
query99	1017	980	911	911
Total cold run time: 248886 ms
Total hot run time: 168774 ms

hello-stephen · 2026-03-16T11:13:43Z

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.68% (19718/37428)
Line Coverage	36.26% (184352/508437)
Region Coverage	32.44% (142295/438657)
Branch Coverage	33.59% (62180/185090)

hello-stephen · 2026-03-16T11:17:52Z

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.59% (26226/36634)
Line Coverage	54.41% (275698/506724)
Region Coverage	51.65% (228652/442706)
Branch Coverage	53.06% (98472/185572)

Yukang-Lian · 2026-03-17T10:17:47Z

run buildall

doris-robot · 2026-03-17T10:42:33Z

TPC-H: Total hot run time: 26743 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 98b10a25524479f1293d8949e4bb6ac412c95111, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17656	4399	4303	4303
q2	q3	10650	781	518	518
q4	4683	357	250	250
q5	7539	1221	1030	1030
q6	179	178	148	148
q7	803	835	697	697
q8	9315	1459	1327	1327
q9	4845	4733	4679	4679
q10	6287	1913	1627	1627
q11	489	254	235	235
q12	791	586	475	475
q13	18069	2955	2160	2160
q14	227	252	217	217
q15	q16	735	749	672	672
q17	735	819	473	473
q18	6020	5494	5253	5253
q19	1220	997	623	623
q20	531	494	377	377
q21	4508	1839	1400	1400
q22	368	504	279	279
Total cold run time: 95650 ms
Total hot run time: 26743 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4912	4758	4610	4610
q2	q3	3847	4368	3863	3863
q4	888	1224	780	780
q5	4081	4434	4328	4328
q6	180	168	135	135
q7	1779	1672	1514	1514
q8	2477	2804	2541	2541
q9	7593	7507	7351	7351
q10	3731	3955	3678	3678
q11	533	490	408	408
q12	508	593	442	442
q13	2869	3182	2358	2358
q14	290	304	288	288
q15	q16	714	783	735	735
q17	1143	1490	1340	1340
q18	7286	6932	6658	6658
q19	897	882	891	882
q20	2039	2177	2041	2041
q21	3988	3479	3298	3298
q22	490	541	487	487
Total cold run time: 50245 ms
Total hot run time: 47737 ms

hello-stephen · 2026-04-09T10:16:02Z

BE Regression && UT Coverage Report

Increment line coverage 12.90% (4/31) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.95% (26757/37186)
Line Coverage	54.94% (283522/516078)
Region Coverage	51.91% (234495/451762)
Branch Coverage	53.38% (101451/190056)

Yukang-Lian · 2026-04-14T07:44:55Z

run buildall

hello-stephen · 2026-04-14T08:56:25Z

BE UT Coverage Report

Increment line coverage 22.58% (7/31) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.06% (20146/37969)
Line Coverage	36.61% (189545/517679)
Region Coverage	32.85% (147062/447671)
Branch Coverage	34.00% (64404/189420)

hello-stephen · 2026-04-14T10:11:41Z

BE Regression && UT Coverage Report

Increment line coverage 22.58% (7/31) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.61% (27369/37183)
Line Coverage	57.28% (295638/516095)
Region Coverage	54.30% (245322/451803)
Branch Coverage	56.08% (106557/190002)

Yukang-Lian · 2026-04-16T03:27:48Z

run nonconcurrent

Yukang-Lian · 2026-04-16T03:28:03Z

/review

github-actions

I found blocking issues in the new single-tablet full-compaction path.

The handler now calls StorageEngine::submit_compaction_task() synchronously. On the local engine that helper runs FullCompaction::prepare_compact() on the HTTP thread, and prepare_compact() blocks on the base/cumulative compaction locks. If another compaction is already running, /api/compaction/run?tablet_id=...&compact_type=full can now hang until it finishes, whereas the old detached-thread path returned after at most 2 seconds.
The new path also inherits a latent submit-status bug in StorageEngine::_submit_compaction_task(): after thread_pool->submit_func(...), the code checks st instead of status, so enqueue failures are silently treated as success and leak the submitted-state bookkeeping. Routing single-tablet full compaction through this helper makes that behavior user-visible.

Critical Checkpoints

Goal / correctness: The goal is to move single-tablet manual full compaction onto the managed thread-pool path and add force. The refactor is incomplete because the new path changes handler responsiveness and can falsely report success on enqueue failure.
Minimality / focus: The code change is focused, but it pulls in existing submit_compaction_task() semantics that do not match the old single-tablet API contract.
Concurrency: Applicable. The change crosses the HTTP worker thread, the base-compaction pool, and the tablet compaction locks. Lock acquisition for full compaction is no longer offloaded from the request thread, which creates a regression.
Lifecycle / static init: No special lifecycle or static-initialization issue found in the patch itself.
Configs: No new configuration items.
Parallel paths: Applicable. Table-level local full compaction already uses the thread-pool path, but the old single-tablet path had different responsiveness and error semantics. The patch aligns implementations without preserving those semantics.
Conditions / special checks: The new force parsing is straightforward; no issue there.
Tests: Coverage is insufficient for the changed behavior. The added UTs only cover parameter parsing / not-found cases, not actual thread-pool submission, lock contention, or enqueue-failure behavior.
Observability: The thread-pool path keeps metrics/tracker integration, which is good.
Transaction / persistence: Not applicable.
Data writes / atomicity: No new storage-format or persistence change reviewed here.
FE/BE variable passing: Not applicable.
Performance / resource usage: Blocking the HTTP worker on compaction locks is a regression.
Other issues: No additional blocking issues beyond the inline comments.

Yukang-Lian · 2026-04-17T02:23:38Z

run nonconcurrent

hello-stephen · 2026-04-17T02:29:36Z

run nonConcurrent

hello-stephen · 2026-04-17T04:25:03Z

BE Regression && UT Coverage Report

Increment line coverage 22.58% (7/31) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.60% (27365/37183)
Line Coverage	57.27% (295552/516095)
Region Coverage	54.28% (245235/451803)
Branch Coverage	56.06% (106517/190002)

dismiss

github-actions · 2026-04-17T06:44:12Z

PR approved by at least one committer and no changes requested.

Cherry-pick the following PRs to `branch-4.1`: - #61696 [Feature](compaction) add CompactionTaskTracker with system table and HTTP API - #61621 [fix](metrics) Fix prepared statement QPS metrics not counted when audit log disabled - #61222 [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread - #62596 [feat](compaction) Support ADMIN COMPACT TABLE type='FULL' and enable it in cloud mode Conflicts resolved for #61696 (schema table enum renumbering on branch-4.1; `be_compaction_tasks` uses id 67) and #61621 (AuditLogHelper context + QueueToken import).

…ool instead of detached thread (apache#61222) Previously, manually triggering full compaction on a single tablet via HTTP API would create a detached thread for each request, which lacks concurrency control, deduplication, and metrics tracking. This PR changes the single-tablet full compaction path to use `submit_compaction_task()`, submitting to the base compaction thread pool — consistent with the table-level path and the cloud engine. Base and cumulative compaction remain unchanged. Added `force` HTTP parameter to skip permit limiter when submitting full compaction, allowing compaction to proceed even when resources are constrained. ### Usage ```bash # single tablet, default (with permit limiter) curl -X POST "http://be_host:http_port/api/compaction/run?tablet_id=12345&compact_type=full" # single tablet, force (skip permit limiter) curl -X POST "http://be_host:http_port/api/compaction/run?tablet_id=12345&compact_type=full&force=true" # table level curl -X POST "http://be_host:http_port/api/compaction/run?table_id=67890&compact_type=full" # table level, force curl -X POST "http://be_host:http_port/api/compaction/run?table_id=67890&compact_type=full&force=true" ```

Yukang-Lian force-pushed the fix/manual-compaction-use-threadpool branch 2 times, most recently from 002ec48 to 7ff50aeCompare March 12, 2026 12:23

Yukang-Lian force-pushed the fix/manual-compaction-use-threadpool branch from 7ff50ae to 827471cCompare March 16, 2026 07:19

[test](compaction) add force parameter unit tests for CompactionAction
522bad5

Yukang-Lian force-pushed the fix/manual-compaction-use-threadpool branch from 4de9c7c to 522bad5Compare April 14, 2026 07:44

github-actionsBot previously requested changes Apr 16, 2026
View reviewed changes

Comment threadbe/src/service/http/action/compaction_action.cpp
Comment threadbe/src/service/http/action/compaction_action.cpp

hello-stephen closed this Apr 17, 2026

hello-stephen reopened this Apr 17, 2026

luwei16 approved these changes Apr 17, 2026
View reviewed changes

luwei16 merged commit 4640c70 into apache:masterApr 17, 2026
46 of 48 checks passed

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Apr 17, 2026

github-actionsBot added the dev/4.1.x-conflict label Apr 17, 2026

mymeiyi mentioned this pull request Apr 21, 2026
[fix](wal) Fix WAL replay loading 0 rows due to columns being marked as missing #62658
Merged

This was referenced Apr 22, 2026
branch-4.1: [feat](compaction) Support ADMIN COMPACT TABLE type='FULL' and enable it in cloud mode #62596 #62675
Closed
branch-4.1: pick #61696 #61621 #61222 #62596 #62707
Merged

yiguolei added dev/4.1.1-merged and removed dev/4.1.x dev/4.1.x-conflict labels Apr 23, 2026

yiguolei mentioned this pull request May 20, 2026
4.1.1 Release Notes #63426
Closed

Yukang-Lian mentioned this pull request Jun 12, 2026
[fix](test) fix flaky compaction profile case #63886
Merged

Uh oh!

Conversation

Yukang-Lian commented Mar 11, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Usage

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Yukang-Lian commented Mar 11, 2026

Uh oh!

Thearas commented Mar 11, 2026

Uh oh!

Yukang-Lian commented Mar 11, 2026

Uh oh!

Yukang-Lian commented Mar 11, 2026

Uh oh!

doris-robot commented Mar 11, 2026

Uh oh!

doris-robot commented Mar 11, 2026

Uh oh!

doris-robot commented Mar 11, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Mar 11, 2026

BE Regression && UT Coverage Report

Uh oh!

Yukang-Lian commented Mar 12, 2026

Uh oh!

Yukang-Lian commented Mar 12, 2026

Uh oh!

doris-robot commented Mar 12, 2026

Uh oh!

doris-robot commented Mar 12, 2026

Uh oh!

doris-robot commented Mar 12, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Mar 12, 2026

BE Regression && UT Coverage Report

Uh oh!

Yukang-Lian commented Mar 13, 2026

Uh oh!

Yukang-Lian commented Mar 13, 2026

Uh oh!

Yukang-Lian commented Mar 13, 2026

Uh oh!

Yukang-Lian commented Mar 13, 2026

Uh oh!

hello-stephen commented Mar 13, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Mar 13, 2026

BE Regression && UT Coverage Report

Uh oh!

Yukang-Lian commented Mar 16, 2026

Uh oh!

doris-robot commented Mar 16, 2026

Uh oh!

doris-robot commented Mar 16, 2026

Uh oh!

hello-stephen commented Mar 16, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Mar 16, 2026

BE Regression && UT Coverage Report

Uh oh!

Yukang-Lian commented Mar 17, 2026

Uh oh!

doris-robot commented Mar 17, 2026

Uh oh!

hello-stephen commented Apr 9, 2026

BE Regression && UT Coverage Report

Uh oh!

Yukang-Lian commented Apr 14, 2026

Uh oh!

hello-stephen commented Apr 14, 2026

BE UT Coverage Report

Uh oh!

Yukang-Lian commented Mar 11, 2026•
edited
Loading