[enhancement](compaction) Control the parallelism for urgent compaction tasks by Hastyshell · Pull Request #37782 · apache/doris

Hastyshell · 2024-07-15T03:54:09Z

Proposed changes

For some urgent compaction tasks, their submittion should take parallelism into account.

Currently, we apply the control policy for data loading in specific. Other source of urgent tasks are considered as eager.

…on tasks

doris-robot · 2024-07-15T03:54:13Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Hastyshell · 2024-07-15T03:54:38Z

run buildall

Hastyshell · 2024-07-15T03:59:38Z

run buildall

github-actions · 2024-07-15T04:00:33Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-15T04:05:06Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-15T04:10:05Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-15T04:19:12Z

TPC-H: Total hot run time: 40273 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6df63cc92e7893428f939114fcd0d483ff4715f5, data reload: false
------ Round 1 ----------------------------------
q1	18213	4505	4377	4377
q2	2977	196	193	193
q3	11065	1211	1276	1211
q4	10281	849	721	721
q5	7617	2749	2706	2706
q6	228	145	143	143
q7	1000	641	611	611
q8	9377	2088	2094	2088
q9	8660	6537	6552	6537
q10	9045	3837	3863	3837
q11	463	255	264	255
q12	405	235	223	223
q13	17750	3008	3055	3008
q14	284	238	240	238
q15	527	483	491	483
q16	500	384	378	378
q17	976	774	676	676
q18	7994	7509	7470	7470
q19	4616	1356	1291	1291
q20	659	332	334	332
q21	5007	3208	3303	3208
q22	358	294	287	287
Total cold run time: 118002 ms
Total hot run time: 40273 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4382	4238	4267	4238
q2	366	291	266	266
q3	3016	2813	2759	2759
q4	1941	1601	1618	1601
q5	5387	5323	5342	5323
q6	217	136	133	133
q7	2142	1771	1766	1766
q8	3211	3338	3316	3316
q9	8466	8370	8490	8370
q10	3903	3730	3673	3673
q11	609	502	498	498
q12	769	584	582	582
q13	17289	3002	2976	2976
q14	308	278	277	277
q15	523	488	474	474
q16	467	413	439	413
q17	1760	1486	1473	1473
q18	7665	7637	7298	7298
q19	1694	1566	1517	1517
q20	1987	1789	1772	1772
q21	4933	4692	4545	4545
q22	628	487	494	487
Total cold run time: 71663 ms
Total hot run time: 53757 ms

doris-robot · 2024-07-15T04:30:33Z

TPC-DS: Total hot run time: 173381 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6df63cc92e7893428f939114fcd0d483ff4715f5, data reload: false
query1	909	366	367	366
query2	6449	1859	1777	1777
query3	6682	207	219	207
query4	23045	17519	17429	17429
query5	4195	512	517	512
query6	260	167	159	159
query7	4591	304	292	292
query8	243	198	201	198
query9	8755	2463	2476	2463
query10	423	291	273	273
query11	11283	10069	10370	10069
query12	128	88	82	82
query13	1632	376	364	364
query14	9433	6750	7915	6750
query15	210	167	167	167
query16	7801	328	319	319
query17	1716	544	530	530
query18	1811	266	267	266
query19	192	149	147	147
query20	89	81	82	81
query21	215	142	131	131
query22	4473	4305	3987	3987
query23	33864	33276	33273	33273
query24	12245	2870	2965	2870
query25	654	371	376	371
query26	1867	155	150	150
query27	3086	280	277	277
query28	7837	2091	2071	2071
query29	1164	635	614	614
query30	287	148	145	145
query31	996	742	765	742
query32	97	53	54	53
query33	778	302	299	299
query34	993	512	497	497
query35	669	580	589	580
query36	1088	949	924	924
query37	286	81	79	79
query38	2869	2716	2764	2716
query39	891	832	828	828
query40	281	134	127	127
query41	51	54	47	47
query42	118	101	105	101
query43	505	470	505	470
query44	1278	763	750	750
query45	198	166	165	165
query46	1085	740	754	740
query47	1846	1765	1767	1765
query48	384	308	308	308
query49	1206	436	443	436
query50	779	413	407	407
query51	6939	6872	6796	6796
query52	107	96	102	96
query53	357	305	298	298
query54	1041	474	490	474
query55	80	76	78	76
query56	325	297	319	297
query57	1166	1068	1044	1044
query58	258	271	275	271
query59	2817	2538	2601	2538
query60	333	310	305	305
query61	118	116	114	114
query62	837	665	659	659
query63	328	305	302	302
query64	10565	2348	1732	1732
query65	3200	3136	3109	3109
query66	1384	346	336	336
query67	15740	15159	15143	15143
query68	4597	581	602	581
query69	595	370	344	344
query70	1116	1134	1150	1134
query71	416	295	297	295
query72	7964	5762	5807	5762
query73	750	327	328	327
query74	6032	5646	5677	5646
query75	3909	2692	2675	2675
query76	2828	989	988	988
query77	694	328	332	328
query78	9618	9014	10527	9014
query79	6615	525	523	523
query80	2646	468	492	468
query81	589	221	222	221
query82	677	136	138	136
query83	305	165	175	165
query84	280	86	91	86
query85	2343	315	289	289
query86	486	306	310	306
query87	3303	3139	3175	3139
query88	4446	2496	2470	2470
query89	473	379	376	376
query90	2070	189	191	189
query91	132	100	140	100
query92	63	49	52	49
query93	3131	522	511	511
query94	1502	213	215	213
query95	407	319	314	314
query96	592	272	274	272
query97	3189	3033	3018	3018
query98	221	201	199	199
query99	1713	1309	1279	1279
Total cold run time: 291234 ms
Total hot run time: 173381 ms

doris-robot · 2024-07-15T04:36:05Z

ClickBench: Total hot run time: 30.72 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6df63cc92e7893428f939114fcd0d483ff4715f5, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.09
query5	0.50	0.49	0.48
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.55	0.50	0.50
query10	0.54	0.54	0.55
query11	0.15	0.12	0.12
query12	0.15	0.13	0.13
query13	0.59	0.59	0.58
query14	0.77	0.77	0.77
query15	0.84	0.82	0.83
query16	0.36	0.37	0.37
query17	0.96	0.96	1.01
query18	0.23	0.22	0.22
query19	1.82	1.73	1.83
query20	0.02	0.01	0.01
query21	15.39	0.76	0.67
query22	4.59	7.38	1.95
query23	18.30	1.30	1.23
query24	2.18	0.22	0.22
query25	0.16	0.08	0.08
query26	0.30	0.23	0.21
query27	0.46	0.24	0.23
query28	13.29	1.03	0.99
query29	12.68	3.36	3.31
query30	0.25	0.05	0.06
query31	2.90	0.39	0.38
query32	3.26	0.48	0.49
query33	2.90	2.91	2.95
query34	17.13	4.32	4.43
query35	4.44	4.44	4.44
query36	0.66	0.46	0.49
query37	0.20	0.15	0.15
query38	0.15	0.15	0.16
query39	0.04	0.04	0.04
query40	0.15	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.37 s
Total hot run time: 30.72 s

Hastyshell · 2024-07-15T04:38:04Z

run buildall

doris-robot · 2024-07-15T04:56:28Z

TPC-H: Total hot run time: 40142 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f2cb6de0dbb91d1f66efbb82f92a825eaddc7d1f, data reload: false
------ Round 1 ----------------------------------
q1	17617	4337	4284	4284
q2	2013	194	191	191
q3	10440	1235	1137	1137
q4	10207	827	811	811
q5	7547	2723	2697	2697
q6	223	140	140	140
q7	963	607	609	607
q8	9232	2093	2122	2093
q9	8677	6536	6575	6536
q10	8672	3839	3839	3839
q11	450	242	250	242
q12	398	238	234	234
q13	18933	3002	2985	2985
q14	269	240	241	240
q15	540	494	493	493
q16	486	398	382	382
q17	964	697	721	697
q18	8188	7485	7338	7338
q19	6703	1458	1417	1417
q20	722	329	335	329
q21	4979	3172	3296	3172
q22	361	293	278	278
Total cold run time: 118584 ms
Total hot run time: 40142 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4369	4294	4244	4244
q2	385	268	264	264
q3	2972	2816	2724	2724
q4	1933	1661	1631	1631
q5	5285	5351	5315	5315
q6	217	132	135	132
q7	2153	1723	1670	1670
q8	3202	3352	3282	3282
q9	8489	8412	8412	8412
q10	3982	3669	3676	3669
q11	603	499	515	499
q12	801	632	617	617
q13	17060	3012	3027	3012
q14	311	283	279	279
q15	515	476	475	475
q16	486	442	432	432
q17	1795	1502	1499	1499
q18	7748	7570	7465	7465
q19	1686	1532	1468	1468
q20	1980	1777	1815	1777
q21	4892	4822	5017	4822
q22	580	488	488	488
Total cold run time: 71444 ms
Total hot run time: 54176 ms

doris-robot · 2024-07-15T05:07:44Z

TPC-DS: Total hot run time: 173918 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f2cb6de0dbb91d1f66efbb82f92a825eaddc7d1f, data reload: false
query1	933	377	368	368
query2	6456	1789	1832	1789
query3	6650	210	225	210
query4	28416	17761	17597	17597
query5	4211	507	501	501
query6	273	171	196	171
query7	4607	301	294	294
query8	262	206	198	198
query9	8635	2433	2401	2401
query10	450	298	286	286
query11	10772	10221	10082	10082
query12	135	88	83	83
query13	1639	368	356	356
query14	10292	7779	8112	7779
query15	232	170	168	168
query16	7891	321	315	315
query17	1940	557	531	531
query18	1876	282	276	276
query19	199	151	150	150
query20	91	80	83	80
query21	211	127	122	122
query22	4352	3992	3985	3985
query23	33865	33154	33398	33154
query24	11203	2827	2830	2827
query25	647	371	363	363
query26	1400	152	147	147
query27	2823	275	275	275
query28	7212	2002	2023	2002
query29	936	647	614	614
query30	283	148	148	148
query31	947	738	769	738
query32	100	55	61	55
query33	775	290	295	290
query34	949	497	495	495
query35	698	587	559	559
query36	1096	968	945	945
query37	161	87	83	83
query38	2848	2781	2751	2751
query39	869	827	830	827
query40	271	122	119	119
query41	51	47	44	44
query42	124	98	107	98
query43	495	455	458	455
query44	1285	761	733	733
query45	191	167	163	163
query46	1094	731	736	731
query47	1862	1778	1739	1739
query48	373	298	299	298
query49	1069	419	415	415
query50	787	395	406	395
query51	6849	6770	6636	6636
query52	110	93	95	93
query53	371	315	301	301
query54	991	461	454	454
query55	78	74	78	74
query56	291	280	274	274
query57	1161	1050	1068	1050
query58	249	244	254	244
query59	2923	2563	2737	2563
query60	309	274	276	274
query61	100	95	97	95
query62	835	642	648	642
query63	325	289	299	289
query64	9686	2272	7155	2272
query65	3168	3120	3118	3118
query66	1296	337	332	332
query67	15347	15149	14941	14941
query68	4530	549	559	549
query69	486	334	339	334
query70	1122	1078	1193	1078
query71	390	292	280	280
query72	6997	5493	5476	5476
query73	758	328	331	328
query74	6112	5812	5768	5768
query75	3384	2706	2645	2645
query76	2719	969	944	944
query77	493	326	328	326
query78	9593	8990	9072	8990
query79	2141	531	525	525
query80	1824	481	480	480
query81	595	225	226	225
query82	779	142	137	137
query83	276	174	169	169
query84	258	94	87	87
query85	2138	312	300	300
query86	490	311	292	292
query87	3297	3093	3159	3093
query88	3706	2464	2443	2443
query89	475	405	393	393
query90	1876	195	197	195
query91	136	101	101	101
query92	62	51	53	51
query93	2516	531	520	520
query94	1244	218	215	215
query95	413	317	321	317
query96	613	284	273	273
query97	3236	3065	3054	3054
query98	227	211	201	201
query99	1545	1272	1279	1272
Total cold run time: 283457 ms
Total hot run time: 173918 ms

doris-robot · 2024-07-15T05:13:16Z

ClickBench: Total hot run time: 31.16 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f2cb6de0dbb91d1f66efbb82f92a825eaddc7d1f, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.22	0.05	0.05
query4	1.67	0.08	0.09
query5	0.50	0.49	0.49
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.56	0.57	0.56
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.58	0.59	0.59
query14	0.79	0.78	0.77
query15	0.85	0.81	0.82
query16	0.36	0.36	0.34
query17	1.00	0.97	0.95
query18	0.22	0.22	0.22
query19	1.79	1.69	1.69
query20	0.01	0.01	0.01
query21	15.42	0.76	0.67
query22	4.42	6.74	2.40
query23	18.33	1.39	1.30
query24	2.08	0.24	0.23
query25	0.15	0.09	0.08
query26	0.31	0.20	0.21
query27	0.45	0.23	0.23
query28	13.26	1.02	1.00
query29	12.61	3.32	3.31
query30	0.26	0.06	0.05
query31	2.87	0.38	0.39
query32	3.29	0.48	0.46
query33	2.89	2.91	2.88
query34	17.10	4.38	4.40
query35	4.44	4.42	4.39
query36	0.66	0.49	0.49
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.14	0.13	0.11
query41	0.10	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110 s
Total hot run time: 31.16 s

Hastyshell · 2024-07-15T06:30:12Z

run p0

gavinchou · 2024-07-15T07:14:22Z

+ copied_base_map = _tablet_submitted_base_compaction;
+ }
+ auto stores = get_stores();
+ bool is_busy = std::none_of(


too many indentions

github-actions · 2024-07-15T07:18:23Z

clang-tidy review says "All clean, LGTM! 👍"

Hastyshell · 2024-07-15T07:26:38Z

run buildall

github-actions · 2024-07-15T07:30:00Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-15T07:31:15Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-15T07:44:04Z

TPC-H: Total hot run time: 40006 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8d32cb2c346eff8e769d98d5733a60684f63ed91, data reload: false
------ Round 1 ----------------------------------
q1	18144	4458	4362	4362
q2	2860	195	202	195
q3	11004	1197	1071	1071
q4	10530	841	821	821
q5	7864	2742	2702	2702
q6	230	143	140	140
q7	972	605	601	601
q8	9770	2086	2046	2046
q9	8590	6542	6544	6542
q10	8768	3776	3789	3776
q11	440	251	236	236
q12	395	217	219	217
q13	17760	3008	2979	2979
q14	274	232	231	231
q15	526	480	502	480
q16	480	384	376	376
q17	963	625	662	625
q18	8058	7596	7406	7406
q19	4853	1369	1363	1363
q20	707	324	328	324
q21	4873	3231	3256	3231
q22	346	286	282	282
Total cold run time: 118407 ms
Total hot run time: 40006 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4365	4253	4256	4253
q2	377	256	257	256
q3	2964	2800	2773	2773
q4	1909	1610	1578	1578
q5	5312	5312	5303	5303
q6	226	133	134	133
q7	2121	1702	1701	1701
q8	3180	3369	3291	3291
q9	8412	8332	8323	8323
q10	3916	3661	3677	3661
q11	583	510	492	492
q12	784	614	628	614
q13	17411	2982	3033	2982
q14	312	262	266	262
q15	515	482	470	470
q16	489	417	429	417
q17	1780	1465	1448	1448
q18	7724	7403	7580	7403
q19	2649	1591	1676	1591
q20	1983	1801	1800	1800
q21	4768	4708	4755	4708
q22	583	488	492	488
Total cold run time: 72363 ms
Total hot run time: 53947 ms

doris-robot · 2024-07-15T07:55:19Z

TPC-DS: Total hot run time: 172959 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8d32cb2c346eff8e769d98d5733a60684f63ed91, data reload: false
query1	902	372	357	357
query2	6472	1919	1871	1871
query3	6652	211	221	211
query4	28404	17632	17063	17063
query5	4222	498	478	478
query6	275	165	156	156
query7	4586	294	293	293
query8	237	191	185	185
query9	8503	2383	2372	2372
query10	452	289	266	266
query11	10676	10103	10185	10103
query12	127	80	80	80
query13	1643	375	363	363
query14	10269	8063	7707	7707
query15	226	164	171	164
query16	7553	310	306	306
query17	1809	547	516	516
query18	1572	277	267	267
query19	193	157	146	146
query20	91	82	81	81
query21	209	123	125	123
query22	4393	4173	3962	3962
query23	33840	33081	33314	33081
query24	12088	2890	2809	2809
query25	622	365	392	365
query26	1672	151	147	147
query27	2758	266	271	266
query28	7129	1986	1985	1985
query29	911	627	601	601
query30	291	150	148	148
query31	979	726	742	726
query32	102	55	55	55
query33	764	308	287	287
query34	954	474	473	473
query35	684	568	573	568
query36	1126	935	924	924
query37	248	82	82	82
query38	2870	2760	2711	2711
query39	899	842	802	802
query40	281	126	124	124
query41	49	48	84	48
query42	112	104	97	97
query43	505	478	472	472
query44	1157	739	732	732
query45	191	158	160	158
query46	1075	723	711	711
query47	1852	1744	1765	1744
query48	367	293	290	290
query49	1190	407	400	400
query50	772	398	389	389
query51	6886	6851	6793	6793
query52	110	92	99	92
query53	355	295	300	295
query54	998	455	445	445
query55	79	75	75	75
query56	284	276	268	268
query57	1144	1059	1067	1059
query58	272	272	254	254
query59	3094	2757	3000	2757
query60	307	273	281	273
query61	95	95	102	95
query62	826	655	663	655
query63	324	292	292	292
query64	10532	2215	1747	1747
query65	3185	3100	3098	3098
query66	1314	347	326	326
query67	15362	15119	14769	14769
query68	4613	549	534	534
query69	480	328	317	317
query70	1090	1122	1098	1098
query71	392	274	274	274
query72	7122	5741	5634	5634
query73	745	329	322	322
query74	6033	5731	5664	5664
query75	3483	2666	2688	2666
query76	2307	939	903	903
query77	474	312	315	312
query78	10399	10390	9235	9235
query79	2772	513	513	513
query80	1423	515	542	515
query81	569	228	225	225
query82	701	145	136	136
query83	276	166	162	162
query84	279	91	91	91
query85	1790	322	305	305
query86	489	332	315	315
query87	3452	3311	3225	3225
query88	4555	2509	2478	2478
query89	467	372	373	372
query90	1859	199	197	197
query91	132	102	104	102
query92	58	47	49	47
query93	3947	514	511	511
query94	1262	216	212	212
query95	414	313	312	312
query96	613	274	271	271
query97	3177	3036	3022	3022
query98	217	205	196	196
query99	1590	1283	1235	1235
Total cold run time: 287301 ms
Total hot run time: 172959 ms

doris-robot · 2024-07-15T08:00:51Z

ClickBench: Total hot run time: 30.47 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8d32cb2c346eff8e769d98d5733a60684f63ed91, data reload: false
query1	0.04	0.04	0.02
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.07	0.08
query5	0.50	0.47	0.49
query6	1.15	0.73	0.73
query7	0.01	0.01	0.02
query8	0.06	0.05	0.05
query9	0.56	0.50	0.49
query10	0.55	0.55	0.54
query11	0.14	0.11	0.11
query12	0.15	0.13	0.14
query13	0.60	0.59	0.59
query14	0.78	0.76	0.79
query15	0.84	0.81	0.81
query16	0.37	0.37	0.37
query17	1.04	1.04	1.00
query18	0.22	0.21	0.21
query19	1.92	1.82	1.76
query20	0.01	0.01	0.01
query21	15.43	0.74	0.64
query22	4.72	7.17	1.70
query23	18.27	1.29	1.36
query24	2.15	0.22	0.22
query25	0.14	0.08	0.09
query26	0.29	0.21	0.20
query27	0.45	0.23	0.22
query28	13.34	1.01	1.00
query29	12.59	3.35	3.32
query30	0.25	0.06	0.05
query31	2.86	0.40	0.38
query32	3.28	0.48	0.47
query33	2.89	2.87	2.99
query34	17.04	4.37	4.35
query35	4.40	4.42	4.42
query36	0.66	0.47	0.49
query37	0.19	0.15	0.15
query38	0.15	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.12	0.12
query41	0.09	0.05	0.04
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.38 s
Total hot run time: 30.47 s

github-actions · 2024-07-16T06:41:43Z

PR approved by anyone and no changes requested.

dataroaring

LGTM

github-actions · 2024-07-17T10:02:00Z

PR approved by at least one committer and no changes requested.

…on tasks (#37782) ## Proposed changes For some urgent compaction tasks, their submittion should take parallelism into account. Currently, we apply the control policy for data loading in specific. Other source of urgent tasks are considered as eager.

…on tasks (apache#37782) For some urgent compaction tasks, their submittion should take parallelism into account. Currently, we apply the control policy for data loading in specific. Other source of urgent tasks are considered as eager.

…n tasks (#37782) (#38189) ## Proposed changes For some urgent compaction tasks, their submittion should take parallelism into account. Currently, we apply the control policy for data loading in specific. Other source of urgent tasks are considered as eager.

…on tasks (apache#37782) ## Proposed changes For some urgent compaction tasks, their submittion should take parallelism into account. Currently, we apply the control policy for data loading in specific. Other source of urgent tasks are considered as eager.

[enhancement](compaction) control the parallelism for urgent compacti…
c0c91e1
…on tasks

Hastyshell marked this pull request as draft July 15, 2024 03:54

format
6df63cc

format
f2cb6de

Hastyshell marked this pull request as ready for review July 15, 2024 04:56

gavinchou reviewed Jul 15, 2024
View reviewed changes

Hastyshell force-pushed the urgent-compaction-paral-cntl branch from b29fefc to 7327080Compare July 15, 2024 07:24

log every n to indicate degree of busyness
8d32cb2

Hastyshell force-pushed the urgent-compaction-paral-cntl branch from 7327080 to 8d32cb2Compare July 15, 2024 07:25

luwei16 approved these changes Jul 16, 2024
View reviewed changes

github-actionsBot added the reviewed label Jul 16, 2024

dataroaring approved these changes Jul 17, 2024
View reviewed changes

dataroaring added the dev/2.1.x label Jul 17, 2024

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 17, 2024

dataroaring merged commit def75ea into apache:masterJul 18, 2024

dataroaring mentioned this pull request Jul 22, 2024
[enhancement](compaction) Control the parallelism for urgent compacton tasks (#37782) #38189
Merged

dataroaring added dev/2.1.6-merged and removed dev/2.1.x labels Jul 22, 2024

dataroaring added dev/3.0.1-merged dev/2.1.6-merged and removed dev/2.1.6-merged labels Jul 31, 2024

gavinchou mentioned this pull request Aug 19, 2024
Release Note 3.0.1 #39570
Closed

Uh oh!

Conversation

Hastyshell commented Jul 15, 2024

Proposed changes

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

Hastyshell commented Jul 15, 2024

Uh oh!

Hastyshell commented Jul 15, 2024

Uh oh!

github-actionsBot commented Jul 15, 2024

Uh oh!

github-actionsBot commented Jul 15, 2024

Uh oh!

github-actionsBot commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

Hastyshell commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

Hastyshell commented Jul 15, 2024

Uh oh!

gavinchouJul 15, 2024

Choose a reason for hiding this comment

Uh oh!

HastyshellJul 15, 2024

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 15, 2024

Uh oh!

Hastyshell commented Jul 15, 2024

Uh oh!

github-actionsBot commented Jul 15, 2024

Uh oh!

github-actionsBot commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

doris-robot commented Jul 15, 2024

Uh oh!

github-actionsBot commented Jul 16, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 17, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants