[fix](recycler) Implement missing delete_prefix() of hdfs accessor by gavinchou · Pull Request #40215 · apache/doris

gavinchou · 2024-08-31T08:54:47Z

To test it,

enable commit_rowset fail with injection point
issue a streamload
check the recycler log or storage vault the uncommitted rowset recycled

curl 'be_ip:http_port/api/injection_point/enable'
curl 'be_ip:http_port/api/injection_point/clear'
curl "be_ip:http_port/api/injection_point/set?name=CloudMetaMgr::commit_rowset&behavior=return_error"
curl -XPUT --location-trusted -vv -T tmp/1.csv -H'format:csv' -H 'column_separator:,' -H'timeout:60' -uroot: be_ip:http_port/api/db1/t2/_stream_load
curl 'be_ip:http_port/api/injection_point/clear'
curl 'be_ip:http_port/api/injection_point/disable'

doris-robot · 2024-08-31T08:54:52Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions · 2024-08-31T09:01:32Z

clang-tidy review says "All clean, LGTM! 👍"

gavinchou · 2024-08-31T09:06:07Z

run buildall

doris-robot · 2024-08-31T09:26:49Z

TPC-H: Total hot run time: 37972 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 21db926afb4516aa8ef28db219082598ace4e50e, data reload: false
------ Round 1 ----------------------------------
q1	17915	4546	4373	4373
q2	2940	200	175	175
q3	11282	1140	1069	1069
q4	10249	687	842	687
q5	8202	2853	2782	2782
q6	237	139	141	139
q7	984	635	616	616
q8	9341	2054	2082	2054
q9	7057	6547	6537	6537
q10	7031	2203	2249	2203
q11	504	248	242	242
q12	398	227	219	219
q13	17903	3025	3008	3008
q14	290	239	234	234
q15	530	485	488	485
q16	596	505	515	505
q17	982	785	711	711
q18	7314	6951	6860	6860
q19	1394	960	904	904
q20	679	337	348	337
q21	3928	3066	2834	2834
q22	1115	998	1005	998
Total cold run time: 110871 ms
Total hot run time: 37972 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4381	4301	4325	4301
q2	377	286	270	270
q3	2909	2664	2654	2654
q4	1950	1666	1717	1666
q5	5423	5406	5398	5398
q6	222	130	130	130
q7	2102	1764	1749	1749
q8	3220	3365	3369	3365
q9	8459	8559	8440	8440
q10	3465	3201	3231	3201
q11	601	506	522	506
q12	811	588	634	588
q13	10638	3087	3030	3030
q14	321	275	277	275
q15	535	481	488	481
q16	581	567	539	539
q17	1784	1500	1450	1450
q18	7709	7450	7496	7450
q19	1664	1517	1613	1517
q20	2066	1844	1832	1832
q21	5658	5405	5268	5268
q22	1134	1064	1015	1015
Total cold run time: 66010 ms
Total hot run time: 55125 ms

doris-robot · 2024-08-31T09:38:30Z

TPC-DS: Total hot run time: 188676 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 21db926afb4516aa8ef28db219082598ace4e50e, data reload: false
query1	934	375	364	364
query2	6467	2102	1994	1994
query3	6651	206	215	206
query4	34489	23151	23250	23151
query5	4180	510	484	484
query6	253	172	177	172
query7	4590	314	292	292
query8	258	217	215	215
query9	8484	2499	2492	2492
query10	434	275	260	260
query11	15423	15309	15295	15295
query12	150	101	98	98
query13	1628	378	370	370
query14	10168	7252	7192	7192
query15	262	169	189	169
query16	7913	442	451	442
query17	1581	593	541	541
query18	1971	284	284	284
query19	229	143	142	142
query20	116	110	112	110
query21	212	105	102	102
query22	4348	4117	4328	4117
query23	34180	33719	33454	33454
query24	11131	2818	2833	2818
query25	625	378	382	378
query26	1201	161	160	160
query27	2347	283	281	281
query28	7421	2145	2110	2110
query29	808	415	407	407
query30	310	151	153	151
query31	987	757	790	757
query32	104	58	61	58
query33	788	286	284	284
query34	991	468	500	468
query35	855	760	718	718
query36	1093	920	933	920
query37	154	90	100	90
query38	3984	3928	3954	3928
query39	1481	1365	1378	1365
query40	205	117	117	117
query41	48	44	44	44
query42	113	94	99	94
query43	513	488	489	488
query44	1272	762	741	741
query45	210	169	169	169
query46	1101	718	737	718
query47	1888	1792	1791	1791
query48	384	283	285	283
query49	1085	438	430	430
query50	805	405	419	405
query51	7189	7055	6991	6991
query52	100	88	87	87
query53	272	191	185	185
query54	995	494	466	466
query55	77	76	76	76
query56	276	267	262	262
query57	1198	1037	1088	1037
query58	230	226	253	226
query59	3044	2867	2818	2818
query60	296	266	259	259
query61	100	98	104	98
query62	836	667	678	667
query63	229	198	181	181
query64	4175	746	666	666
query65	3243	3154	3161	3154
query66	1416	337	339	337
query67	15748	15241	15197	15197
query68	3119	579	587	579
query69	404	280	270	270
query70	1193	1136	1142	1136
query71	334	273	268	268
query72	6337	4026	3974	3974
query73	745	326	337	326
query74	9122	8870	8894	8870
query75	3403	2686	2707	2686
query76	1882	1035	1003	1003
query77	472	321	319	319
query78	9626	9082	9538	9082
query79	1043	540	594	540
query80	715	509	509	509
query81	520	247	241	241
query82	247	158	148	148
query83	174	151	154	151
query84	226	77	75	75
query85	887	290	282	282
query86	308	295	300	295
query87	4413	4327	4291	4291
query88	3186	2314	2299	2299
query89	375	299	288	288
query90	1816	197	196	196
query91	128	102	157	102
query92	59	51	52	51
query93	1029	547	532	532
query94	590	292	295	292
query95	350	264	259	259
query96	588	269	271	269
query97	3200	3108	3031	3031
query98	222	202	203	202
query99	1480	1284	1295	1284
Total cold run time: 284442 ms
Total hot run time: 188676 ms

doris-robot · 2024-08-31T09:44:01Z

ClickBench: Total hot run time: 32.48 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 21db926afb4516aa8ef28db219082598ace4e50e, data reload: false
query1	0.05	0.05	0.04
query2	0.08	0.04	0.03
query3	0.22	0.06	0.05
query4	1.66	0.08	0.10
query5	0.50	0.48	0.50
query6	1.13	0.74	0.73
query7	0.02	0.02	0.01
query8	0.06	0.05	0.04
query9	0.54	0.50	0.50
query10	0.55	0.54	0.55
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.61	0.59	0.59
query14	2.09	2.07	2.07
query15	0.85	0.81	0.82
query16	0.38	0.38	0.35
query17	0.97	0.96	0.97
query18	0.21	0.20	0.21
query19	1.86	1.74	1.77
query20	0.02	0.01	0.01
query21	15.39	0.66	0.65
query22	4.23	6.49	2.32
query23	18.23	1.39	1.31
query24	2.17	0.22	0.22
query25	0.15	0.09	0.08
query26	0.27	0.17	0.18
query27	0.08	0.08	0.07
query28	13.23	1.01	1.01
query29	12.56	3.41	3.35
query30	0.24	0.06	0.06
query31	2.89	0.40	0.40
query32	3.25	0.48	0.48
query33	3.02	2.98	3.00
query34	16.84	4.38	4.44
query35	4.42	4.44	4.44
query36	0.66	0.50	0.48
query37	0.18	0.16	0.16
query38	0.16	0.15	0.14
query39	0.05	0.04	0.05
query40	0.16	0.13	0.13
query41	0.10	0.04	0.05
query42	0.06	0.04	0.05
query43	0.04	0.05	0.05
Total cold run time: 110.49 s
Total hot run time: 32.48 s

gavinchou · 2024-08-31T13:25:52Z

run buildall

github-actions · 2024-08-31T13:32:26Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-31T14:04:25Z

TPC-H: Total hot run time: 38280 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 15ce85a75d46fbdcb423e5bd86bec302bd8bbbf8, data reload: false
------ Round 1 ----------------------------------
q1	17630	4590	4343	4343
q2	2030	186	177	177
q3	11489	997	1121	997
q4	10448	727	700	700
q5	7749	2843	2764	2764
q6	226	143	143	143
q7	964	622	603	603
q8	9315	2061	2093	2061
q9	7247	6545	6546	6545
q10	7058	2166	2241	2166
q11	451	234	239	234
q12	402	227	225	225
q13	18647	3024	3033	3024
q14	289	246	246	246
q15	519	507	516	507
q16	626	524	485	485
q17	1002	727	684	684
q18	7354	6971	6860	6860
q19	1398	1027	1023	1023
q20	703	338	325	325
q21	4252	3141	3260	3141
q22	1123	1032	1027	1027
Total cold run time: 110922 ms
Total hot run time: 38280 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4435	4378	4378	4378
q2	390	266	272	266
q3	2912	2727	2696	2696
q4	1957	1721	1655	1655
q5	5727	5762	5816	5762
q6	242	137	141	137
q7	2203	1877	1819	1819
q8	3344	3494	3543	3494
q9	9061	9083	9119	9083
q10	3713	3461	3342	3342
q11	643	513	533	513
q12	913	718	708	708
q13	15325	3259	3108	3108
q14	315	291	289	289
q15	546	479	498	479
q16	598	607	571	571
q17	1845	1636	1542	1542
q18	8301	7946	7792	7792
q19	1770	1615	1572	1572
q20	2154	1904	1901	1901
q21	5673	5550	5579	5550
q22	1170	1076	1089	1076
Total cold run time: 73237 ms
Total hot run time: 57733 ms

doris-robot · 2024-08-31T14:16:38Z

TPC-DS: Total hot run time: 192593 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 15ce85a75d46fbdcb423e5bd86bec302bd8bbbf8, data reload: false
query1	1241	890	868	868
query2	6242	2045	2014	2014
query3	10646	4156	3786	3786
query4	59295	23746	23293	23293
query5	5424	497	502	497
query6	416	168	161	161
query7	5781	298	298	298
query8	298	225	207	207
query9	9049	2507	2515	2507
query10	498	320	267	267
query11	17295	14985	15066	14985
query12	154	110	103	103
query13	1565	393	378	378
query14	11180	6731	7248	6731
query15	247	179	179	179
query16	7606	479	473	473
query17	1129	574	581	574
query18	2053	296	300	296
query19	285	149	144	144
query20	126	113	112	112
query21	206	103	104	103
query22	4706	4634	4444	4444
query23	34177	33485	33428	33428
query24	5985	2912	2812	2812
query25	523	385	385	385
query26	691	153	152	152
query27	1799	282	277	277
query28	3552	2121	2101	2101
query29	702	414	398	398
query30	231	158	149	149
query31	945	733	755	733
query32	84	58	61	58
query33	457	319	290	290
query34	865	481	503	481
query35	811	733	716	716
query36	1087	982	943	943
query37	146	88	90	88
query38	4049	3862	3913	3862
query39	1426	1393	1419	1393
query40	197	114	114	114
query41	46	47	44	44
query42	117	101	92	92
query43	531	494	479	479
query44	1101	747	749	747
query45	200	169	168	168
query46	1097	742	757	742
query47	1868	1788	1800	1788
query48	375	306	292	292
query49	758	424	444	424
query50	810	420	409	409
query51	7183	7137	7070	7070
query52	98	87	87	87
query53	249	182	177	177
query54	556	477	443	443
query55	77	78	78	78
query56	283	260	255	255
query57	1183	1084	1057	1057
query58	241	238	288	238
query59	3246	2752	2871	2752
query60	300	291	292	291
query61	127	119	117	117
query62	739	650	633	633
query63	216	185	191	185
query64	2939	756	747	747
query65	3191	3167	3151	3151
query66	701	355	357	355
query67	15348	15206	15026	15026
query68	3008	579	593	579
query69	408	291	283	283
query70	1199	1151	1143	1143
query71	373	283	288	283
query72	6431	3988	4035	3988
query73	757	334	327	327
query74	9173	8799	8820	8799
query75	3350	2703	2699	2699
query76	1414	1057	1014	1014
query77	534	320	320	320
query78	11700	9672	9116	9116
query79	1158	555	547	547
query80	875	498	513	498
query81	572	238	238	238
query82	311	150	144	144
query83	174	148	148	148
query84	268	76	79	76
query85	721	287	319	287
query86	366	304	302	302
query87	4454	4339	4407	4339
query88	3178	2309	2307	2307
query89	386	290	293	290
query90	1864	192	189	189
query91	123	96	98	96
query92	61	50	48	48
query93	1073	527	530	527
query94	754	301	290	290
query95	349	254	259	254
query96	586	275	264	264
query97	3236	3101	3100	3100
query98	222	212	207	207
query99	1569	1268	1244	1244
Total cold run time: 307396 ms
Total hot run time: 192593 ms

doris-robot · 2024-08-31T14:22:15Z

ClickBench: Total hot run time: 31.59 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 15ce85a75d46fbdcb423e5bd86bec302bd8bbbf8, data reload: false
query1	0.05	0.04	0.05
query2	0.08	0.04	0.04
query3	0.22	0.06	0.06
query4	1.66	0.08	0.07
query5	0.49	0.49	0.49
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.50
query10	0.54	0.52	0.55
query11	0.17	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.60	0.58
query14	2.03	2.05	2.04
query15	0.89	0.80	0.80
query16	0.36	0.36	0.37
query17	1.00	1.00	1.05
query18	0.21	0.20	0.22
query19	1.77	1.80	1.78
query20	0.02	0.01	0.01
query21	15.39	0.65	0.65
query22	4.19	8.04	1.48
query23	18.30	1.36	1.38
query24	2.11	0.21	0.22
query25	0.15	0.09	0.09
query26	0.26	0.18	0.17
query27	0.07	0.08	0.07
query28	13.27	1.00	1.01
query29	12.66	3.36	3.32
query30	0.24	0.06	0.05
query31	2.87	0.40	0.39
query32	3.27	0.50	0.48
query33	2.95	2.99	3.01
query34	17.12	4.37	4.41
query35	4.45	4.45	4.39
query36	0.65	0.48	0.48
query37	0.20	0.17	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.12	0.13
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.71 s
Total hot run time: 31.59 s

github-actions · 2024-09-01T04:31:31Z

PR approved by anyone and no changes requested.

liaoxin01

LGTM

github-actions · 2024-09-01T04:36:36Z

PR approved by at least one committer and no changes requested.

dataroaring

LGTM

To test it, 1. enable commit_rowset fail with injection point 2. issue a streamload 3. check the recycler log or storage vault the uncommitted rowset recycled ``` curl 'be_ip:http_port/api/injection_point/enable' curl 'be_ip:http_port/api/injection_point/clear' curl "be_ip:http_port/api/injection_point/set?name=CloudMetaMgr::commit_rowset&behavior=return_error" curl -XPUT --location-trusted -vv -T tmp/1.csv -H'format:csv' -H 'column_separator:,' -H'timeout:60' -uroot: be_ip:http_port/api/db1/t2/_stream_load curl 'be_ip:http_port/api/injection_point/clear' curl 'be_ip:http_port/api/injection_point/disable' ```

gavinchou · 2024-09-02T17:03:37Z

run buildall

github-actions · 2024-09-02T17:10:03Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-09-02T17:42:24Z

TPC-H: Total hot run time: 38185 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7c87f68db93b5469735e863aeff8fdc10444ca7c, data reload: false
------ Round 1 ----------------------------------
q1	17599	4340	4280	4280
q2	2024	180	174	174
q3	11863	979	1108	979
q4	10526	712	721	712
q5	7755	2839	2742	2742
q6	220	134	141	134
q7	967	614	604	604
q8	9340	2060	2055	2055
q9	7202	6520	6507	6507
q10	6994	2252	2277	2252
q11	450	238	241	238
q12	387	229	228	228
q13	18901	3064	3102	3064
q14	285	239	230	230
q15	528	476	488	476
q16	590	519	507	507
q17	974	646	714	646
q18	7323	6950	6942	6942
q19	1379	1022	1028	1022
q20	672	338	337	337
q21	4136	3184	3018	3018
q22	1138	1056	1038	1038
Total cold run time: 111253 ms
Total hot run time: 38185 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4335	4279	4308	4279
q2	368	275	277	275
q3	2887	2597	2665	2597
q4	1917	1676	1657	1657
q5	5663	5681	5724	5681
q6	229	139	140	139
q7	2255	1862	1834	1834
q8	3316	3414	3467	3414
q9	8815	8812	8787	8787
q10	3542	3399	3399	3399
q11	616	511	517	511
q12	830	672	673	672
q13	13314	3138	3285	3138
q14	324	289	282	282
q15	550	493	494	493
q16	639	582	584	582
q17	1848	1541	1483	1483
q18	8084	7939	7893	7893
q19	1742	1632	1566	1566
q20	2186	1884	1939	1884
q21	5745	5379	5379	5379
q22	1143	1044	1033	1033
Total cold run time: 70348 ms
Total hot run time: 56978 ms

doris-robot · 2024-09-02T17:54:37Z

TPC-DS: Total hot run time: 192956 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7c87f68db93b5469735e863aeff8fdc10444ca7c, data reload: false
query1	1249	886	855	855
query2	6404	2012	1970	1970
query3	10695	4141	4061	4061
query4	59899	25550	23244	23244
query5	5399	506	503	503
query6	423	162	160	160
query7	5896	290	291	290
query8	282	203	221	203
query9	8729	2485	2472	2472
query10	493	267	259	259
query11	17675	15072	15251	15072
query12	161	105	99	99
query13	1533	393	388	388
query14	11287	7403	7065	7065
query15	259	178	174	174
query16	7569	478	490	478
query17	1164	583	613	583
query18	1958	306	296	296
query19	285	146	149	146
query20	117	109	115	109
query21	212	105	106	105
query22	4462	4521	4352	4352
query23	34161	33553	33778	33553
query24	5914	2863	2840	2840
query25	521	380	389	380
query26	691	152	153	152
query27	1817	284	281	281
query28	3913	2051	2046	2046
query29	686	397	404	397
query30	239	148	160	148
query31	937	765	770	765
query32	83	61	52	52
query33	470	284	291	284
query34	859	480	488	480
query35	854	709	707	707
query36	1088	942	927	927
query37	151	97	92	92
query38	3939	3881	3917	3881
query39	1436	1403	1403	1403
query40	203	116	117	116
query41	49	50	46	46
query42	114	97	95	95
query43	520	482	479	479
query44	1096	748	745	745
query45	204	174	170	170
query46	1120	772	735	735
query47	1927	1809	1814	1809
query48	366	298	311	298
query49	778	432	452	432
query50	812	424	429	424
query51	6975	6951	6833	6833
query52	103	92	89	89
query53	250	182	182	182
query54	579	470	473	470
query55	78	75	76	75
query56	296	271	295	271
query57	1195	1105	1098	1098
query58	223	244	232	232
query59	3055	2890	2786	2786
query60	298	271	272	271
query61	102	97	104	97
query62	753	655	635	635
query63	211	178	181	178
query64	2868	710	669	669
query65	3271	3143	3150	3143
query66	699	353	339	339
query67	15410	15170	15260	15170
query68	2926	599	590	590
query69	400	293	278	278
query70	1117	1150	1140	1140
query71	358	277	275	275
query72	6203	4176	3980	3980
query73	762	330	337	330
query74	9230	8875	8938	8875
query75	3335	2646	2706	2646
query76	1410	999	1004	999
query77	515	316	369	316
query78	9664	9048	9108	9048
query79	1040	528	523	523
query80	672	501	512	501
query81	467	241	237	237
query82	238	152	146	146
query83	165	147	148	147
query84	252	74	78	74
query85	672	281	280	280
query86	302	267	300	267
query87	4364	4483	4358	4358
query88	3433	2315	2330	2315
query89	388	281	287	281
query90	1970	188	191	188
query91	126	97	98	97
query92	61	53	51	51
query93	1068	535	540	535
query94	707	295	291	291
query95	320	306	263	263
query96	585	261	268	261
query97	3200	3072	3059	3059
query98	221	204	210	204
query99	1523	1242	1253	1242
Total cold run time: 305159 ms
Total hot run time: 192956 ms

doris-robot · 2024-09-02T18:00:14Z

ClickBench: Total hot run time: 32.03 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7c87f68db93b5469735e863aeff8fdc10444ca7c, data reload: false
query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.51	0.47	0.50
query6	1.13	0.73	0.71
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.54	0.48	0.48
query10	0.54	0.54	0.56
query11	0.17	0.12	0.12
query12	0.15	0.13	0.13
query13	0.60	0.59	0.58
query14	2.06	2.05	2.12
query15	0.88	0.81	0.82
query16	0.34	0.37	0.36
query17	0.99	1.01	0.99
query18	0.21	0.20	0.20
query19	1.96	1.76	1.78
query20	0.01	0.02	0.01
query21	15.39	0.67	0.66
query22	4.48	6.66	2.01
query23	18.32	1.33	1.17
query24	2.16	0.22	0.21
query25	0.16	0.08	0.09
query26	0.27	0.18	0.18
query27	0.08	0.09	0.08
query28	13.25	1.02	0.99
query29	12.59	3.30	3.28
query30	0.24	0.05	0.06
query31	2.88	0.40	0.39
query32	3.26	0.50	0.48
query33	2.92	3.09	3.05
query34	17.05	4.44	4.41
query35	4.47	4.47	4.48
query36	0.65	0.47	0.49
query37	0.18	0.16	0.15
query38	0.16	0.15	0.16
query39	0.05	0.03	0.04
query40	0.16	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 111.11 s
Total hot run time: 32.03 s

dataroaring

LGTM

…40215) To test it, 1. enable commit_rowset fail with injection point 2. issue a streamload 3. check the recycler log or storage vault the uncommitted rowset recycled ``` curl 'be_ip:http_port/api/injection_point/enable' curl 'be_ip:http_port/api/injection_point/clear' curl "be_ip:http_port/api/injection_point/set?name=CloudMetaMgr::commit_rowset&behavior=return_error" curl -XPUT --location-trusted -vv -T tmp/1.csv -H'format:csv' -H 'column_separator:,' -H'timeout:60' -uroot: be_ip:http_port/api/db1/t2/_stream_load curl 'be_ip:http_port/api/injection_point/clear' curl 'be_ip:http_port/api/injection_point/disable' ```

…pache#40215) To test it, 1. enable commit_rowset fail with injection point 2. issue a streamload 3. check the recycler log or storage vault the uncommitted rowset recycled ``` curl 'be_ip:http_port/api/injection_point/enable' curl 'be_ip:http_port/api/injection_point/clear' curl "be_ip:http_port/api/injection_point/set?name=CloudMetaMgr::commit_rowset&behavior=return_error" curl -XPUT --location-trusted -vv -T tmp/1.csv -H'format:csv' -H 'column_separator:,' -H'timeout:60' -uroot: be_ip:http_port/api/db1/t2/_stream_load curl 'be_ip:http_port/api/injection_point/clear' curl 'be_ip:http_port/api/injection_point/disable' ```

gavinchou added p0_c dev/3.0.x labels Aug 31, 2024

gavinchou force-pushed the gavin-fix-hdfs-accessor-delete-prefix branch from 21db926 to 15ce85aCompare August 31, 2024 13:25

Yukang-Lian approved these changes Sep 1, 2024
View reviewed changes

bobhan1 approved these changes Sep 1, 2024
View reviewed changes

github-actionsBot added the reviewed label Sep 1, 2024

liaoxin01 previously approved these changes Sep 1, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 1, 2024

dataroaring previously approved these changes Sep 2, 2024
View reviewed changes

gavinchou dismissed stale reviews from dataroaring and liaoxin01 via 7c87f68September 2, 2024 17:03

gavinchou force-pushed the gavin-fix-hdfs-accessor-delete-prefix branch from 15ce85a to 7c87f68Compare September 2, 2024 17:03

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Sep 2, 2024

dataroaring approved these changes Sep 3, 2024
View reviewed changes

dataroaring merged commit 34754ca into apache:masterSep 3, 2024

gavinchou added dev/3.0.2-merged and removed dev/3.0.x labels Sep 3, 2024

gavinchou mentioned this pull request Oct 13, 2024
Release Note 3.0.2 #41558
Closed

Uh oh!

Conversation

gavinchou commented Aug 31, 2024

Uh oh!

doris-robot commented Aug 31, 2024

Uh oh!

github-actionsBot commented Aug 31, 2024

Uh oh!

gavinchou commented Aug 31, 2024

Uh oh!

doris-robot commented Aug 31, 2024

Uh oh!

doris-robot commented Aug 31, 2024

Uh oh!

doris-robot commented Aug 31, 2024

Uh oh!

gavinchou commented Aug 31, 2024

Uh oh!

github-actionsBot commented Aug 31, 2024

Uh oh!

doris-robot commented Aug 31, 2024

Uh oh!

doris-robot commented Aug 31, 2024

Uh oh!

doris-robot commented Aug 31, 2024

Uh oh!

github-actionsBot commented Sep 1, 2024

Uh oh!

liaoxin01 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Sep 1, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

gavinchou commented Sep 2, 2024

Uh oh!

github-actionsBot commented Sep 2, 2024

Uh oh!

doris-robot commented Sep 2, 2024

Uh oh!

doris-robot commented Sep 2, 2024

Uh oh!

doris-robot commented Sep 2, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants