[feat](catalog)Support for Loading Catalog Credentials via AwsCredentialsProviderChain by CalvinKirs · Pull Request #58740 · apache/doris

CalvinKirs · 2025-12-05T02:57:28Z

Changes

This update enables Catalogs to automatically load credentials for S3 or S3-compatible storage using AwsCredentialsProviderChain. Users no longer need to explicitly specify AK/SK in Catalog properties. The feature provides stronger support for cloud-native environments such as IRSA, containers, and EC2 Instance Profiles. ↳

All credential resolution is unified through DefaultDorisAwsCredentialsProviderChain, ensuring consistent and secure handling across the project.

Key Behavior

No need to configure AK/SK manually:

Credentials can be automatically derived from IRSA, WebIdentity, container metadata, EC2 instance profile, environment variables, system properties, or AWS profile files. ↳

Unified credential resolution:

All modules use DefaultDorisAwsCredentialsProviderChain to avoid inconsistent custom implementations. ↳

eg


CREATE CATALOG `iceberg_fs_catalog` PROPERTIES (
"type" = "iceberg",
"iceberg.catalog.type" = "hadoop",
"warehouse" = "s3://<bucket>/iceberg/fs/s3/warehouse/",
"s3.region"="us-east-1",
"s3.endpoint" = "s3.us-east-1.amazonaws.com"
);

CalvinKirs · 2025-12-05T02:57:58Z

run buildall

hello-stephen · 2025-12-05T02:58:00Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

CalvinKirs · 2025-12-05T09:38:02Z

run buildall

[feat](catalog)Support for Loading Catalog Credentials via AwsCredentialsProviderChain This update enables Catalogs to automatically load credentials for S3 or S3-compatible storage using AwsCredentialsProviderChain. Users no longer need to explicitly specify AK/SK in Catalog properties. The feature provides stronger support for cloud-native environments such as IRSA, containers, and EC2 Instance Profiles. All credential resolution is unified through DefaultDorisAwsCredentialsProviderChain, ensuring consistent and secure handling across the project. Key Behavior No need to configure AK/SK manually: Credentials can be automatically derived from IRSA, WebIdentity, container metadata, EC2 instance profile, environment variables, system properties, or AWS profile files. Simplified Catalog configuration: Users can set s3.auth_type = INSTANCE_PROFILE or rely on environment detection without providing any keys. Unified credential resolution: All modules use DefaultDorisAwsCredentialsProviderChain to avoid inconsistent custom implementations. Backwards compatible: If s3.access_key / s3.secret_key are provided explicitly, the previous behavior is preserved.

CalvinKirs · 2025-12-05T09:44:41Z

run buildall

doris-robot · 2025-12-05T10:22:24Z

TPC-H: Total hot run time: 34464 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5b4ca59d6f0dc0413f8918091e773a3e23a12d85, data reload: false
------ Round 1 ----------------------------------
q1	17637	5036	4882	4882
q2	2050	303	190	190
q3	10263	1301	736	736
q4	10227	876	311	311
q5	7513	2305	2233	2233
q6	187	174	141	141
q7	941	758	635	635
q8	9331	1401	1108	1108
q9	6926	5321	5344	5321
q10	6843	2193	1827	1827
q11	526	305	300	300
q12	342	359	226	226
q13	17803	3641	2966	2966
q14	239	239	221	221
q15	591	528	532	528
q16	946	900	810	810
q17	664	800	514	514
q18	7647	7208	7110	7110
q19	1482	986	610	610
q20	382	354	236	236
q21	4085	3408	2592	2592
q22	1056	1015	967	967
Total cold run time: 107681 ms
Total hot run time: 34464 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4971	4965	4917	4917
q2	346	402	342	342
q3	2212	2744	2293	2293
q4	1306	1712	1272	1272
q5	4212	4431	4542	4431
q6	232	182	141	141
q7	2055	1904	1885	1885
q8	2665	2657	2510	2510
q9	7499	7456	7528	7456
q10	3020	3315	2824	2824
q11	593	509	500	500
q12	694	735	563	563
q13	3644	3906	3329	3329
q14	288	301	295	295
q15	545	532	508	508
q16	877	936	874	874
q17	1160	1389	1413	1389
q18	8073	7793	7391	7391
q19	922	861	887	861
q20	2007	1963	1822	1822
q21	4647	4307	4095	4095
q22	1086	1067	1004	1004
Total cold run time: 53054 ms
Total hot run time: 50702 ms

doris-robot · 2025-12-05T10:33:42Z

TPC-DS: Total hot run time: 179491 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5b4ca59d6f0dc0413f8918091e773a3e23a12d85, data reload: false
query5	4544	652	509	509
query6	374	218	201	201
query7	4656	474	291	291
query8	334	252	234	234
query9	8724	2622	2649	2622
query10	536	316	269	269
query11	15416	14937	14590	14590
query12	184	119	112	112
query13	1661	471	377	377
query14	5620	3258	3038	3038
query14_1	2875	2934	2898	2898
query15	211	201	186	186
query16	7256	471	470	470
query17	1278	723	581	581
query18	1996	426	334	334
query19	202	184	158	158
query20	129	122	114	114
query21	216	134	112	112
query22	3857	3999	3780	3780
query23	16551	16068	15782	15782
query23_1	15971	15976	15816	15816
query24	7292	1629	1219	1219
query24_1	1230	1197	1159	1159
query25	604	476	418	418
query26	1260	278	170	170
query27	2938	459	307	307
query28	4438	2170	2159	2159
query29	826	568	451	451
query30	310	241	211	211
query31	822	676	621	621
query32	83	70	77	70
query33	668	351	290	290
query34	866	885	554	554
query35	785	803	722	722
query36	900	922	810	810
query37	119	89	81	81
query38	3876	3801	3824	3801
query39	767	737	719	719
query39_1	765	683	695	683
query40	234	138	127	127
query41	74	68	67	67
query42	129	102	96	96
query43	438	429	402	402
query44	1316	764	773	764
query45	205	197	194	194
query46	915	986	596	596
query47	1689	1726	1643	1643
query48	402	327	249	249
query49	786	455	373	373
query50	687	309	242	242
query51	3883	3924	3990	3924
query52	118	104	93	93
query53	238	242	188	188
query54	339	285	268	268
query55	96	87	74	74
query56	347	321	333	321
query57	1183	1162	1088	1088
query58	305	267	263	263
query59	2348	2401	2222	2222
query60	372	350	318	318
query61	197	192	187	187
query62	777	691	640	640
query63	238	183	178	178
query64	4701	1283	1036	1036
query65	4002	3943	3968	3943
query66	1282	463	368	368
query67	15187	14986	14733	14733
query68	8342	936	666	666
query69	515	297	265	265
query70	1114	991	959	959
query71	450	294	277	277
query72	5830	4970	4929	4929
query73	708	594	314	314
query74	8833	8812	8650	8650
query75	3288	3030	2532	2532
query76	3415	1157	759	759
query77	741	402	288	288
query78	9416	9582	8886	8886
query79	1689	808	592	592
query80	658	550	464	464
query81	499	273	240	240
query82	476	121	115	115
query83	270	268	250	250
query84	261	122	100	100
query85	1005	488	449	449
query86	392	286	279	279
query87	4021	4044	4053	4044
query88	4072	2105	2127	2105
query89	399	316	274	274
query90	2042	163	157	157
query91	168	164	143	143
query92	87	69	67	67
query93	1649	1026	691	691
query94	821	308	297	297
query95	564	329	332	329
query96	542	505	210	210
query97	2629	2666	2585	2585
query98	244	200	191	191
query99	1398	1342	1227	1227
Total cold run time: 269920 ms
Total hot run time: 179491 ms

doris-robot · 2025-12-05T10:38:44Z

ClickBench: Total hot run time: 27.12 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5b4ca59d6f0dc0413f8918091e773a3e23a12d85, data reload: false
query1	0.05	0.05	0.05
query2	0.10	0.04	0.05
query3	0.26	0.09	0.09
query4	1.60	0.11	0.11
query5	0.27	0.26	0.27
query6	1.17	0.64	0.63
query7	0.03	0.02	0.02
query8	0.05	0.04	0.05
query9	0.58	0.51	0.50
query10	0.57	0.56	0.55
query11	0.16	0.10	0.10
query12	0.16	0.12	0.12
query13	0.64	0.60	0.60
query14	0.98	0.98	0.97
query15	0.81	0.80	0.80
query16	0.40	0.42	0.40
query17	1.02	0.99	1.05
query18	0.22	0.21	0.21
query19	1.94	1.88	1.76
query20	0.02	0.01	0.01
query21	15.44	0.28	0.14
query22	4.91	0.05	0.05
query23	16.19	0.28	0.10
query24	1.44	0.42	0.19
query25	0.08	0.05	0.07
query26	0.15	0.13	0.14
query27	0.05	0.06	0.06
query28	3.25	1.27	1.02
query29	12.59	4.14	3.19
query30	0.28	0.13	0.12
query31	2.81	0.64	0.39
query32	3.23	0.56	0.47
query33	3.02	2.99	3.07
query34	16.57	5.17	4.61
query35	4.55	4.54	4.62
query36	0.67	0.50	0.48
query37	0.10	0.07	0.07
query38	0.08	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.82 s
Total hot run time: 27.12 s

hello-stephen · 2025-12-05T13:28:27Z

FE Regression Coverage Report

Increment line coverage 56.52% (39/69) 🎉
Increment coverage report
Complete coverage report

CalvinKirs · 2025-12-13T06:51:29Z

run buildall

CalvinKirs · 2025-12-13T07:16:06Z

run buildall

doris-robot · 2025-12-13T08:35:43Z

TPC-H: Total hot run time: 36973 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2f5468a03581291b4ca9e67b78dcda30f3eacbe8, data reload: false
------ Round 1 ----------------------------------
q1	17616	4234	4136	4136
q2	2070	373	246	246
q3	10117	1336	736	736
q4	10234	936	331	331
q5	7731	2191	2003	2003
q6	259	168	141	141
q7	1005	877	742	742
q8	9372	1635	1255	1255
q9	7250	5369	5410	5369
q10	6856	2409	1984	1984
q11	533	344	295	295
q12	691	733	583	583
q13	17787	3687	3040	3040
q14	287	303	270	270
q15	596	527	519	519
q16	723	704	616	616
q17	710	809	596	596
q18	7603	7958	8089	7958
q19	1302	1031	650	650
q20	439	371	262	262
q21	4790	4221	4226	4221
q22	1163	1033	1020	1020
Total cold run time: 109134 ms
Total hot run time: 36973 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4463	4336	4326	4326
q2	322	407	335	335
q3	2358	2862	2542	2542
q4	1582	1881	1414	1414
q5	4388	4464	4707	4464
q6	219	164	125	125
q7	2094	1925	1825	1825
q8	2695	2549	2552	2549
q9	7491	7515	7451	7451
q10	3109	3264	2884	2884
q11	620	526	505	505
q12	652	717	574	574
q13	3336	3676	3061	3061
q14	283	286	258	258
q15	542	490	487	487
q16	639	652	612	612
q17	1139	1376	1353	1353
q18	7376	7036	7069	7036
q19	840	835	850	835
q20	1926	1988	1838	1838
q21	4767	4290	4224	4224
q22	1087	1041	973	973
Total cold run time: 51928 ms
Total hot run time: 49671 ms

doris-robot · 2025-12-13T08:46:43Z

TPC-DS: Total hot run time: 178534 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2f5468a03581291b4ca9e67b78dcda30f3eacbe8, data reload: false
query5	4632	631	490	490
query6	351	233	220	220
query7	4208	473	276	276
query8	299	254	238	238
query9	8775	2583	2568	2568
query10	515	364	324	324
query11	15198	14735	14593	14593
query12	178	115	115	115
query13	1262	513	377	377
query14	5710	3276	2973	2973
query14_1	2902	2933	2937	2933
query15	206	202	186	186
query16	922	437	451	437
query17	1116	689	585	585
query18	2417	430	332	332
query19	230	228	206	206
query20	119	115	118	115
query21	219	141	110	110
query22	3850	4058	3895	3895
query23	16436	16147	16028	16028
query23_1	15968	16083	15987	15987
query24	7371	1650	1262	1262
query24_1	1240	1239	1255	1239
query25	549	474	427	427
query26	1248	270	159	159
query27	2763	462	303	303
query28	4484	2139	2131	2131
query29	820	556	451	451
query30	321	242	216	216
query31	827	698	614	614
query32	80	74	71	71
query33	537	328	298	298
query34	899	890	540	540
query35	787	850	725	725
query36	876	904	831	831
query37	133	93	84	84
query38	2845	2884	2805	2805
query39	774	785	725	725
query39_1	740	700	700	700
query40	226	137	125	125
query41	71	62	61	61
query42	107	113	114	113
query43	446	441	415	415
query44	1331	767	743	743
query45	199	198	193	193
query46	879	997	616	616
query47	1644	1708	1595	1595
query48	308	325	273	273
query49	632	436	352	352
query50	684	293	220	220
query51	3790	3829	3795	3795
query52	104	110	99	99
query53	326	350	296	296
query54	303	263	255	255
query55	81	78	73	73
query56	311	299	315	299
query57	1148	1140	1074	1074
query58	266	288	263	263
query59	2433	2430	2355	2355
query60	321	321	294	294
query61	175	160	158	158
query62	719	657	628	628
query63	330	298	307	298
query64	4964	1324	1078	1078
query65	4011	3976	3981	3976
query66	1497	469	337	337
query67	14892	14961	14698	14698
query68	2807	1051	769	769
query69	438	360	326	326
query70	1065	1028	991	991
query71	331	323	301	301
query72	6449	5240	5413	5240
query73	512	617	314	314
query74	8723	8991	8611	8611
query75	3098	3109	2765	2765
query76	2888	1142	741	741
query77	365	411	288	288
query78	9540	9789	8868	8868
query79	1512	885	614	614
query80	1244	662	546	546
query81	543	274	230	230
query82	493	135	104	104
query83	360	248	230	230
query84	260	124	105	105
query85	955	518	461	461
query86	384	288	307	288
query87	2984	3053	2979	2979
query88	3270	2270	2275	2270
query89	470	422	384	384
query90	2047	153	153	153
query91	186	167	148	148
query92	67	67	72	67
query93	1137	893	548	548
query94	539	308	275	275
query95	578	335	304	304
query96	589	471	206	206
query97	2276	2320	2238	2238
query98	210	194	191	191
query99	1329	1287	1205	1205
Total cold run time: 252634 ms
Total hot run time: 178534 ms

doris-robot · 2025-12-13T08:51:48Z

ClickBench: Total hot run time: 27.49 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2f5468a03581291b4ca9e67b78dcda30f3eacbe8, data reload: false
query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.10
query4	1.61	0.11	0.11
query5	0.28	0.28	0.26
query6	1.17	0.65	0.63
query7	0.03	0.03	0.02
query8	0.06	0.05	0.05
query9	0.57	0.50	0.50
query10	0.56	0.55	0.55
query11	0.15	0.12	0.10
query12	0.15	0.12	0.11
query13	0.62	0.61	0.61
query14	0.99	0.99	0.98
query15	0.81	0.80	0.82
query16	0.40	0.40	0.42
query17	1.05	1.05	1.07
query18	0.24	0.22	0.22
query19	1.94	1.80	1.77
query20	0.02	0.01	0.01
query21	15.43	0.28	0.15
query22	4.78	0.05	0.05
query23	16.05	0.29	0.10
query24	1.38	0.46	0.67
query25	0.12	0.13	0.04
query26	0.15	0.13	0.13
query27	0.08	0.05	0.05
query28	5.33	1.22	1.02
query29	12.59	4.11	3.23
query30	0.29	0.13	0.12
query31	2.81	0.62	0.39
query32	3.24	0.55	0.46
query33	3.04	3.04	3.09
query34	16.87	5.25	4.54
query35	4.55	4.52	4.56
query36	0.66	0.50	0.49
query37	0.10	0.06	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 98.97 s
Total hot run time: 27.49 s

CalvinKirs · 2025-12-13T14:13:24Z

run buildall

doris-robot · 2025-12-13T14:51:44Z

TPC-H: Total hot run time: 36412 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2f5468a03581291b4ca9e67b78dcda30f3eacbe8, data reload: false
------ Round 1 ----------------------------------
q1	17615	4299	4158	4158
q2	2030	347	249	249
q3	10179	1300	735	735
q4	10229	932	321	321
q5	7551	2099	1949	1949
q6	189	168	138	138
q7	1016	847	726	726
q8	9361	1455	1139	1139
q9	6970	5305	5368	5305
q10	6836	2395	1993	1993
q11	517	321	298	298
q12	630	755	554	554
q13	17804	3737	3048	3048
q14	294	306	279	279
q15	601	514	516	514
q16	708	699	628	628
q17	683	798	560	560
q18	7798	7682	8011	7682
q19	1939	1031	643	643
q20	443	377	258	258
q21	4697	4246	4161	4161
q22	1125	1090	1074	1074
Total cold run time: 109215 ms
Total hot run time: 36412 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4339	4330	4300	4300
q2	376	421	313	313
q3	2429	3090	2427	2427
q4	1423	1912	1410	1410
q5	4737	4486	4492	4486
q6	215	164	132	132
q7	2007	1969	1943	1943
q8	2656	2565	2577	2565
q9	7360	7390	7527	7390
q10	2929	3104	2673	2673
q11	575	492	478	478
q12	645	691	538	538
q13	3307	3697	3040	3040
q14	273	276	257	257
q15	537	493	490	490
q16	626	645	608	608
q17	1102	1349	1317	1317
q18	7276	7151	7019	7019
q19	885	804	827	804
q20	1914	1968	1802	1802
q21	4653	4379	4132	4132
q22	1082	1009	993	993
Total cold run time: 51346 ms
Total hot run time: 49117 ms

doris-robot · 2025-12-13T15:02:49Z

TPC-DS: Total hot run time: 178607 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2f5468a03581291b4ca9e67b78dcda30f3eacbe8, data reload: false
query5	5082	626	494	494
query6	334	231	216	216
query7	4219	472	279	279
query8	292	244	237	237
query9	8738	2556	2565	2556
query10	549	358	324	324
query11	15411	14761	14792	14761
query12	173	119	113	113
query13	1279	488	397	397
query14	6195	3295	3035	3035
query14_1	2901	2955	2936	2936
query15	206	208	190	190
query16	849	493	478	478
query17	1155	697	595	595
query18	2626	428	331	331
query19	233	224	201	201
query20	123	117	113	113
query21	230	134	113	113
query22	4054	3915	3867	3867
query23	16624	16112	15859	15859
query23_1	16045	16018	16059	16018
query24	7367	1641	1240	1240
query24_1	1251	1243	1263	1243
query25	560	466	407	407
query26	1256	271	157	157
query27	2764	464	303	303
query28	4465	2164	2120	2120
query29	820	542	440	440
query30	307	241	218	218
query31	818	688	623	623
query32	81	75	65	65
query33	528	337	294	294
query34	914	901	556	556
query35	758	818	722	722
query36	899	920	827	827
query37	127	92	77	77
query38	2847	2890	2815	2815
query39	778	744	730	730
query39_1	707	695	704	695
query40	221	137	126	126
query41	67	64	62	62
query42	111	105	106	105
query43	444	433	397	397
query44	1338	753	752	752
query45	197	194	182	182
query46	873	986	626	626
query47	1674	1707	1598	1598
query48	317	318	245	245
query49	635	429	353	353
query50	669	294	216	216
query51	3817	3988	3851	3851
query52	105	109	96	96
query53	331	351	297	297
query54	286	257	257	257
query55	84	79	72	72
query56	324	308	304	304
query57	1139	1133	1089	1089
query58	267	253	254	253
query59	2357	2533	2329	2329
query60	304	317	305	305
query61	197	192	188	188
query62	707	674	623	623
query63	329	300	307	300
query64	5164	1435	1106	1106
query65	3995	3944	3947	3944
query66	1441	458	353	353
query67	15393	15022	14680	14680
query68	5223	1026	735	735
query69	508	357	328	328
query70	1067	1007	1004	1004
query71	373	320	286	286
query72	6301	5032	5084	5032
query73	703	620	312	312
query74	8890	8784	8590	8590
query75	3184	3137	2783	2783
query76	3876	1145	747	747
query77	521	407	287	287
query78	9428	9635	8824	8824
query79	1656	875	627	627
query80	719	652	549	549
query81	513	271	234	234
query82	210	141	107	107
query83	281	264	244	244
query84	261	118	102	102
query85	886	521	474	474
query86	379	300	285	285
query87	3024	3145	3026	3026
query88	3189	2293	2276	2276
query89	470	422	395	395
query90	2228	160	154	154
query91	174	167	143	143
query92	84	65	65	65
query93	1505	921	547	547
query94	464	290	288	288
query95	579	388	300	300
query96	594	471	212	212
query97	2313	2338	2231	2231
query98	218	205	188	188
query99	1303	1302	1200	1200
Total cold run time: 258275 ms
Total hot run time: 178607 ms

doris-robot · 2025-12-13T15:07:54Z

ClickBench: Total hot run time: 27.57 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2f5468a03581291b4ca9e67b78dcda30f3eacbe8, data reload: false
query1	0.05	0.05	0.04
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.11	0.11
query5	0.27	0.26	0.28
query6	1.15	0.63	0.62
query7	0.04	0.03	0.02
query8	0.05	0.04	0.05
query9	0.56	0.51	0.51
query10	0.55	0.54	0.55
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.63	0.60	0.60
query14	0.99	0.99	0.98
query15	0.81	0.80	0.81
query16	0.39	0.42	0.39
query17	1.00	1.01	1.06
query18	0.22	0.21	0.21
query19	1.97	1.90	1.92
query20	0.02	0.01	0.02
query21	15.44	0.32	0.14
query22	4.87	0.05	0.05
query23	15.88	0.29	0.11
query24	1.90	0.64	0.38
query25	0.08	0.07	0.08
query26	0.14	0.13	0.13
query27	0.06	0.06	0.06
query28	4.75	1.20	1.03
query29	12.59	4.08	3.23
query30	0.28	0.14	0.12
query31	2.83	0.63	0.40
query32	3.24	0.55	0.46
query33	3.05	3.08	3.12
query34	16.79	5.21	4.53
query35	4.61	4.58	4.53
query36	0.66	0.50	0.51
query37	0.11	0.07	0.08
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.13
query41	0.09	0.03	0.02
query42	0.04	0.04	0.02
query43	0.04	0.04	0.04
Total cold run time: 98.71 s
Total hot run time: 27.57 s

hello-stephen · 2025-12-13T15:41:02Z

FE UT Coverage Report

Increment line coverage 51.40% (55/107) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-12-13T18:13:39Z

FE Regression Coverage Report

Increment line coverage 30.39% (31/102) 🎉
Increment coverage report
Complete coverage report

CalvinKirs · 2025-12-14T01:32:14Z

run p0

CalvinKirs · 2025-12-14T02:24:21Z

run external

hello-stephen · 2025-12-14T03:47:44Z

FE Regression Coverage Report

Increment line coverage 30.39% (31/102) 🎉
Increment coverage report
Complete coverage report

github-actions · 2025-12-15T07:13:35Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-12-15T07:13:38Z

PR approved by anyone and no changes requested.

hello-stephen · 2025-12-15T08:14:51Z

FE Regression Coverage Report

Increment line coverage 30.39% (31/102) 🎉
Increment coverage report
Complete coverage report

…ialsProviderChain (#58740) ### Changes This update enables Catalogs to automatically load credentials for S3 or S3-compatible storage using AwsCredentialsProviderChain. Users no longer need to explicitly specify AK/SK in Catalog properties. The feature provides stronger support for cloud-native environments such as IRSA, containers, and EC2 Instance Profiles. ↳ All credential resolution is unified through DefaultDorisAwsCredentialsProviderChain, ensuring consistent and secure handling across the project. Key Behavior #### No need to configure AK/SK manually: Credentials can be automatically derived from IRSA, WebIdentity, container metadata, EC2 instance profile, environment variables, system properties, or AWS profile files. ↳ #### Unified credential resolution: All modules use DefaultDorisAwsCredentialsProviderChain to avoid inconsistent custom implementations. ↳ eg ``` CREATE CATALOG `iceberg_fs_catalog` PROPERTIES ( "type" = "iceberg", "iceberg.catalog.type" = "hadoop", "warehouse" = "s3://<bucket>/iceberg/fs/s3/warehouse/", "s3.region"="us-east-1", "s3.endpoint" = "s3.us-east-1.amazonaws.com" ); ```

vietwow · 2025-12-15T19:30:05Z

Hi @CalvinKirs , I would know if this PR also supports EKS Pod Identity as well ? Thanks

…ialsProviderChain (#58740) ### Changes This update enables Catalogs to automatically load credentials for S3 or S3-compatible storage using AwsCredentialsProviderChain. Users no longer need to explicitly specify AK/SK in Catalog properties. The feature provides stronger support for cloud-native environments such as IRSA, containers, and EC2 Instance Profiles. ↳ All credential resolution is unified through DefaultDorisAwsCredentialsProviderChain, ensuring consistent and secure handling across the project. Key Behavior #### No need to configure AK/SK manually: Credentials can be automatically derived from IRSA, WebIdentity, container metadata, EC2 instance profile, environment variables, system properties, or AWS profile files. ↳ #### Unified credential resolution: All modules use DefaultDorisAwsCredentialsProviderChain to avoid inconsistent custom implementations. ↳ eg ``` CREATE CATALOG `iceberg_fs_catalog` PROPERTIES ( "type" = "iceberg", "iceberg.catalog.type" = "hadoop", "warehouse" = "s3://<bucket>/iceberg/fs/s3/warehouse/", "s3.region"="us-east-1", "s3.endpoint" = "s3.us-east-1.amazonaws.com" ); ```

…a AwsCredentialsProviderChain #58740 (#59054) Cherry-picked from #58740 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

morningman · 2025-12-18T06:20:29Z

Hi @CalvinKirs , I would know if this PR also supports EKS Pod Identity as well ? Thanks

Not support, but we will figure it out.

…3 access (#59082) #58740 This is the backend change for PR #58740, with additional test cases added on top of it. --------- Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

…a AwsCredentialsProviderChain #58740 (#59055) Cherry-picked from #58740 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

…3 access (#59082) #58740 This is the backend change for PR #58740, with additional test cases added on top of it. --------- Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

CalvinKirs force-pushed the master-aws-instance-profile branch from 0f4f0c1 to 5b4ca59Compare December 5, 2025 09:44

remove gcs
27faaa3

CalvinKirs requested a review from morningman as a code owner December 10, 2025 02:53

fix
dd340bf

fix
2f5468a

morningman approved these changes Dec 15, 2025
View reviewed changes

morningman added the dev/3.1.x label Dec 15, 2025

morningman added the dev/4.0.x label Dec 15, 2025

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 15, 2025

github-actionsBot added the reviewed label Dec 15, 2025

hubgeter approved these changes Dec 15, 2025
View reviewed changes

morningman merged commit 58e3f6d into apache:masterDec 15, 2025
29 of 30 checks passed

github-actionsBot mentioned this pull request Dec 15, 2025
branch-3.1: [feat](catalog)Support for Loading Catalog Credentials via AwsCredentialsProviderChain #58740 #59054
Merged

github-actionsBot mentioned this pull request Dec 15, 2025
branch-4.0: [feat](catalog)Support for Loading Catalog Credentials via AwsCredentialsProviderChain #58740 #59055
Merged

morrySnow pushed a commit that referenced this pull request Dec 18, 2025
branch-3.1: [feat](catalog)Support for Loading Catalog Credentials vi…
5cc69b3
…a AwsCredentialsProviderChain #58740 (#59054) Cherry-picked from #58740 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

morrySnow added dev/3.1.4-merged and removed dev/3.1.x labels Dec 18, 2025

CalvinKirs mentioned this pull request Dec 18, 2025
[feat](catalog) Support passing credentials_provider_type to BE for S3 access #59082
Merged

zy-kkk added dev/4.0.x and removed dev/4.0.x labels Dec 22, 2025

yiguolei pushed a commit that referenced this pull request Dec 22, 2025
branch-4.0: [feat](catalog)Support for Loading Catalog Credentials vi…
ba1b0e3
…a AwsCredentialsProviderChain #58740 (#59055) Cherry-picked from #58740 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

yiguolei added dev/4.0.3-merged and removed dev/4.0.x labels Dec 22, 2025

yiguolei mentioned this pull request Jan 23, 2026
4.0.3 Release Notes #60193
Closed

Uh oh!

Conversation

CalvinKirs commented Dec 5, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

No need to configure AK/SK manually:

Unified credential resolution:

Uh oh!

CalvinKirs commented Dec 5, 2025

Uh oh!

hello-stephen commented Dec 5, 2025

Uh oh!

CalvinKirs commented Dec 5, 2025

Uh oh!

CalvinKirs commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

hello-stephen commented Dec 5, 2025

FE Regression Coverage Report

Uh oh!

CalvinKirs commented Dec 13, 2025

Uh oh!

CalvinKirs commented Dec 13, 2025

Uh oh!

doris-robot commented Dec 13, 2025

Uh oh!

doris-robot commented Dec 13, 2025

Uh oh!

doris-robot commented Dec 13, 2025

Uh oh!

CalvinKirs commented Dec 13, 2025

Uh oh!

doris-robot commented Dec 13, 2025

Uh oh!

doris-robot commented Dec 13, 2025

Uh oh!

doris-robot commented Dec 13, 2025

Uh oh!

hello-stephen commented Dec 13, 2025

FE UT Coverage Report

Uh oh!

hello-stephen commented Dec 13, 2025

FE Regression Coverage Report

Uh oh!

CalvinKirs commented Dec 14, 2025

Uh oh!

CalvinKirs commented Dec 14, 2025

Uh oh!

hello-stephen commented Dec 14, 2025

FE Regression Coverage Report

Uh oh!

github-actionsBot commented Dec 15, 2025

Uh oh!

github-actionsBot commented Dec 15, 2025

Uh oh!

hello-stephen commented Dec 15, 2025

FE Regression Coverage Report

Uh oh!

Uh oh!

vietwow commented Dec 15, 2025

Uh oh!

morningman commented Dec 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

CalvinKirs commented Dec 5, 2025•
edited
Loading