Skip to content

[feat](catalog)Support OneLake - #57933

Merged
CalvinKirs merged 4 commits into
apache:masterfrom
CalvinKirs:master-ms-onelake
Nov 14, 2025
Merged

[feat](catalog)Support OneLake#57933
CalvinKirs merged 4 commits into
apache:masterfrom
CalvinKirs:master-ms-onelake

Conversation

@CalvinKirs

@CalvinKirsCalvinKirs commented Nov 12, 2025•
edited
Loading

Copy link
Copy Markdown
Member

Microsoft OneLake, part of the Fabric ecosystem, is a unified and open SaaS data lake that provides organizations with a centralized logical data storage layer. Data in OneLake is stored in Parquet format and can simultaneously maintain Delta Lake and Apache Iceberg metadata. This design allows multiple analytics engines to directly access shared datasets without data duplication or migration, significantly simplifying data management and governance.
By leveraging Apache Doris’s Iceberg REST Catalog, users can directly query and analyze data stored in OneLake—again, without copying or moving it.

This integration enables building end-to-end analytics pipelines on a single data lake, combining OneLake’s unified storage and governance capabilities with Doris’s high-performance analytical compute.
At the technical level, Doris interacts with OneLake through open table formats and standardized interfaces to access both metadata and Parquet files. The architecture preserves centralized governance, access control, and security policies—ensuring platform reliability, scalability, and maintainability.


create catalog onelake_doris PROPERTIES (
'type' = 'iceberg',
'iceberg.catalog.type' = 'rest',
'uri'='https://onelake.table.fabric.microsoft.com/iceberg',
'warehouse'='<workerspace_id>/<data_item_id>',
'iceberg.rest.security.type'='oauth2',
'iceberg.rest.oauth2.server-uri'='https://login.microsoftonline.com/<talent_id>/oauth2/v2.0/token',
'iceberg.rest.oauth2.credential'='<oauth2.client_id>:'<oauth2.client_secret>,
'iceberg.rest.oauth2.scope'='https://storage.azure.com/.default', 'fs.azure.support'='true',
'azure.endpoint'='https://onelake.dfs.fabric.microsoft.com',
'azure.auth_type'='OAuth2',
'azure.oauth2_account_host'='onelake.dfs.fabric.microsoft.com',
'azure.oauth2_server_uri'='https://login.microsoftonline.com/<talent_id>/oauth2/v2.0/token',
'azure.oauth2_client_id'='<oauth2.client_id>',
'azure.oauth2_client_secret'='<oauth2.client_secret>'
);

@hello-stephen

hello-stephen commented Nov 12, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs

CalvinKirs commented Nov 12, 2025

Copy link
Copy Markdown
MemberAuthor

run buildall

@doris-robot

doris-robot commented Nov 12, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34220 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 16987f3139c1daa541a220b7bc4b494938993572, data reload: false
------ Round 1 ----------------------------------
q1	17634	5232	5111	5111
q2	1994	305	198	198
q3	10308	1289	714	714
q4	10240	918	377	377
q5	7518	2352	2390	2352
q6	194	175	142	142
q7	924	777	633	633
q8	9367	1356	1047	1047
q9	6934	5247	5188	5188
q10	6900	2219	1803	1803
q11	515	308	290	290
q12	385	366	237	237
q13	17801	3644	3005	3005
q14	224	245	216	216
q15	573	514	518	514
q16	996	1001	952	952
q17	601	864	386	386
q18	7462	7051	6934	6934
q19	1442	959	590	590
q20	373	337	227	227
q21	3847	3242	2334	2334
q22	1044	1031	970	970
Total cold run time: 107276 ms
Total hot run time: 34220 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5225	5155	5141	5141
q2	263	333	238	238
q3	2140	2658	2273	2273
q4	1336	1755	1346	1346
q5	4220	4485	4494	4485
q6	211	179	137	137
q7	2094	1944	1939	1939
q8	2729	2586	2531	2531
q9	7267	7417	7371	7371
q10	3024	3351	2794	2794
q11	629	534	518	518
q12	696	819	650	650
q13	3795	3974	3353	3353
q14	283	315	297	297
q15	562	510	506	506
q16	1068	1109	1057	1057
q17	1239	1605	1379	1379
q18	7925	7706	7613	7613
q19	868	931	1024	931
q20	2006	2039	1880	1880
q21	4745	4317	4420	4317
q22	1102	1058	1008	1008
Total cold run time: 53427 ms
Total hot run time: 51764 ms

@doris-robot

doris-robot commented Nov 12, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 188940 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 16987f3139c1daa541a220b7bc4b494938993572, data reload: false
query1	1048	401	388	388
query2	6565	1704	1719	1704
query3	6774	240	225	225
query4	26596	23539	23614	23539
query5	4414	617	461	461
query6	365	256	225	225
query7	4649	492	304	304
query8	316	288	262	262
query9	8724	2641	2633	2633
query10	487	348	289	289
query11	16041	15135	15225	15135
query12	185	120	113	113
query13	1685	563	439	439
query14	11251	9240	9198	9198
query15	202	184	168	168
query16	7642	657	521	521
query17	1243	751	615	615
query18	2019	424	315	315
query19	204	196	170	170
query20	136	130	121	121
query21	216	133	122	122
query22	4121	4050	3988	3988
query23	33838	33318	33198	33198
query24	8389	2394	2401	2394
query25	600	509	457	457
query26	1230	272	156	156
query27	2762	493	358	358
query28	4353	2276	2228	2228
query29	788	618	492	492
query30	307	225	196	196
query31	935	823	735	735
query32	85	75	70	70
query33	595	364	336	336
query34	788	839	523	523
query35	813	833	755	755
query36	924	1011	910	910
query37	116	106	84	84
query38	3518	3516	3475	3475
query39	1506	1432	1400	1400
query40	227	125	118	118
query41	62	60	64	60
query42	133	114	112	112
query43	498	513	465	465
query44	1225	739	724	724
query45	190	176	180	176
query46	870	1013	646	646
query47	1758	1754	1708	1708
query48	387	432	317	317
query49	791	513	418	418
query50	628	675	402	402
query51	3954	3921	3880	3880
query52	111	105	100	100
query53	231	280	198	198
query54	313	294	291	291
query55	92	114	83	83
query56	348	309	305	305
query57	1157	1174	1108	1108
query58	291	269	269	269
query59	2595	2764	2581	2581
query60	362	360	334	334
query61	161	161	178	161
query62	809	739	684	684
query63	237	195	197	195
query64	4506	1174	903	903
query65	4070	3963	3970	3963
query66	1087	436	334	334
query67	15075	15034	14886	14886
query68	5707	894	594	594
query69	489	337	298	298
query70	1368	1241	1230	1230
query71	427	341	316	316
query72	6199	5163	5165	5163
query73	606	597	361	361
query74	9297	8759	9037	8759
query75	3296	3317	2944	2944
query76	3301	1184	754	754
query77	512	406	310	310
query78	9654	9949	8968	8968
query79	2074	812	597	597
query80	1683	567	493	493
query81	547	262	229	229
query82	432	153	127	127
query83	356	259	253	253
query84	258	114	91	91
query85	924	505	433	433
query86	385	298	302	298
query87	3640	3788	3593	3593
query88	2918	2283	2252	2252
query89	372	331	289	289
query90	1669	219	221	219
query91	167	166	139	139
query92	73	62	64	62
query93	1154	995	672	672
query94	740	453	345	345
query95	412	325	316	316
query96	496	563	280	280
query97	2908	2961	2919	2919
query98	236	218	210	210
query99	1310	1408	1303	1303
Total cold run time: 272208 ms
Total hot run time: 188940 ms

@doris-robot

doris-robot commented Nov 12, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 16987f3139c1daa541a220b7bc4b494938993572, data reload: false
query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.08
query4	1.61	0.12	0.12
query5	0.28	0.25	0.25
query6	1.18	0.64	0.64
query7	0.03	0.03	0.02
query8	0.05	0.04	0.04
query9	0.60	0.53	0.52
query10	0.59	0.58	0.57
query11	0.16	0.11	0.11
query12	0.16	0.12	0.13
query13	0.62	0.60	0.59
query14	1.01	1.01	1.01
query15	0.85	0.84	0.84
query16	0.40	0.39	0.39
query17	1.02	0.99	1.02
query18	0.22	0.20	0.20
query19	1.94	1.75	1.83
query20	0.02	0.02	0.01
query21	15.47	0.18	0.12
query22	5.06	0.07	0.05
query23	15.65	0.28	0.11
query24	2.64	0.78	0.43
query25	0.07	0.06	0.06
query26	0.13	0.13	0.14
query27	0.06	0.06	0.05
query28	3.66	1.14	0.92
query29	12.57	3.90	3.17
query30	0.29	0.13	0.11
query31	2.83	0.60	0.39
query32	3.23	0.55	0.46
query33	3.08	3.06	3.08
query34	15.88	5.18	4.58
query35	4.61	4.55	4.57
query36	0.67	0.51	0.49
query37	0.10	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.13	0.14
query41	0.09	0.03	0.03
query42	0.04	0.02	0.02
query43	0.05	0.04	0.03
Total cold run time: 97.6 s
Total hot run time: 27.38 s

@CalvinKirs

CalvinKirs commented Nov 12, 2025

Copy link
Copy Markdown
MemberAuthor

run buildall

@doris-robot

doris-robot commented Nov 12, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 35029 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f5cd12d3afabfa15466e2dc6eeb6ca9d054c9d12, data reload: false
------ Round 1 ----------------------------------
q1	17609	5114	4979	4979
q2	2018	322	202	202
q3	10245	1276	720	720
q4	10236	892	358	358
q5	7554	2325	2325	2325
q6	186	167	134	134
q7	922	779	635	635
q8	9325	1319	1043	1043
q9	6945	5199	5097	5097
q10	6878	2225	1806	1806
q11	528	311	308	308
q12	354	364	236	236
q13	18090	3655	3043	3043
q14	255	252	228	228
q15	583	508	510	508
q16	1070	1048	952	952
q17	604	890	375	375
q18	7646	7626	7840	7626
q19	1584	969	603	603
q20	371	390	244	244
q21	4312	3474	2550	2550
q22	1142	1127	1057	1057
Total cold run time: 108457 ms
Total hot run time: 35029 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5460	5300	5328	5300
q2	246	325	235	235
q3	2441	2929	2466	2466
q4	1486	1888	1470	1470
q5	4464	4706	4406	4406
q6	205	159	121	121
q7	1997	1972	1817	1817
q8	2604	2634	2492	2492
q9	7493	7125	7479	7125
q10	3041	3223	2672	2672
q11	574	509	495	495
q12	605	703	602	602
q13	3232	3635	3012	3012
q14	267	285	267	267
q15	528	504	495	495
q16	1024	1046	996	996
q17	1108	1414	1331	1331
q18	7425	7194	7046	7046
q19	753	752	852	752
q20	1908	1977	1842	1842
q21	4876	4264	4355	4264
q22	1119	1059	1029	1029
Total cold run time: 52856 ms
Total hot run time: 50235 ms

@doris-robot

doris-robot commented Nov 12, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 187980 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f5cd12d3afabfa15466e2dc6eeb6ca9d054c9d12, data reload: false
query1	1040	403	393	393
query2	6566	1711	1677	1677
query3	6775	229	220	220
query4	26306	23696	23312	23312
query5	4431	628	471	471
query6	341	245	256	245
query7	4642	505	293	293
query8	310	266	248	248
query9	8714	2618	2622	2618
query10	499	353	286	286
query11	15638	15025	14815	14815
query12	192	122	117	117
query13	1689	563	439	439
query14	11255	9223	9169	9169
query15	201	188	174	174
query16	7666	671	512	512
query17	1221	753	615	615
query18	2019	415	317	317
query19	212	202	170	170
query20	133	125	117	117
query21	217	135	114	114
query22	4147	4321	4030	4030
query23	33946	32969	32982	32969
query24	8524	2367	2404	2367
query25	578	510	456	456
query26	1227	272	161	161
query27	2756	498	347	347
query28	4364	2207	2201	2201
query29	823	618	483	483
query30	294	222	199	199
query31	925	809	694	694
query32	83	78	73	73
query33	585	379	342	342
query34	799	856	512	512
query35	813	841	745	745
query36	957	995	910	910
query37	112	107	82	82
query38	3539	3497	3469	3469
query39	1503	1434	1388	1388
query40	217	132	117	117
query41	63	61	61	61
query42	123	116	112	112
query43	505	489	466	466
query44	1216	742	739	739
query45	190	184	174	174
query46	882	987	630	630
query47	1745	1827	1751	1751
query48	381	415	322	322
query49	777	513	441	441
query50	641	680	400	400
query51	3917	4034	3950	3950
query52	114	113	105	105
query53	234	261	201	201
query54	304	292	276	276
query55	86	85	86	85
query56	366	325	317	317
query57	1192	1193	1131	1131
query58	294	271	261	261
query59	2555	2666	2561	2561
query60	347	363	325	325
query61	156	166	160	160
query62	826	743	668	668
query63	219	193	196	193
query64	4424	1173	870	870
query65	3992	3959	3934	3934
query66	1082	448	342	342
query67	15356	14996	15135	14996
query68	5001	937	593	593
query69	490	319	287	287
query70	1405	1280	1238	1238
query71	412	342	316	316
query72	6118	5164	5109	5109
query73	669	588	366	366
query74	8849	9199	8723	8723
query75	3314	3342	2824	2824
query76	3336	1150	788	788
query77	533	417	317	317
query78	9577	9796	8958	8958
query79	2183	887	594	594
query80	697	585	517	517
query81	536	259	225	225
query82	221	174	139	139
query83	264	264	248	248
query84	252	114	95	95
query85	892	489	438	438
query86	383	305	277	277
query87	3694	3764	3620	3620
query88	3152	2278	2261	2261
query89	386	334	289	289
query90	1970	224	221	221
query91	168	170	140	140
query92	83	68	66	66
query93	2453	997	641	641
query94	725	456	331	331
query95	415	326	321	321
query96	490	563	282	282
query97	2908	2981	2903	2903
query98	247	235	214	214
query99	1342	1407	1301	1301
Total cold run time: 271411 ms
Total hot run time: 187980 ms

@doris-robot

doris-robot commented Nov 12, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.42 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f5cd12d3afabfa15466e2dc6eeb6ca9d054c9d12, data reload: false
query1	0.05	0.05	0.05
query2	0.09	0.06	0.05
query3	0.26	0.09	0.08
query4	1.60	0.11	0.11
query5	0.27	0.25	0.25
query6	1.18	0.65	0.64
query7	0.03	0.02	0.03
query8	0.05	0.04	0.04
query9	0.58	0.53	0.52
query10	0.58	0.57	0.56
query11	0.16	0.11	0.12
query12	0.16	0.12	0.12
query13	0.63	0.60	0.59
query14	1.00	1.02	0.99
query15	0.84	0.84	0.84
query16	0.40	0.39	0.40
query17	0.99	1.01	1.06
query18	0.21	0.19	0.19
query19	1.92	1.74	1.84
query20	0.02	0.02	0.01
query21	15.46	0.19	0.13
query22	5.13	0.06	0.05
query23	15.65	0.26	0.10
query24	2.04	1.16	0.44
query25	0.08	0.07	0.07
query26	0.15	0.13	0.13
query27	0.06	0.06	0.06
query28	4.92	1.15	0.95
query29	12.57	3.89	3.29
query30	0.28	0.14	0.12
query31	2.82	0.58	0.38
query32	3.23	0.55	0.48
query33	3.08	3.11	3.03
query34	15.86	5.15	4.52
query35	4.55	4.60	4.52
query36	0.70	0.52	0.48
query37	0.10	0.07	0.07
query38	0.07	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.04
Total cold run time: 98.14 s
Total hot run time: 27.42 s

@CalvinKirs

CalvinKirs commented Nov 14, 2025

Copy link
Copy Markdown
MemberAuthor

run buildall

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 14, 2025
@github-actions

github-actionsBot commented Nov 14, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Nov 14, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@doris-robot

doris-robot commented Nov 14, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34587 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 36ea7b31748ebe35a4ee7eea0c13c3853a9eb3e1, data reload: false
------ Round 1 ----------------------------------
q1	17618	5195	5057	5057
q2	2010	316	236	236
q3	10218	1288	713	713
q4	10246	1012	403	403
q5	7490	2424	2346	2346
q6	185	175	138	138
q7	923	762	638	638
q8	9352	1366	1164	1164
q9	7057	5178	5213	5178
q10	6913	2244	1802	1802
q11	495	299	281	281
q12	363	383	237	237
q13	17774	3665	3045	3045
q14	234	235	209	209
q15	584	498	504	498
q16	1009	992	947	947
q17	602	879	358	358
q18	7621	7217	7193	7193
q19	1096	985	577	577
q20	355	340	232	232
q21	3745	3180	2343	2343
q22	1082	1031	992	992
Total cold run time: 106972 ms
Total hot run time: 34587 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5170	5152	5154	5152
q2	249	333	236	236
q3	2181	2648	2242	2242
q4	1378	1803	1323	1323
q5	4243	4585	4861	4585
q6	216	175	133	133
q7	2071	2005	1783	1783
q8	2636	2808	2573	2573
q9	7575	7348	7324	7324
q10	3214	3236	2822	2822
q11	594	527	506	506
q12	682	818	642	642
q13	3565	3903	3360	3360
q14	274	312	309	309
q15	563	527	537	527
q16	1078	1135	1071	1071
q17	1230	1619	1379	1379
q18	7761	7680	7546	7546
q19	838	827	964	827
q20	1985	2096	1905	1905
q21	5122	4609	4334	4334
q22	1107	1063	991	991
Total cold run time: 53732 ms
Total hot run time: 51570 ms

@doris-robot

doris-robot commented Nov 14, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 187671 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 36ea7b31748ebe35a4ee7eea0c13c3853a9eb3e1, data reload: false
query1	1044	411	429	411
query2	6549	1727	1722	1722
query3	6755	236	241	236
query4	26614	23286	23255	23255
query5	4370	615	472	472
query6	342	235	211	211
query7	4643	489	293	293
query8	298	252	259	252
query9	8701	2588	2596	2588
query10	495	337	293	293
query11	15528	15212	14825	14825
query12	171	115	113	113
query13	1673	568	438	438
query14	10255	9083	9137	9083
query15	201	187	166	166
query16	7148	695	516	516
query17	949	759	603	603
query18	1981	411	311	311
query19	209	211	183	183
query20	132	124	121	121
query21	216	133	117	117
query22	3995	4116	3961	3961
query23	33980	33070	33067	33067
query24	8455	2454	2392	2392
query25	607	523	480	480
query26	1245	281	158	158
query27	2747	493	354	354
query28	4396	2222	2214	2214
query29	813	601	484	484
query30	296	231	195	195
query31	902	792	719	719
query32	90	75	73	73
query33	591	378	327	327
query34	791	852	527	527
query35	811	841	739	739
query36	962	986	861	861
query37	121	116	87	87
query38	3516	3515	3478	3478
query39	1467	1393	1429	1393
query40	224	132	119	119
query41	62	61	62	61
query42	132	115	109	109
query43	489	506	474	474
query44	1233	748	754	748
query45	183	182	172	172
query46	880	985	637	637
query47	1771	1811	1757	1757
query48	390	422	314	314
query49	781	511	425	425
query50	667	681	413	413
query51	3866	3934	3818	3818
query52	114	112	103	103
query53	248	278	208	208
query54	323	323	293	293
query55	88	90	88	88
query56	335	340	322	322
query57	1155	1174	1130	1130
query58	300	279	285	279
query59	2513	2686	2461	2461
query60	357	367	356	356
query61	194	184	188	184
query62	800	712	671	671
query63	232	201	195	195
query64	4643	1297	839	839
query65	4057	3971	3956	3956
query66	1182	427	332	332
query67	15348	15034	15015	15015
query68	7411	917	602	602
query69	504	334	283	283
query70	1298	1253	1211	1211
query71	433	337	346	337
query72	5869	4944	4949	4944
query73	662	608	363	363
query74	8944	9105	8870	8870
query75	3284	3305	2820	2820
query76	3244	1147	724	724
query77	495	420	317	317
query78	9399	9621	8909	8909
query79	2006	832	596	596
query80	803	567	481	481
query81	544	263	231	231
query82	400	164	132	132
query83	285	270	253	253
query84	297	111	90	90
query85	889	477	443	443
query86	403	316	291	291
query87	3694	3740	3640	3640
query88	2838	2279	2264	2264
query89	385	331	290	290
query90	1837	228	218	218
query91	170	167	136	136
query92	79	69	67	67
query93	1231	994	642	642
query94	717	463	335	335
query95	404	323	309	309
query96	489	583	291	291
query97	2928	2956	2867	2867
query98	231	215	208	208
query99	1312	1402	1310	1310
Total cold run time: 270111 ms
Total hot run time: 187671 ms

@doris-robot

doris-robot commented Nov 14, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.48 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 36ea7b31748ebe35a4ee7eea0c13c3853a9eb3e1, data reload: false
query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.11	0.11
query5	0.28	0.26	0.25
query6	1.16	0.66	0.64
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.57	0.52	0.52
query10	0.58	0.58	0.57
query11	0.17	0.12	0.12
query12	0.14	0.12	0.12
query13	0.63	0.60	0.59
query14	1.00	1.00	1.00
query15	0.86	0.84	0.83
query16	0.40	0.39	0.40
query17	1.00	1.01	1.01
query18	0.21	0.20	0.20
query19	1.88	1.82	1.81
query20	0.02	0.01	0.02
query21	15.44	0.20	0.14
query22	5.09	0.07	0.04
query23	15.69	0.26	0.10
query24	3.04	1.21	0.39
query25	0.09	0.07	0.07
query26	0.14	0.14	0.14
query27	0.06	0.05	0.06
query28	4.35	1.12	0.93
query29	12.57	3.86	3.28
query30	0.28	0.13	0.12
query31	2.82	0.60	0.38
query32	3.23	0.57	0.46
query33	3.01	3.03	3.07
query34	15.86	5.15	4.53
query35	4.55	4.57	4.60
query36	0.68	0.52	0.49
query37	0.10	0.07	0.06
query38	0.06	0.04	0.03
query39	0.03	0.03	0.03
query40	0.17	0.14	0.13
query41	0.09	0.03	0.03
query42	0.03	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.41 s
Total hot run time: 27.48 s

@hello-stephen

hello-stephen commented Nov 14, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 79.25% (42/53) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented Nov 14, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 1.89% (1/53) 🎉
Increment coverage report
Complete coverage report

hubgeter
hubgeter approved these changes Nov 14, 2025
View reviewed changes

@hubgeterhubgeter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@CalvinKirs
CalvinKirs merged commit 8aa1a74 into apache:masterNov 14, 2025
30 of 31 checks passed
@CalvinKirs
CalvinKirs deleted the master-ms-onelake branch November 14, 2025 09:15
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Dec 1, 2025
@CalvinKirs
branch-4.0: [feat](catalog)Support OneLake (apache#57933)
7a40609
@CalvinKirsCalvinKirs mentioned this pull request Dec 1, 2025
Closed
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Dec 1, 2025
@CalvinKirs
[feat](catalog)Support OneLake (apache#57933)
b0e1d8d
@CalvinKirsCalvinKirs mentioned this pull request Dec 1, 2025
Merged
yiguolei pushed a commit that referenced this pull request Dec 2, 2025
@CalvinKirs
branch-4.0: [fix](paimon)Support user-defined S3 config prefixes and …
7ce9386
…unify to HDFS S3A protocol #57116 (#58544)
#57116#58467#57933
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
@CalvinKirs@nagisa-kunhah
[feat](catalog)Support OneLake (apache#57933)
9bf91d0
Microsoft OneLake, part of the Fabric ecosystem, is a unified and open
SaaS data lake that provides organizations with a centralized logical
data storage layer. Data in OneLake is stored in Parquet format and can
simultaneously maintain Delta Lake and Apache Iceberg metadata. This
design allows multiple analytics engines to directly access shared
datasets without data duplication or migration, significantly
simplifying data management and governance.
By leveraging Apache Doris’s Iceberg REST Catalog, users can directly
query and analyze data stored in OneLake—again, without copying or
moving it.
This integration enables building end-to-end analytics pipelines on a
single data lake, combining OneLake’s unified storage and governance
capabilities with Doris’s high-performance analytical compute.
At the technical level, Doris interacts with OneLake through open table
formats and standardized interfaces to access both metadata and Parquet
files. The architecture preserves centralized governance, access
control, and security policies—ensuring platform reliability,
scalability, and maintainability.
```
create catalog onelake_doris PROPERTIES (
'type' = 'iceberg',
'iceberg.catalog.type' = 'rest',
'uri'='https://onelake.table.fabric.microsoft.com/iceberg',
'warehouse'='<workerspace_id>/<data_item_id>',
'iceberg.rest.security.type'='oauth2',
'iceberg.rest.oauth2.server-uri'='https://login.microsoftonline.com/<talent_id>/oauth2/v2.0/token',
'iceberg.rest.oauth2.credential'='<oauth2.client_id>:'<oauth2.client_secret>,
'iceberg.rest.oauth2.scope'='https://storage.azure.com/.default', 'fs.azure.support'='true',
'azure.endpoint'='https://onelake.dfs.fabric.microsoft.com',
'azure.auth_type'='OAuth2',
'azure.oauth2_account_host'='onelake.dfs.fabric.microsoft.com',
'azure.oauth2_server_uri'='https://login.microsoftonline.com/<talent_id>/oauth2/v2.0/token',
'azure.oauth2_client_id'='<oauth2.client_id>',
'azure.oauth2_client_secret'='<oauth2.client_secret>'
);
```
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningmanmorningmanmorningman approved these changes
@hubgeterhubgeterhubgeter approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/4.0.2-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@CalvinKirs@hello-stephen@doris-robot@morningman@hubgeter@yiguolei