[Fix](tvf) Pass through user-defined properties by BePPPower · Pull Request #35515 · apache/doris

BePPPower · 2024-05-28T08:04:47Z

Proposed changes

Issue Number: close #xxx

Previously, irrelevant properties passed by users when using TVF were directly ignored by Doris.
Now, we retain and pass these additional user-defined properties to the S3 SDK.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-05-28T08:04:56Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

BePPPower · 2024-05-28T08:05:01Z

run buildall

doris-robot · 2024-05-28T09:19:07Z

TPC-H: Total hot run time: 40045 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1a173ec12b743aa6baaafa72abbd15ed45214f07, data reload: false
------ Round 1 ----------------------------------
q1	18043	5323	4354	4354
q2	2574	202	202	202
q3	12060	1183	1265	1183
q4	10512	844	758	758
q5	8013	2738	2699	2699
q6	220	130	135	130
q7	995	621	613	613
q8	9220	2081	2067	2067
q9	9006	6523	6510	6510
q10	8941	3716	3731	3716
q11	495	258	238	238
q12	471	222	224	222
q13	17778	2999	2973	2973
q14	262	216	217	216
q15	510	469	460	460
q16	521	393	376	376
q17	958	636	719	636
q18	8213	7418	7412	7412
q19	3025	1540	1519	1519
q20	654	309	316	309
q21	5059	3177	3294	3177
q22	352	275	288	275
Total cold run time: 117882 ms
Total hot run time: 40045 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4312	4234	4195	4195
q2	376	257	273	257
q3	2963	2799	2720	2720
q4	1866	1624	1669	1624
q5	5246	5318	5335	5318
q6	214	126	129	126
q7	2128	1701	1708	1701
q8	3187	3342	3317	3317
q9	8348	8372	8322	8322
q10	3929	3749	3683	3683
q11	581	490	487	487
q12	771	604	604	604
q13	17483	3045	2991	2991
q14	292	273	258	258
q15	523	479	481	479
q16	489	414	417	414
q17	1758	1510	1463	1463
q18	7639	7573	7454	7454
q19	3135	1584	1513	1513
q20	1995	1803	1755	1755
q21	7348	4661	4652	4652
q22	576	521	503	503
Total cold run time: 75159 ms
Total hot run time: 53836 ms

doris-robot · 2024-05-28T09:30:25Z

TPC-DS: Total hot run time: 173721 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1a173ec12b743aa6baaafa72abbd15ed45214f07, data reload: false
query1	923	396	388	388
query2	7895	2455	2339	2339
query3	6648	208	202	202
query4	19527	17325	17330	17325
query5	4166	429	416	416
query6	254	172	151	151
query7	4585	314	301	301
query8	239	188	179	179
query9	8478	2461	2417	2417
query10	446	286	261	261
query11	10467	10179	9926	9926
query12	137	94	89	89
query13	1631	371	360	360
query14	9396	7578	7580	7578
query15	224	173	171	171
query16	7809	269	271	269
query17	1828	540	539	539
query18	1957	279	301	279
query19	190	156	163	156
query20	94	87	83	83
query21	196	133	140	133
query22	4235	3950	3940	3940
query23	33669	33119	33049	33049
query24	11952	2901	2864	2864
query25	691	380	377	377
query26	1787	162	165	162
query27	2936	327	339	327
query28	7589	2106	2114	2106
query29	1168	640	633	633
query30	290	152	155	152
query31	972	764	731	731
query32	96	56	64	56
query33	781	295	291	291
query34	931	476	477	476
query35	774	626	639	626
query36	1088	926	913	913
query37	187	68	71	68
query38	2880	2772	2760	2760
query39	860	799	809	799
query40	274	133	132	132
query41	53	48	49	48
query42	109	102	98	98
query43	586	547	556	547
query44	1225	739	764	739
query45	186	163	168	163
query46	1062	730	752	730
query47	1827	1743	1772	1743
query48	379	303	301	301
query49	1198	396	403	396
query50	782	416	398	398
query51	6886	6873	6830	6830
query52	102	97	93	93
query53	363	288	289	288
query54	998	467	449	449
query55	74	74	76	74
query56	292	265	272	265
query57	1140	1034	1060	1034
query58	242	221	223	221
query59	3365	3077	3091	3077
query60	305	279	280	279
query61	110	107	106	106
query62	664	445	451	445
query63	316	293	295	293
query64	9905	2344	1832	1832
query65	3247	3150	3171	3150
query66	1397	338	332	332
query67	15632	14838	14873	14838
query68	7165	570	570	570
query69	527	304	289	289
query70	1191	1146	1117	1117
query71	503	291	282	282
query72	7835	5361	5475	5361
query73	856	330	333	330
query74	6131	5687	5623	5623
query75	4258	2601	2630	2601
query76	4665	1034	1001	1001
query77	653	270	273	270
query78	10446	9960	9746	9746
query79	7985	518	520	518
query80	1266	455	440	440
query81	488	221	227	221
query82	228	93	95	93
query83	213	176	173	173
query84	260	86	86	86
query85	960	279	266	266
query86	349	304	309	304
query87	3331	3119	3105	3105
query88	4880	2456	2449	2449
query89	533	388	393	388
query90	2152	195	208	195
query91	128	104	96	96
query92	62	51	54	51
query93	6543	536	502	502
query94	1377	191	192	191
query95	412	329	319	319
query96	604	269	269	269
query97	3166	3042	3009	3009
query98	242	221	220	220
query99	1178	843	853	843
Total cold run time: 293282 ms
Total hot run time: 173721 ms

doris-robot · 2024-05-28T09:35:56Z

ClickBench: Total hot run time: 31.14 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1a173ec12b743aa6baaafa72abbd15ed45214f07, data reload: false
query1	0.04	0.04	0.04
query2	0.09	0.04	0.05
query3	0.23	0.05	0.04
query4	1.67	0.09	0.09
query5	0.51	0.48	0.50
query6	1.12	0.72	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.47
query10	0.54	0.56	0.55
query11	0.16	0.11	0.11
query12	0.15	0.13	0.12
query13	0.59	0.59	0.59
query14	0.78	0.78	0.77
query15	0.83	0.81	0.81
query16	0.36	0.38	0.36
query17	0.94	1.05	0.97
query18	0.21	0.24	0.26
query19	1.87	1.67	1.73
query20	0.02	0.01	0.01
query21	15.71	0.69	0.65
query22	4.22	6.43	2.64
query23	18.35	1.42	1.26
query24	1.39	0.33	0.24
query25	0.14	0.09	0.07
query26	0.25	0.17	0.16
query27	0.08	0.08	0.08
query28	13.45	1.02	0.99
query29	13.19	3.30	3.27
query30	0.24	0.05	0.06
query31	2.89	0.39	0.38
query32	3.29	0.47	0.46
query33	2.86	2.87	2.95
query34	17.13	4.36	4.42
query35	4.47	4.49	4.48
query36	0.65	0.46	0.46
query37	0.18	0.16	0.15
query38	0.16	0.14	0.15
query39	0.04	0.03	0.04
query40	0.16	0.14	0.14
query41	0.09	0.04	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.76 s
Total hot run time: 31.14 s

BePPPower · 2024-05-31T03:32:42Z

run buildall

doris-robot · 2024-05-31T04:09:23Z

TPC-H: Total hot run time: 41699 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1a173ec12b743aa6baaafa72abbd15ed45214f07, data reload: false
------ Round 1 ----------------------------------
q1	17624	4294	4211	4211
q2	2021	192	190	190
q3	10470	1308	1236	1236
q4	10185	894	956	894
q5	7488	2693	2821	2693
q6	216	131	135	131
q7	959	627	618	618
q8	9221	2112	2074	2074
q9	9095	6696	6710	6696
q10	9213	3862	3940	3862
q11	444	243	234	234
q12	414	214	226	214
q13	17241	3148	3270	3148
q14	263	229	238	229
q15	511	452	472	452
q16	455	376	367	367
q17	986	720	697	697
q18	8394	7683	7816	7683
q19	3348	1607	1606	1606
q20	643	318	303	303
q21	5217	3833	4060	3833
q22	388	328	332	328
Total cold run time: 114796 ms
Total hot run time: 41699 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4495	4423	4407	4407
q2	374	274	275	274
q3	3171	2906	2812	2812
q4	1883	1581	1647	1581
q5	5465	5510	5494	5494
q6	216	124	121	121
q7	2143	1822	1798	1798
q8	3228	3395	3351	3351
q9	8501	8583	8665	8583
q10	4016	3660	3844	3660
q11	564	521	538	521
q12	827	638	604	604
q13	16852	3140	3196	3140
q14	302	280	274	274
q15	533	486	476	476
q16	507	435	434	434
q17	1836	1485	1457	1457
q18	7808	7842	7491	7491
q19	3649	1564	1557	1557
q20	2044	1782	1766	1766
q21	8274	4678	4816	4678
q22	603	541	513	513
Total cold run time: 77291 ms
Total hot run time: 54992 ms

doris-robot · 2024-05-31T04:19:50Z

TPC-DS: Total hot run time: 167914 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1a173ec12b743aa6baaafa72abbd15ed45214f07, data reload: false
query1	941	385	364	364
query2	6436	2482	2291	2291
query3	6630	200	198	198
query4	19410	17136	17342	17136
query5	4100	409	413	409
query6	239	171	151	151
query7	4584	316	295	295
query8	298	267	271	267
query9	8538	2379	2346	2346
query10	454	281	266	266
query11	10375	10207	10068	10068
query12	131	92	94	92
query13	1648	357	355	355
query14	10112	7364	6658	6658
query15	240	183	185	183
query16	7901	278	257	257
query17	1712	519	508	508
query18	2005	269	272	269
query19	200	147	155	147
query20	93	85	87	85
query21	195	131	126	126
query22	4160	3881	3807	3807
query23	33582	32789	32944	32789
query24	6731	2801	2833	2801
query25	502	353	350	350
query26	687	151	151	151
query27	1903	315	324	315
query28	3743	2031	2053	2031
query29	847	637	596	596
query30	224	147	150	147
query31	932	771	744	744
query32	82	53	53	53
query33	496	270	256	256
query34	839	467	458	458
query35	723	601	602	601
query36	1054	928	924	924
query37	101	71	66	66
query38	2879	2757	2819	2757
query39	872	808	780	780
query40	191	124	121	121
query41	52	51	50	50
query42	103	97	93	93
query43	575	547	562	547
query44	1100	716	723	716
query45	190	201	170	170
query46	1060	718	694	694
query47	1873	1772	1770	1770
query48	361	283	290	283
query49	764	372	376	372
query50	767	375	382	375
query51	6805	6620	6667	6620
query52	98	96	84	84
query53	345	288	279	279
query54	525	444	420	420
query55	72	73	73	73
query56	253	241	244	241
query57	1120	1030	1054	1030
query58	230	224	212	212
query59	3520	3207	3243	3207
query60	280	251	256	251
query61	87	85	90	85
query62	551	435	439	435
query63	309	282	277	277
query64	3299	1696	1790	1696
query65	3171	3099	3107	3099
query66	795	324	318	318
query67	15052	14623	14589	14589
query68	4589	528	528	528
query69	437	269	273	269
query70	1142	1082	1137	1082
query71	357	264	270	264
query72	7619	2714	2520	2520
query73	701	322	314	314
query74	5998	5563	5552	5552
query75	3269	2621	2606	2606
query76	2233	1002	1010	1002
query77	376	255	255	255
query78	10167	9836	10058	9836
query79	2263	506	508	506
query80	1015	443	432	432
query81	520	222	222	222
query82	599	91	97	91
query83	238	184	166	166
query84	247	84	85	84
query85	1566	261	263	261
query86	538	294	314	294
query87	3330	3138	3101	3101
query88	3979	2309	2337	2309
query89	478	390	376	376
query90	2053	185	179	179
query91	121	98	145	98
query92	56	46	51	46
query93	2394	510	499	499
query94	1205	182	185	182
query95	406	308	302	302
query96	595	264	261	261
query97	3196	3033	3008	3008
query98	244	216	208	208
query99	1177	857	876	857
Total cold run time: 253255 ms
Total hot run time: 167914 ms

doris-robot · 2024-05-31T04:25:20Z

ClickBench: Total hot run time: 30.65 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1a173ec12b743aa6baaafa72abbd15ed45214f07, data reload: false
query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.08	0.07
query5	0.50	0.50	0.50
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.52	0.49	0.47
query10	0.54	0.55	0.53
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.76	0.80	0.78
query15	0.82	0.81	0.81
query16	0.35	0.36	0.38
query17	0.99	1.02	1.01
query18	0.22	0.24	0.25
query19	1.90	1.64	1.67
query20	0.01	0.02	0.02
query21	15.42	0.72	0.69
query22	4.64	6.29	1.97
query23	18.27	1.29	1.25
query24	1.97	0.23	0.21
query25	0.13	0.08	0.09
query26	0.28	0.17	0.17
query27	0.09	0.08	0.08
query28	13.27	1.02	1.01
query29	12.78	3.32	3.30
query30	0.24	0.07	0.05
query31	2.87	0.37	0.36
query32	3.31	0.47	0.47
query33	2.85	2.87	2.92
query34	17.09	4.43	4.49
query35	4.51	4.52	4.61
query36	0.71	0.46	0.46
query37	0.17	0.15	0.16
query38	0.15	0.15	0.14
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.09	0.05	0.04
query42	0.06	0.05	0.05
query43	0.04	0.03	0.04
Total cold run time: 109.87 s
Total hot run time: 30.65 s

morningman

LGTM

github-actions · 2024-05-31T06:15:10Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-05-31T06:15:12Z

PR approved by anyone and no changes requested.

wsjz

LGTM

morningman · 2024-05-31T06:47:47Z

run buildall

doris-robot · 2024-05-31T07:12:21Z

TPC-H: Total hot run time: 39706 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ecc779ff501c750421b894d90db999e20aa1cd77, data reload: false
------ Round 1 ----------------------------------
q1	18228	4482	4378	4378
q2	2791	203	200	200
q3	11543	1131	1081	1081
q4	10557	891	705	705
q5	8289	2692	2674	2674
q6	226	144	148	144
q7	966	615	597	597
q8	9220	2049	2102	2049
q9	8752	6459	6444	6444
q10	8801	3675	3749	3675
q11	439	254	246	246
q12	451	228	232	228
q13	18951	2967	3010	2967
q14	256	218	236	218
q15	522	484	486	484
q16	516	416	387	387
q17	968	666	765	666
q18	8073	7509	7315	7315
q19	5362	1482	1452	1452
q20	645	325	325	325
q21	5032	3142	3934	3142
q22	394	335	329	329
Total cold run time: 120982 ms
Total hot run time: 39706 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4387	4238	4281	4238
q2	387	281	281	281
q3	2953	2771	2751	2751
q4	1863	1581	1600	1581
q5	5251	5254	5308	5254
q6	215	130	130	130
q7	2091	1768	1775	1768
q8	3219	3305	3331	3305
q9	8338	8321	8329	8321
q10	3891	3697	3732	3697
q11	597	486	507	486
q12	776	608	613	608
q13	17623	2993	3011	2993
q14	302	254	266	254
q15	510	473	486	473
q16	481	411	419	411
q17	1765	1492	1454	1454
q18	7729	7742	7532	7532
q19	1701	1479	1512	1479
q20	1980	1791	1803	1791
q21	4892	4713	4819	4713
q22	630	513	526	513
Total cold run time: 71581 ms
Total hot run time: 54033 ms

doris-robot · 2024-05-31T13:44:10Z

TPC-H: Total hot run time: 40133 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ecc779ff501c750421b894d90db999e20aa1cd77, data reload: false
------ Round 1 ----------------------------------
q1	17630	4430	4397	4397
q2	2033	195	193	193
q3	10495	1245	1133	1133
q4	10147	795	793	793
q5	7553	2673	2703	2673
q6	222	132	137	132
q7	956	613	594	594
q8	9212	2100	2080	2080
q9	9098	6514	6522	6514
q10	9008	3752	3726	3726
q11	448	263	236	236
q12	466	223	222	222
q13	17771	2995	2961	2961
q14	272	218	231	218
q15	526	467	472	467
q16	525	395	374	374
q17	973	700	687	687
q18	8072	7551	7399	7399
q19	1665	1495	1476	1476
q20	647	310	310	310
q21	4996	3228	3244	3228
q22	393	320	337	320
Total cold run time: 113108 ms
Total hot run time: 40133 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4296	4246	4234	4234
q2	382	266	265	265
q3	3006	2743	2806	2743
q4	1864	1592	1599	1592
q5	5267	5273	5310	5273
q6	219	126	124	124
q7	2153	1777	1697	1697
q8	3189	3314	3323	3314
q9	8384	8402	8413	8402
q10	3882	3640	3688	3640
q11	621	514	498	498
q12	777	611	627	611
q13	17337	2964	2980	2964
q14	308	266	261	261
q15	524	467	491	467
q16	496	419	426	419
q17	1822	1492	1487	1487
q18	7694	7317	7522	7317
q19	5117	1728	1521	1521
q20	2028	1771	1810	1771
q21	4854	4802	4831	4802
q22	596	541	535	535
Total cold run time: 74816 ms
Total hot run time: 53937 ms

doris-robot · 2024-05-31T13:55:05Z

TPC-DS: Total hot run time: 172701 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ecc779ff501c750421b894d90db999e20aa1cd77, data reload: false
query1	935	381	380	380
query2	6449	2528	2423	2423
query3	6644	209	208	208
query4	20312	17350	17271	17271
query5	4141	469	455	455
query6	273	163	156	156
query7	4605	294	285	285
query8	308	282	273	273
query9	8605	2376	2381	2376
query10	442	306	288	288
query11	10609	10208	10119	10119
query12	141	91	93	91
query13	1642	399	364	364
query14	10045	7693	6888	6888
query15	232	190	191	190
query16	7693	280	259	259
query17	1820	548	534	534
query18	1884	274	276	274
query19	208	161	159	159
query20	95	86	85	85
query21	204	135	130	130
query22	4241	4145	3917	3917
query23	33934	33122	33081	33081
query24	11081	2825	2822	2822
query25	582	357	376	357
query26	714	157	158	157
query27	2243	322	331	322
query28	5668	2105	2078	2078
query29	851	616	605	605
query30	281	147	155	147
query31	940	769	732	732
query32	90	51	52	51
query33	773	296	281	281
query34	965	468	462	462
query35	748	614	646	614
query36	1086	940	880	880
query37	132	69	74	69
query38	2935	2829	2800	2800
query39	891	806	782	782
query40	210	128	122	122
query41	57	54	49	49
query42	121	96	99	96
query43	580	559	564	559
query44	1181	729	765	729
query45	202	171	169	169
query46	1087	706	728	706
query47	1851	1772	1764	1764
query48	372	292	302	292
query49	1057	398	412	398
query50	778	396	389	389
query51	6722	6661	6660	6660
query52	106	90	90	90
query53	350	290	300	290
query54	900	457	452	452
query55	73	74	72	72
query56	303	249	260	249
query57	1126	1066	1074	1066
query58	251	251	238	238
query59	3528	3162	3228	3162
query60	302	272	267	267
query61	92	92	93	92
query62	663	446	447	446
query63	316	290	289	289
query64	8680	2237	1735	1735
query65	3167	3148	3128	3128
query66	794	335	330	330
query67	15647	14682	14736	14682
query68	4590	556	546	546
query69	468	303	321	303
query70	1164	1099	1154	1099
query71	379	289	283	283
query72	7118	5556	5234	5234
query73	749	328	320	320
query74	6140	5605	5733	5605
query75	3336	2691	2687	2687
query76	2390	889	932	889
query77	454	298	298	298
query78	10242	9911	9869	9869
query79	2759	511	508	508
query80	2256	456	456	456
query81	600	220	218	218
query82	776	108	106	106
query83	312	174	174	174
query84	264	88	85	85
query85	1880	317	352	317
query86	488	316	318	316
query87	3358	3177	3107	3107
query88	3685	2350	2342	2342
query89	474	409	372	372
query90	1783	190	195	190
query91	125	96	98	96
query92	62	47	52	47
query93	2015	521	507	507
query94	1206	211	186	186
query95	420	319	312	312
query96	581	273	267	267
query97	3190	2981	3051	2981
query98	250	224	217	217
query99	1235	878	884	878
Total cold run time: 270707 ms
Total hot run time: 172701 ms

doris-robot · 2024-05-31T14:00:36Z

ClickBench: Total hot run time: 29.85 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ecc779ff501c750421b894d90db999e20aa1cd77, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.05
query3	0.24	0.04	0.05
query4	1.68	0.07	0.08
query5	0.50	0.49	0.50
query6	1.12	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.53	0.50	0.50
query10	0.54	0.55	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.60
query14	0.78	0.78	0.78
query15	0.84	0.81	0.81
query16	0.37	0.36	0.35
query17	1.04	1.01	0.93
query18	0.21	0.26	0.22
query19	1.90	1.68	1.73
query20	0.02	0.01	0.00
query21	15.43	0.71	0.70
query22	4.98	7.31	1.21
query23	18.28	1.40	1.23
query24	1.55	0.31	0.21
query25	0.15	0.08	0.08
query26	0.26	0.17	0.17
query27	0.07	0.08	0.08
query28	13.37	1.01	1.00
query29	13.03	3.34	3.29
query30	0.24	0.06	0.06
query31	2.87	0.38	0.38
query32	3.29	0.46	0.47
query33	2.86	2.89	2.98
query34	17.13	4.41	4.45
query35	4.46	4.56	4.54
query36	0.68	0.47	0.47
query37	0.17	0.16	0.16
query38	0.15	0.15	0.15
query39	0.05	0.03	0.03
query40	0.17	0.14	0.15
query41	0.08	0.04	0.05
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.23 s
Total hot run time: 29.85 s

## Proposed changes Issue Number: close #xxx Previously, irrelevant properties passed by users when using TVF were directly ignored by Doris. Now, we retain and pass these additional user-defined properties to the S3 SDK.  ## Further comments If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...

bp #35515 Co-authored-by: Tiewei Fang <43782773+BePPPower@users.noreply.github.com>

## Proposed changes Introduced from #35515 1. Fix invalid `to_int()` method logic 2. Remove unnecessary properties when creating s3 resource Before, after recreating s3 resource, there will be some extra properties being added to the resource properties, such as AWS_ACCESS_KEY, but this keys are only for s3 client on BE side, don' t needed when ping s3. But it will add some invalid properties such as `AWS_TOKEN=null`

## Proposed changes Issue Number: close #xxx Previously, irrelevant properties passed by users when using TVF were directly ignored by Doris. Now, we retain and pass these additional user-defined properties to the S3 SDK.  ## Further comments If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...

## Proposed changes Introduced from #35515 1. Fix invalid `to_int()` method logic 2. Remove unnecessary properties when creating s3 resource Before, after recreating s3 resource, there will be some extra properties being added to the resource properties, such as AWS_ACCESS_KEY, but this keys are only for s3 client on BE side, don' t needed when ping s3. But it will add some invalid properties such as `AWS_TOKEN=null`

## Proposed changes Issue Number: close #xxx Previously, irrelevant properties passed by users when using TVF were directly ignored by Doris. Now, we retain and pass these additional user-defined properties to the S3 SDK.  ## Further comments If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...

## Proposed changes Introduced from apache#35515 1. Fix invalid `to_int()` method logic 2. Remove unnecessary properties when creating s3 resource Before, after recreating s3 resource, there will be some extra properties being added to the resource properties, such as AWS_ACCESS_KEY, but this keys are only for s3 client on BE side, don' t needed when ping s3. But it will add some invalid properties such as `AWS_TOKEN=null`

Issue Number: close #xxx Previously, irrelevant properties passed by users when using TVF were directly ignored by Doris. Now, we retain and pass these additional user-defined properties to the S3 SDK.  If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...

bp: #35515 Previously, irrelevant properties passed by users when using TVF were directly ignored by Doris. Now, we retain and pass these additional user-defined properties to the S3 SDK.

bp: apache#35515 Previously, irrelevant properties passed by users when using TVF were directly ignored by Doris. Now, we retain and pass these additional user-defined properties to the S3 SDK.

## Proposed changes Issue Number: close #xxx Previously, irrelevant properties passed by users when using TVF were directly ignored by Doris. Now, we retain and pass these additional user-defined properties to the S3 SDK.  ## Further comments If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...

## Proposed changes Introduced from apache#35515 1. Fix invalid `to_int()` method logic 2. Remove unnecessary properties when creating s3 resource Before, after recreating s3 resource, there will be some extra properties being added to the resource properties, such as AWS_ACCESS_KEY, but this keys are only for s3 client on BE side, don' t needed when ping s3. But it will add some invalid properties such as `AWS_TOKEN=null`

morningman approved these changes May 31, 2024
View reviewed changes

morningman added usercase Important user case type label dev/2.1.x labels May 31, 2024

github-actionsBot added the approved Indicates a PR has been approved by one committer. label May 31, 2024

github-actionsBot added the reviewed label May 31, 2024

wsjz approved these changes May 31, 2024
View reviewed changes

fix
ecc779f

morningman force-pushed the fixTvfProperties branch from 1a173ec to ecc779fCompare May 31, 2024 06:47

morningman approved these changes May 31, 2024
View reviewed changes

yiguolei merged commit f53aee4 into apache:masterMay 31, 2024

morningman mentioned this pull request May 31, 2024
[Fix](tvf) Pass through user-defined properties (#35515) #35747
Merged

morningman added a commit that referenced this pull request May 31, 2024
[Fix](tvf) Pass through user-defined properties (#35515) (#35747)
9468227
bp #35515 Co-authored-by: Tiewei Fang <43782773+BePPPower@users.noreply.github.com>

morningman removed the dev/2.1.x label May 31, 2024

morningman added the dev/2.1.4-merged label May 31, 2024

yiguolei mentioned this pull request Jun 1, 2024
Release Notes 2.1.4 #35752
Closed

morningman mentioned this pull request Jun 1, 2024
[fix](s3) fix invalid s3 properties checking logic #35762
Merged

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

morningman added the dev/2.0.x label Aug 12, 2024

BePPPower mentioned this pull request Aug 13, 2024
[Fix](tvf) Pass through user-defined properties #39285
Merged

morningman added dev/2.0.15-merged and removed dev/2.0.x labels Aug 14, 2024

Uh oh!

Conversation

BePPPower commented May 28, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

doris-robot commented May 28, 2024

Uh oh!

BePPPower commented May 28, 2024

Uh oh!

doris-robot commented May 28, 2024

Uh oh!

doris-robot commented May 28, 2024

Uh oh!

doris-robot commented May 28, 2024

Uh oh!

BePPPower commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented May 31, 2024

Uh oh!

github-actionsBot commented May 31, 2024

Uh oh!

wsjz left a comment

Choose a reason for hiding this comment

Uh oh!

morningman commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

BePPPower commented May 28, 2024•
edited
Loading