[fix](tvf) Partition columns in CTAS need to be compatible with the STRING type of external tables/TVF by BePPPower · Pull Request #35489 · apache/doris

BePPPower · 2024-05-28T03:38:22Z

Proposed changes

Issue Number: close #xxx

Partition columns in CTAS need to be compatible with the STRING type of external tables/TVF

For TVF queries with partition columns in the file path, map these partition columns to the VARCHAR(65533) type. (Previously, they were of STRING type.)
If you use the STRING type of an external table as a partition column, CTAS will automatically convert it to VARCHAR(65535) type."

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-05-28T03:38:27Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

BePPPower · 2024-05-28T03:41:11Z

run buildall

BePPPower · 2024-05-28T07:07:30Z

run buildall

doris-robot · 2024-05-28T07:48:46Z

TPC-H: Total hot run time: 41484 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 46daf70f66ddb80723e70898e25c282ee99fc416, data reload: false
------ Round 1 ----------------------------------
q1	17609	4621	4276	4276
q2	2019	204	206	204
q3	10416	1187	1201	1187
q4	10202	749	857	749
q5	7477	2716	2722	2716
q6	235	135	140	135
q7	973	612	607	607
q8	9228	2113	2084	2084
q9	9947	6713	6735	6713
q10	9581	3947	3955	3947
q11	462	250	254	250
q12	535	232	225	225
q13	18021	3249	3268	3249
q14	258	214	211	211
q15	512	489	469	469
q16	513	396	394	394
q17	980	758	707	707
q18	8351	7754	7854	7754
q19	3216	1568	1558	1558
q20	643	331	313	313
q21	5222	3455	4012	3455
q22	353	287	281	281
Total cold run time: 116753 ms
Total hot run time: 41484 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4579	4452	4456	4452
q2	367	275	277	275
q3	3159	2922	2945	2922
q4	1946	1617	1594	1594
q5	5380	5492	5533	5492
q6	217	126	125	125
q7	2203	1804	1813	1804
q8	3281	3425	3399	3399
q9	8608	8672	8772	8672
q10	4063	3727	3760	3727
q11	612	515	520	515
q12	814	628	658	628
q13	17076	3137	3174	3137
q14	317	289	279	279
q15	521	484	487	484
q16	495	437	427	427
q17	1776	1517	1484	1484
q18	7843	7551	7385	7385
q19	1694	1576	1567	1567
q20	2082	1779	1790	1779
q21	13896	4750	4677	4677
q22	561	483	501	483
Total cold run time: 81490 ms
Total hot run time: 55307 ms

doris-robot · 2024-05-28T07:59:18Z

TPC-DS: Total hot run time: 172657 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 46daf70f66ddb80723e70898e25c282ee99fc416, data reload: false
query1	936	391	383	383
query2	6454	2388	2267	2267
query3	6636	205	204	204
query4	19885	17350	17434	17350
query5	4148	421	422	421
query6	263	158	151	151
query7	4586	303	293	293
query8	237	190	181	181
query9	8658	2468	2438	2438
query10	462	306	267	267
query11	10674	10085	10059	10059
query12	139	91	91	91
query13	1651	370	373	370
query14	8482	7609	7582	7582
query15	210	165	173	165
query16	7226	258	261	258
query17	1311	532	516	516
query18	1844	274	266	266
query19	197	161	153	153
query20	90	88	87	87
query21	196	138	130	130
query22	4166	4176	4171	4171
query23	33475	32924	33093	32924
query24	5549	2857	2946	2857
query25	464	352	371	352
query26	698	154	157	154
query27	1846	345	328	328
query28	3753	2084	2104	2084
query29	838	600	597	597
query30	233	147	152	147
query31	952	750	728	728
query32	61	53	55	53
query33	500	280	263	263
query34	880	491	483	483
query35	706	605	591	591
query36	1048	910	899	899
query37	109	66	69	66
query38	2860	2759	2769	2759
query39	891	801	785	785
query40	196	125	123	123
query41	45	43	43	43
query42	108	96	96	96
query43	604	544	548	544
query44	1076	725	741	725
query45	178	167	163	163
query46	1054	733	732	732
query47	1846	1759	1789	1759
query48	382	305	287	287
query49	769	384	400	384
query50	784	410	395	395
query51	6896	6726	6619	6619
query52	108	91	95	91
query53	367	287	291	287
query54	541	430	424	424
query55	73	73	69	69
query56	277	250	300	250
query57	1079	1069	1029	1029
query58	226	209	218	209
query59	3160	3074	3129	3074
query60	294	262	255	255
query61	91	91	91	91
query62	553	436	458	436
query63	328	289	282	282
query64	2580	1814	1808	1808
query65	3164	3081	3134	3081
query66	921	337	329	329
query67	15396	14885	14742	14742
query68	4559	547	545	545
query69	453	269	276	269
query70	1149	1126	1148	1126
query71	360	274	266	266
query72	8013	5165	5403	5165
query73	726	333	329	329
query74	6124	5552	5610	5552
query75	3299	2680	2576	2576
query76	2292	960	985	960
query77	389	263	264	263
query78	10328	9762	9763	9762
query79	2353	518	509	509
query80	890	502	442	442
query81	501	232	226	226
query82	902	91	91	91
query83	242	167	165	165
query84	251	85	83	83
query85	1077	277	261	261
query86	471	323	301	301
query87	3340	3091	3113	3091
query88	4146	2457	2433	2433
query89	490	388	374	374
query90	2089	196	191	191
query91	122	98	96	96
query92	60	49	50	49
query93	2080	565	505	505
query94	1147	189	187	187
query95	395	311	314	311
query96	592	270	273	270
query97	3188	3006	3021	3006
query98	235	224	219	219
query99	1233	860	834	834
Total cold run time: 249906 ms
Total hot run time: 172657 ms

doris-robot · 2024-05-28T08:04:50Z

ClickBench: Total hot run time: 29.65 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 46daf70f66ddb80723e70898e25c282ee99fc416, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.05
query3	0.23	0.05	0.04
query4	1.68	0.07	0.07
query5	0.50	0.47	0.50
query6	1.12	0.72	0.71
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.53	0.49	0.50
query10	0.55	0.56	0.53
query11	0.17	0.11	0.11
query12	0.15	0.13	0.12
query13	0.59	0.59	0.61
query14	0.78	0.78	0.78
query15	0.84	0.82	0.80
query16	0.35	0.35	0.37
query17	0.94	0.96	0.96
query18	0.22	0.24	0.24
query19	1.77	1.74	1.71
query20	0.02	0.01	0.01
query21	15.64	0.66	0.66
query22	4.21	8.16	1.14
query23	18.33	1.42	1.30
query24	1.83	0.24	0.21
query25	0.16	0.09	0.08
query26	0.26	0.16	0.16
query27	0.07	0.07	0.07
query28	13.33	1.01	0.99
query29	13.80	3.33	3.28
query30	0.24	0.05	0.06
query31	2.86	0.38	0.37
query32	3.29	0.47	0.47
query33	2.90	2.86	2.95
query34	17.22	4.39	4.41
query35	4.66	4.51	4.65
query36	0.65	0.45	0.47
query37	0.17	0.15	0.15
query38	0.16	0.15	0.14
query39	0.04	0.04	0.03
query40	0.16	0.13	0.14
query41	0.09	0.04	0.05
query42	0.05	0.04	0.04
query43	0.04	0.03	0.03
Total cold run time: 110.78 s
Total hot run time: 29.65 s

BePPPower · 2024-05-29T07:25:34Z

run buildall

doris-robot · 2024-05-29T08:37:24Z

TPC-H: Total hot run time: 41732 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 46daf70f66ddb80723e70898e25c282ee99fc416, data reload: false
------ Round 1 ----------------------------------
q1	17619	4877	4236	4236
q2	2033	198	195	195
q3	10477	1183	1203	1183
q4	10207	793	787	787
q5	7469	2738	2824	2738
q6	217	138	136	136
q7	963	624	617	617
q8	9220	2163	2099	2099
q9	9429	6713	6713	6713
q10	9324	3988	3892	3892
q11	447	241	239	239
q12	530	225	219	219
q13	17381	3166	3239	3166
q14	283	228	225	225
q15	522	473	473	473
q16	524	398	378	378
q17	1011	679	734	679
q18	8408	7847	7774	7774
q19	6350	1608	1537	1537
q20	669	310	321	310
q21	5290	3988	3860	3860
q22	358	283	276	276
Total cold run time: 118731 ms
Total hot run time: 41732 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4499	4436	4337	4337
q2	371	276	273	273
q3	3206	2974	2799	2799
q4	1873	1581	1689	1581
q5	5491	5497	5495	5495
q6	218	125	121	121
q7	2257	1826	1863	1826
q8	3253	3324	3394	3324
q9	8686	8724	8609	8609
q10	3866	3886	3862	3862
q11	605	492	499	492
q12	794	606	647	606
q13	15814	3139	3230	3139
q14	313	270	277	270
q15	517	486	487	486
q16	505	433	408	408
q17	1793	1497	1473	1473
q18	7628	7494	7303	7303
q19	1662	1557	1613	1557
q20	1967	1766	1802	1766
q21	4880	4731	4778	4731
q22	555	481	475	475
Total cold run time: 70753 ms
Total hot run time: 54933 ms

doris-robot · 2024-05-29T08:47:58Z

TPC-DS: Total hot run time: 171530 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 46daf70f66ddb80723e70898e25c282ee99fc416, data reload: false
query1	962	373	376	373
query2	6454	2457	2341	2341
query3	6641	205	202	202
query4	19731	17289	17459	17289
query5	4175	414	433	414
query6	268	169	176	169
query7	4585	294	292	292
query8	250	188	177	177
query9	8509	2356	2331	2331
query10	437	277	261	261
query11	10546	10122	10096	10096
query12	137	93	86	86
query13	1634	375	375	375
query14	9640	7781	7002	7002
query15	219	177	167	167
query16	7560	257	256	256
query17	1735	527	533	527
query18	1866	269	289	269
query19	198	152	144	144
query20	93	89	83	83
query21	204	130	132	130
query22	4188	3919	3890	3890
query23	33546	32718	32992	32718
query24	6596	2840	2884	2840
query25	534	369	355	355
query26	694	158	158	158
query27	1894	313	327	313
query28	3976	2100	2068	2068
query29	849	620	587	587
query30	231	151	154	151
query31	954	774	762	762
query32	63	57	53	53
query33	497	275	266	266
query34	868	490	468	468
query35	739	612	599	599
query36	1057	919	925	919
query37	106	63	69	63
query38	2905	2761	2738	2738
query39	848	785	796	785
query40	198	124	124	124
query41	46	44	47	44
query42	104	94	96	94
query43	591	549	544	544
query44	1093	736	753	736
query45	177	165	163	163
query46	1053	691	721	691
query47	1850	1757	1772	1757
query48	370	298	306	298
query49	760	376	408	376
query50	769	381	392	381
query51	6882	6644	6708	6644
query52	99	91	91	91
query53	353	285	287	285
query54	566	448	427	427
query55	74	70	74	70
query56	297	246	249	246
query57	1163	1052	1025	1025
query58	224	208	213	208
query59	3276	3189	3170	3170
query60	273	258	258	258
query61	90	89	88	88
query62	555	439	467	439
query63	305	285	281	281
query64	2848	1833	1699	1699
query65	3171	3088	3117	3088
query66	815	326	320	320
query67	15291	14675	14683	14675
query68	4578	540	553	540
query69	441	269	267	267
query70	1168	1089	1057	1057
query71	385	265	268	265
query72	7342	5669	5263	5263
query73	733	323	332	323
query74	6032	5600	5599	5599
query75	3424	2616	2634	2616
query76	2258	971	960	960
query77	408	264	262	262
query78	10300	9748	9685	9685
query79	2679	514	518	514
query80	1017	438	443	438
query81	500	224	217	217
query82	866	89	90	89
query83	247	172	165	165
query84	244	88	85	85
query85	1160	279	270	270
query86	453	319	290	290
query87	3301	3130	3123	3123
query88	3947	2458	2457	2457
query89	478	382	392	382
query90	2036	193	189	189
query91	137	110	111	110
query92	68	51	52	51
query93	2020	526	505	505
query94	1170	201	205	201
query95	408	323	375	323
query96	606	272	271	271
query97	3154	3014	3042	3014
query98	249	217	213	213
query99	1181	847	854	847
Total cold run time: 252672 ms
Total hot run time: 171530 ms

doris-robot · 2024-05-29T08:53:26Z

ClickBench: Total hot run time: 30.96 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 46daf70f66ddb80723e70898e25c282ee99fc416, data reload: false
query1	0.04	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.05	0.06
query4	1.67	0.10	0.09
query5	0.51	0.48	0.50
query6	1.12	0.72	0.72
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.53	0.46	0.48
query10	0.55	0.55	0.54
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.59	0.59	0.59
query14	0.78	0.78	0.78
query15	0.82	0.81	0.81
query16	0.36	0.36	0.36
query17	1.00	1.01	1.03
query18	0.22	0.25	0.23
query19	1.78	1.66	1.74
query20	0.01	0.02	0.01
query21	15.49	0.69	0.68
query22	4.26	6.34	2.40
query23	18.31	1.38	1.23
query24	1.80	0.24	0.21
query25	0.14	0.08	0.08
query26	0.26	0.17	0.17
query27	0.08	0.08	0.08
query28	13.28	1.02	1.00
query29	13.13	3.37	3.36
query30	0.23	0.06	0.05
query31	2.87	0.39	0.38
query32	3.28	0.46	0.46
query33	2.88	2.86	2.88
query34	17.12	4.42	4.38
query35	4.55	4.47	4.53
query36	0.68	0.47	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.16	0.14	0.14
query41	0.09	0.05	0.04
query42	0.05	0.04	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.7 s
Total hot run time: 30.96 s

BePPPower · 2024-05-29T09:48:33Z

run buildall

doris-robot · 2024-05-29T10:43:25Z

TPC-H: Total hot run time: 40241 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cdacbfd84d4651805ce264052f0e57970675582b, data reload: false
------ Round 1 ----------------------------------
q1	17628	4514	4277	4277
q2	2028	200	199	199
q3	10459	1213	1304	1213
q4	10150	818	788	788
q5	7465	2675	2692	2675
q6	214	130	129	129
q7	962	598	577	577
q8	9220	2082	2064	2064
q9	8902	6555	6538	6538
q10	8918	3723	3702	3702
q11	474	246	239	239
q12	420	217	220	217
q13	18142	2992	2949	2949
q14	257	218	236	218
q15	513	481	480	480
q16	512	392	376	376
q17	983	690	709	690
q18	8110	7484	7728	7484
q19	3615	1609	1553	1553
q20	651	311	310	310
q21	5043	3248	3317	3248
q22	389	325	315	315
Total cold run time: 115055 ms
Total hot run time: 40241 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4285	4201	4214	4201
q2	367	270	269	269
q3	3001	2791	2713	2713
q4	1876	1613	1606	1606
q5	5262	5307	5261	5261
q6	212	124	127	124
q7	2132	1762	1715	1715
q8	3196	3310	3297	3297
q9	8372	8372	8361	8361
q10	3905	3681	3742	3681
q11	603	487	488	487
q12	765	595	574	574
q13	17390	2969	3005	2969
q14	277	256	272	256
q15	520	466	480	466
q16	474	411	415	411
q17	1799	1485	1492	1485
q18	7683	7512	7258	7258
q19	2049	1620	1606	1606
q20	1995	1813	1768	1768
q21	4955	4727	4712	4712
q22	578	560	511	511
Total cold run time: 71696 ms
Total hot run time: 53731 ms

doris-robot · 2024-05-29T10:54:19Z

TPC-DS: Total hot run time: 168917 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cdacbfd84d4651805ce264052f0e57970675582b, data reload: false
query1	924	372	370	370
query2	6451	2343	2310	2310
query3	6653	213	226	213
query4	19304	17491	17250	17250
query5	4138	412	414	412
query6	261	159	167	159
query7	4591	296	288	288
query8	313	289	286	286
query9	8557	2417	2392	2392
query10	458	280	261	261
query11	10547	9938	9938	9938
query12	133	93	84	84
query13	1640	376	365	365
query14	10020	7369	6770	6770
query15	230	188	180	180
query16	7809	261	260	260
query17	1853	516	501	501
query18	1988	265	265	265
query19	194	147	150	147
query20	94	92	88	88
query21	208	129	125	125
query22	4391	3973	4071	3973
query23	33716	32841	32974	32841
query24	11983	2887	2870	2870
query25	620	346	372	346
query26	1691	150	151	150
query27	2953	323	327	323
query28	7493	2069	2086	2069
query29	1010	597	603	597
query30	277	145	151	145
query31	963	754	741	741
query32	91	51	54	51
query33	754	268	254	254
query34	1007	467	467	467
query35	724	593	595	593
query36	1091	905	920	905
query37	138	65	69	65
query38	2930	2771	2757	2757
query39	868	785	785	785
query40	269	122	124	122
query41	53	77	49	49
query42	99	100	94	94
query43	580	558	543	543
query44	1206	732	757	732
query45	175	171	166	166
query46	1067	698	721	698
query47	1850	1734	1769	1734
query48	359	291	295	291
query49	1135	375	395	375
query50	760	389	378	378
query51	6866	6758	6736	6736
query52	108	90	92	90
query53	353	294	281	281
query54	904	433	418	418
query55	73	73	73	73
query56	258	236	242	236
query57	1123	1034	1014	1014
query58	231	213	216	213
query59	3547	3244	3247	3244
query60	306	257	253	253
query61	88	84	85	84
query62	663	456	455	455
query63	319	283	282	282
query64	9782	2191	1726	1726
query65	3161	3060	3099	3060
query66	1103	377	322	322
query67	15450	14813	14910	14813
query68	4584	525	534	525
query69	448	272	264	264
query70	1112	1122	1128	1122
query71	407	268	263	263
query72	7740	5047	2695	2695
query73	701	315	317	315
query74	6037	5621	5531	5531
query75	3330	2615	2611	2611
query76	2909	986	917	917
query77	419	264	260	260
query78	10259	9989	9824	9824
query79	2092	519	510	510
query80	882	439	488	439
query81	538	222	219	219
query82	656	89	90	89
query83	280	170	169	169
query84	238	85	82	82
query85	1747	269	269	269
query86	497	280	306	280
query87	3260	3107	3068	3068
query88	4077	2322	2323	2322
query89	490	407	375	375
query90	2013	184	191	184
query91	124	95	98	95
query92	64	48	52	48
query93	2369	511	498	498
query94	1236	181	183	181
query95	409	299	305	299
query96	578	268	266	266
query97	3167	3038	3039	3038
query98	239	232	237	232
query99	1144	849	914	849
Total cold run time: 275000 ms
Total hot run time: 168917 ms

doris-robot · 2024-05-29T10:59:47Z

ClickBench: Total hot run time: 30.42 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cdacbfd84d4651805ce264052f0e57970675582b, data reload: false
query1	0.03	0.03	0.04
query2	0.09	0.04	0.03
query3	0.23	0.06	0.05
query4	1.67	0.07	0.07
query5	0.49	0.47	0.48
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.53	0.49	0.47
query10	0.54	0.55	0.55
query11	0.15	0.11	0.11
query12	0.14	0.12	0.11
query13	0.59	0.60	0.59
query14	0.78	0.77	0.78
query15	0.83	0.82	0.80
query16	0.36	0.35	0.35
query17	0.95	0.98	0.99
query18	0.20	0.28	0.22
query19	1.87	1.65	1.67
query20	0.02	0.00	0.01
query21	15.46	0.72	0.67
query22	4.37	7.12	1.97
query23	18.30	1.38	1.21
query24	1.78	0.30	0.21
query25	0.14	0.08	0.08
query26	0.26	0.16	0.18
query27	0.08	0.08	0.08
query28	13.30	1.00	0.98
query29	13.16	3.31	3.30
query30	0.24	0.07	0.07
query31	2.85	0.37	0.38
query32	3.29	0.47	0.48
query33	2.86	2.94	2.86
query34	17.08	4.40	4.45
query35	4.50	4.51	4.51
query36	0.65	0.45	0.47
query37	0.18	0.15	0.15
query38	0.14	0.16	0.15
query39	0.05	0.04	0.03
query40	0.17	0.14	0.15
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.03	0.03
Total cold run time: 109.73 s
Total hot run time: 30.42 s

BePPPower · 2024-05-29T12:53:53Z

run buildall

doris-robot · 2024-05-29T13:23:03Z

TPC-H: Total hot run time: 40639 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8798afd88e4e907aa6e6f4566013fa17ab9f0056, data reload: false
------ Round 1 ----------------------------------
q1	18005	4523	4411	4411
q2	2544	196	194	194
q3	11903	1190	1212	1190
q4	10298	827	841	827
q5	8764	2815	2662	2662
q6	211	131	130	130
q7	942	593	591	591
q8	9232	2081	2058	2058
q9	9032	6465	6495	6465
q10	8834	3631	3649	3631
q11	430	244	237	237
q12	432	217	222	217
q13	17775	3001	3012	3001
q14	272	222	215	215
q15	515	466	470	466
q16	526	393	374	374
q17	961	673	734	673
q18	8003	7514	7359	7359
q19	4966	1585	1519	1519
q20	656	311	308	308
q21	4919	3790	3869	3790
q22	395	321	324	321
Total cold run time: 119615 ms
Total hot run time: 40639 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4330	4201	4210	4201
q2	367	271	260	260
q3	2984	2764	2748	2748
q4	1887	1609	1581	1581
q5	5253	5309	5272	5272
q6	209	122	123	122
q7	2142	1702	1761	1702
q8	3198	3306	3309	3306
q9	8378	8381	8353	8353
q10	3846	3686	3658	3658
q11	598	499	486	486
q12	758	605	600	600
q13	16483	2981	3032	2981
q14	301	258	267	258
q15	514	474	493	474
q16	476	408	408	408
q17	1759	1470	1460	1460
q18	7745	7639	7273	7273
q19	2313	1641	1574	1574
q20	1993	1773	1793	1773
q21	4914	4844	4844	4844
q22	591	525	534	525
Total cold run time: 71039 ms
Total hot run time: 53859 ms

doris-robot · 2024-05-29T13:33:52Z

TPC-DS: Total hot run time: 167949 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8798afd88e4e907aa6e6f4566013fa17ab9f0056, data reload: false
query1	918	377	376	376
query2	6460	2533	2346	2346
query3	6654	211	208	208
query4	19300	17529	17045	17045
query5	4151	417	420	417
query6	257	168	151	151
query7	4600	291	286	286
query8	307	283	291	283
query9	8459	2389	2374	2374
query10	447	283	270	270
query11	10444	10127	10091	10091
query12	139	87	88	87
query13	1645	356	351	351
query14	8370	6934	6210	6210
query15	235	190	181	181
query16	7790	261	258	258
query17	1355	543	517	517
query18	1894	276	269	269
query19	200	153	154	153
query20	93	91	89	89
query21	208	141	125	125
query22	4133	4092	4014	4014
query23	33675	32947	33109	32947
query24	12240	2812	2860	2812
query25	663	346	361	346
query26	1810	157	151	151
query27	3102	314	318	314
query28	7770	2041	2041	2041
query29	1170	603	594	594
query30	291	147	150	147
query31	960	747	739	739
query32	97	55	53	53
query33	761	262	263	262
query34	1011	470	462	462
query35	769	599	609	599
query36	1105	904	904	904
query37	276	68	64	64
query38	2871	2728	2775	2728
query39	866	781	787	781
query40	272	125	129	125
query41	54	50	75	50
query42	104	97	95	95
query43	601	564	553	553
query44	1195	720	739	720
query45	188	172	164	164
query46	1068	742	727	727
query47	1844	1770	1797	1770
query48	375	298	296	296
query49	1193	378	375	375
query50	770	381	377	377
query51	6870	6812	6761	6761
query52	104	89	90	89
query53	361	288	284	284
query54	1079	428	426	426
query55	77	71	75	71
query56	257	238	243	238
query57	1163	1042	1044	1042
query58	246	211	241	211
query59	3616	3100	3318	3100
query60	268	250	243	243
query61	89	84	89	84
query62	675	451	464	451
query63	312	287	294	287
query64	9868	2267	1678	1678
query65	3210	3112	3109	3109
query66	1419	334	333	333
query67	15126	14707	14769	14707
query68	4521	531	511	511
query69	455	261	266	261
query70	1126	1141	1076	1076
query71	399	271	259	259
query72	7429	2718	2561	2561
query73	713	322	321	321
query74	6041	5603	5688	5603
query75	3434	2595	2614	2595
query76	2722	1050	948	948
query77	402	263	266	263
query78	10283	9810	9672	9672
query79	2665	506	517	506
query80	1086	434	428	428
query81	507	218	223	218
query82	851	95	90	90
query83	242	187	169	169
query84	234	84	86	84
query85	1623	266	261	261
query86	530	308	324	308
query87	3303	3114	3092	3092
query88	4072	2349	2351	2349
query89	476	386	396	386
query90	1925	194	187	187
query91	135	170	95	95
query92	60	48	47	47
query93	1849	505	487	487
query94	1219	182	184	182
query95	405	306	315	306
query96	583	262	262	262
query97	3196	2989	3038	2989
query98	273	220	204	204
query99	1091	828	818	818
Total cold run time: 273750 ms
Total hot run time: 167949 ms

doris-robot · 2024-05-29T13:39:22Z

ClickBench: Total hot run time: 30.2 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8798afd88e4e907aa6e6f4566013fa17ab9f0056, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.09	0.10
query5	0.52	0.49	0.49
query6	1.14	0.72	0.73
query7	0.01	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.50	0.49
query10	0.55	0.56	0.54
query11	0.16	0.12	0.11
query12	0.14	0.11	0.11
query13	0.60	0.59	0.59
query14	0.79	0.77	0.79
query15	0.84	0.81	0.81
query16	0.37	0.37	0.38
query17	0.97	0.97	0.95
query18	0.20	0.22	0.27
query19	1.75	1.72	1.78
query20	0.01	0.02	0.02
query21	15.53	0.68	0.66
query22	4.26	7.23	1.59
query23	18.28	1.45	1.24
query24	1.77	0.25	0.22
query25	0.15	0.09	0.08
query26	0.26	0.16	0.16
query27	0.08	0.08	0.08
query28	13.35	1.03	1.01
query29	12.82	3.30	3.27
query30	0.24	0.06	0.06
query31	2.86	0.39	0.37
query32	3.30	0.46	0.48
query33	2.85	2.87	2.94
query34	17.20	4.38	4.41
query35	4.50	4.56	4.50
query36	0.65	0.48	0.46
query37	0.18	0.16	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.15	0.15
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.51 s
Total hot run time: 30.2 s

BePPPower · 2024-05-30T03:20:03Z

run p0 external

BePPPower · 2024-05-30T07:21:09Z

run buildall

BePPPower · 2024-05-31T04:05:23Z

run buildall

doris-robot · 2024-05-31T04:31:46Z

TPC-H: Total hot run time: 41406 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 970690f1a5f74cf68a7aa557ab8c29a3ebce7291, data reload: false
------ Round 1 ----------------------------------
q1	17673	4339	4302	4302
q2	2024	190	195	190
q3	10533	1191	1237	1191
q4	10230	872	844	844
q5	7483	2751	2741	2741
q6	229	133	135	133
q7	962	616	630	616
q8	9242	2145	2095	2095
q9	9416	6759	6762	6759
q10	9207	3874	3831	3831
q11	459	244	239	239
q12	434	228	228	228
q13	17424	3188	3285	3188
q14	266	207	216	207
q15	494	472	468	468
q16	504	410	406	406
q17	988	749	708	708
q18	8470	7757	7641	7641
q19	2885	1620	1626	1620
q20	652	319	311	311
q21	5250	4143	3361	3361
q22	400	335	327	327
Total cold run time: 115225 ms
Total hot run time: 41406 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4490	4432	4431	4431
q2	372	267	270	267
q3	3123	2921	3006	2921
q4	1886	1580	1567	1567
q5	5416	5545	5444	5444
q6	210	124	126	124
q7	2201	1815	1845	1815
q8	3263	3401	3400	3400
q9	8613	8751	8722	8722
q10	4043	3756	3794	3756
q11	588	507	517	507
q12	796	628	623	623
q13	16108	3117	3157	3117
q14	310	283	269	269
q15	533	501	484	484
q16	502	432	449	432
q17	1809	1543	1542	1542
q18	7681	7517	7422	7422
q19	2642	1598	1606	1598
q20	1980	1803	1782	1782
q21	9965	4768	4679	4679
q22	585	523	527	523
Total cold run time: 77116 ms
Total hot run time: 55425 ms

doris-robot · 2024-05-31T04:42:34Z

TPC-DS: Total hot run time: 168882 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 970690f1a5f74cf68a7aa557ab8c29a3ebce7291, data reload: false
query1	920	383	371	371
query2	6434	2475	2449	2449
query3	6649	206	206	206
query4	20547	17127	17226	17127
query5	4119	422	418	418
query6	247	154	149	149
query7	4586	300	287	287
query8	332	286	287	286
query9	8574	2370	2351	2351
query10	450	293	263	263
query11	10429	10095	9989	9989
query12	131	94	89	89
query13	1650	357	357	357
query14	9920	6831	6899	6831
query15	242	188	189	188
query16	7681	256	261	256
query17	1468	537	512	512
query18	1709	273	264	264
query19	222	150	152	150
query20	93	86	88	86
query21	202	130	128	128
query22	4184	3890	3894	3890
query23	33627	32884	33029	32884
query24	11954	2973	2823	2823
query25	649	359	357	357
query26	1660	154	153	153
query27	2859	322	323	322
query28	7532	2056	2042	2042
query29	1042	610	594	594
query30	221	151	154	151
query31	955	742	740	740
query32	93	58	53	53
query33	757	314	272	272
query34	949	475	475	475
query35	718	618	588	588
query36	1028	923	891	891
query37	260	67	71	67
query38	2894	2782	2727	2727
query39	850	791	835	791
query40	250	124	121	121
query41	55	53	49	49
query42	102	92	97	92
query43	599	548	538	538
query44	1201	735	741	735
query45	189	176	164	164
query46	1079	704	715	704
query47	1855	1751	1772	1751
query48	375	298	295	295
query49	946	381	390	381
query50	774	404	384	384
query51	6680	6693	6761	6693
query52	102	90	93	90
query53	367	283	285	283
query54	916	439	415	415
query55	71	71	72	71
query56	253	236	240	236
query57	1137	1051	1034	1034
query58	242	207	209	207
query59	3521	3234	3301	3234
query60	276	259	263	259
query61	95	86	94	86
query62	619	453	477	453
query63	315	287	287	287
query64	9668	2206	1713	1713
query65	3198	3138	3150	3138
query66	1289	331	325	325
query67	15091	14656	14838	14656
query68	4489	535	566	535
query69	441	270	269	269
query70	1201	1108	1109	1108
query71	403	273	269	269
query72	7137	2710	2590	2590
query73	707	319	320	319
query74	6096	5559	5880	5559
query75	3343	2620	2612	2612
query76	2421	1060	964	964
query77	406	264	258	258
query78	10148	9751	9752	9751
query79	2455	509	515	509
query80	1035	438	439	438
query81	512	223	222	222
query82	1132	97	91	91
query83	240	168	182	168
query84	248	89	86	86
query85	1368	326	346	326
query86	468	319	318	318
query87	3300	3127	3129	3127
query88	4315	2355	2346	2346
query89	481	399	380	380
query90	1998	183	184	183
query91	125	95	97	95
query92	62	48	48	48
query93	1636	506	493	493
query94	1271	188	182	182
query95	400	307	314	307
query96	589	265	267	265
query97	3182	3020	3023	3020
query98	248	226	224	224
query99	1097	849	850	849
Total cold run time: 273056 ms
Total hot run time: 168882 ms

doris-robot · 2024-05-31T04:48:05Z

ClickBench: Total hot run time: 30.81 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 970690f1a5f74cf68a7aa557ab8c29a3ebce7291, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.49	0.49
query6	1.13	0.72	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.53	0.48	0.48
query10	0.55	0.56	0.54
query11	0.16	0.11	0.12
query12	0.15	0.11	0.12
query13	0.60	0.59	0.60
query14	0.78	0.79	0.77
query15	0.83	0.81	0.81
query16	0.35	0.37	0.35
query17	0.98	0.96	0.93
query18	0.23	0.23	0.24
query19	1.72	1.70	1.66
query20	0.01	0.01	0.01
query21	15.74	0.68	0.66
query22	4.20	6.90	2.28
query23	18.31	1.43	1.28
query24	1.98	0.23	0.21
query25	0.15	0.10	0.07
query26	0.26	0.16	0.17
query27	0.07	0.08	0.07
query28	13.29	1.02	1.03
query29	13.06	3.35	3.28
query30	0.25	0.06	0.05
query31	2.88	0.38	0.38
query32	3.29	0.47	0.46
query33	2.85	2.94	2.89
query34	16.90	4.50	4.43
query35	4.55	4.48	4.55
query36	0.71	0.50	0.51
query37	0.18	0.16	0.15
query38	0.15	0.14	0.14
query39	0.04	0.03	0.04
query40	0.18	0.14	0.15
query41	0.09	0.05	0.04
query42	0.05	0.04	0.04
query43	0.04	0.04	0.03
Total cold run time: 109.83 s
Total hot run time: 30.81 s

morningman

LGTM

github-actions · 2024-05-31T06:21:34Z

PR approved by at least one committer and no changes requested.

doris-robot · 2024-06-25T09:37:59Z

ClickBench: Total hot run time: 30.7 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9434c92df638a7b67510715246b0f28a0a72cbf9, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.06	0.05
query4	1.65	0.07	0.08
query5	0.49	0.48	0.48
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.56	0.49	0.49
query10	0.54	0.54	0.55
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.75	0.77	0.81
query15	0.84	0.81	0.81
query16	0.37	0.37	0.37
query17	0.99	0.97	0.97
query18	0.22	0.24	0.26
query19	1.85	1.79	1.74
query20	0.01	0.01	0.01
query21	15.43	0.66	0.66
query22	3.85	7.76	1.90
query23	18.24	1.32	1.30
query24	2.15	0.22	0.23
query25	0.15	0.10	0.09
query26	0.26	0.17	0.18
query27	0.08	0.08	0.08
query28	13.18	1.03	1.01
query29	12.65	3.34	3.28
query30	0.25	0.06	0.05
query31	2.84	0.38	0.38
query32	3.29	0.48	0.48
query33	2.84	2.91	2.97
query34	17.12	4.46	4.41
query35	4.49	4.54	4.51
query36	0.66	0.47	0.48
query37	0.18	0.16	0.16
query38	0.15	0.14	0.13
query39	0.04	0.04	0.03
query40	0.17	0.13	0.15
query41	0.09	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 108.9 s
Total hot run time: 30.7 s

BePPPower · 2024-06-25T12:14:36Z

run feut

BePPPower · 2024-06-26T02:22:11Z

run buildall

doris-robot · 2024-06-26T02:41:50Z

TPC-H: Total hot run time: 39907 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0fef775a4da7c7a7c625839f60f46e6f8aadbdf0, data reload: false
------ Round 1 ----------------------------------
q1	17938	4484	4326	4326
q2	2700	195	187	187
q3	11707	1160	1197	1160
q4	10373	800	793	793
q5	7616	2688	2669	2669
q6	219	139	138	138
q7	956	606	614	606
q8	9361	2064	2067	2064
q9	8924	6508	6470	6470
q10	8963	3720	3688	3688
q11	458	235	241	235
q12	393	230	218	218
q13	18875	2995	3004	2995
q14	281	218	219	218
q15	506	465	481	465
q16	524	384	367	367
q17	969	746	709	709
q18	8044	7567	7367	7367
q19	3688	1372	1510	1372
q20	650	313	310	310
q21	4970	3217	3229	3217
q22	391	339	333	333
Total cold run time: 118506 ms
Total hot run time: 39907 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4429	4217	4229	4217
q2	377	267	272	267
q3	2987	2785	2736	2736
q4	1896	1615	1642	1615
q5	5232	5247	5280	5247
q6	216	129	129	129
q7	2111	1769	1753	1753
q8	3192	3340	3298	3298
q9	8329	8358	8308	8308
q10	3882	3673	3604	3604
q11	572	484	481	481
q12	756	624	575	575
q13	17584	2997	2971	2971
q14	290	267	259	259
q15	512	484	480	480
q16	470	398	423	398
q17	1807	1472	1463	1463
q18	7752	7415	7288	7288
q19	1695	1569	1576	1569
q20	1993	1813	1772	1772
q21	4777	4759	4628	4628
q22	613	574	539	539
Total cold run time: 71472 ms
Total hot run time: 53597 ms

doris-robot · 2024-06-26T02:53:15Z

TPC-DS: Total hot run time: 173249 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0fef775a4da7c7a7c625839f60f46e6f8aadbdf0, data reload: false
query1	929	402	389	389
query2	6447	2392	2316	2316
query3	6629	200	201	200
query4	19430	17473	17235	17235
query5	4117	485	478	478
query6	246	161	173	161
query7	4589	297	296	296
query8	341	286	268	268
query9	8584	2472	2440	2440
query10	595	288	261	261
query11	10527	9996	10039	9996
query12	126	85	79	79
query13	1643	395	379	379
query14	9982	7174	7266	7174
query15	246	184	186	184
query16	7771	279	266	266
query17	1880	553	532	532
query18	1962	283	288	283
query19	204	162	168	162
query20	89	80	83	80
query21	210	126	129	126
query22	4363	4058	4004	4004
query23	33450	32813	33232	32813
query24	11914	2769	2862	2769
query25	684	383	387	383
query26	1816	160	156	156
query27	2938	317	326	317
query28	7476	2104	2090	2090
query29	1140	635	633	633
query30	289	152	149	149
query31	987	736	770	736
query32	89	58	59	58
query33	791	309	308	308
query34	904	474	480	474
query35	761	619	630	619
query36	1095	950	944	944
query37	178	83	74	74
query38	2863	2754	2783	2754
query39	886	788	829	788
query40	278	134	129	129
query41	55	49	47	47
query42	128	104	103	103
query43	607	534	552	534
query44	1208	748	721	721
query45	195	162	168	162
query46	1072	732	722	722
query47	1862	1795	1737	1737
query48	389	299	303	299
query49	1190	437	426	426
query50	768	392	401	392
query51	6955	6752	6813	6752
query52	104	102	95	95
query53	369	290	287	287
query54	1032	483	460	460
query55	77	74	80	74
query56	309	280	294	280
query57	1161	1038	1052	1038
query58	274	249	261	249
query59	3500	3048	3274	3048
query60	314	295	294	294
query61	111	112	111	111
query62	657	465	561	465
query63	328	295	296	295
query64	9817	2216	1781	1781
query65	3171	3240	3106	3106
query66	1371	336	340	336
query67	15605	14820	14918	14820
query68	8427	539	552	539
query69	677	503	429	429
query70	1195	1096	1093	1093
query71	526	307	274	274
query72	8872	5650	5746	5650
query73	1521	329	328	328
query74	5904	5522	5458	5458
query75	5356	2625	2675	2625
query76	5021	984	942	942
query77	770	309	310	309
query78	10649	9809	9762	9762
query79	8451	512	527	512
query80	983	475	507	475
query81	562	222	219	219
query82	768	111	106	106
query83	348	174	171	171
query84	276	85	89	85
query85	1395	282	274	274
query86	412	307	305	305
query87	3267	3073	3112	3073
query88	4784	2456	2447	2447
query89	547	393	372	372
query90	1932	223	192	192
query91	131	99	101	99
query92	62	54	50	50
query93	6125	503	504	503
query94	1176	191	196	191
query95	410	313	322	313
query96	617	277	278	277
query97	3195	3025	3035	3025
query98	214	201	191	191
query99	1155	838	823	823
Total cold run time: 297806 ms
Total hot run time: 173249 ms

doris-robot · 2024-06-26T02:58:45Z

ClickBench: Total hot run time: 30.54 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0fef775a4da7c7a7c625839f60f46e6f8aadbdf0, data reload: false
query1	0.04	0.03	0.04
query2	0.08	0.05	0.05
query3	0.22	0.05	0.05
query4	1.67	0.08	0.08
query5	0.49	0.47	0.49
query6	1.12	0.72	0.72
query7	0.02	0.02	0.01
query8	0.06	0.04	0.04
query9	0.55	0.49	0.48
query10	0.54	0.55	0.54
query11	0.15	0.11	0.12
query12	0.14	0.12	0.13
query13	0.59	0.59	0.59
query14	0.76	0.77	0.80
query15	0.82	0.81	0.80
query16	0.37	0.38	0.37
query17	1.03	1.03	1.00
query18	0.22	0.25	0.23
query19	1.90	1.76	1.80
query20	0.02	0.01	0.01
query21	15.49	0.64	0.64
query22	4.33	7.19	1.80
query23	18.27	1.37	1.21
query24	2.10	0.25	0.23
query25	0.14	0.10	0.09
query26	0.28	0.18	0.17
query27	0.08	0.08	0.09
query28	13.17	1.02	1.00
query29	12.58	3.39	3.34
query30	0.26	0.06	0.06
query31	2.86	0.39	0.39
query32	3.28	0.47	0.47
query33	2.89	2.93	2.90
query34	17.27	4.44	4.39
query35	4.55	4.48	4.51
query36	0.65	0.48	0.46
query37	0.19	0.15	0.15
query38	0.16	0.14	0.14
query39	0.05	0.04	0.04
query40	0.16	0.14	0.14
query41	0.09	0.05	0.06
query42	0.05	0.04	0.05
query43	0.05	0.04	0.05
Total cold run time: 109.74 s
Total hot run time: 30.54 s

morningman

LGTM

BePPPower · 2024-06-26T11:58:12Z

run p0

BePPPower · 2024-06-26T14:25:59Z

run p0

BePPPower · 2024-06-27T02:23:39Z

run cloud_p1

…TRING type of external tables/TVF (#35489) Partition columns in CTAS need to be compatible with the STRING type of external tables/TVF 1. For TVF queries with partition columns in the file path, map these partition columns to the `VARCHAR(65533)` type. (Previously, they were of `STRING` type.) 3. If you use the `STRING` type of an external table as a partition column, CTAS will automatically convert it to `VARCHAR(65535)` type."

…TRING type of external tables/TVF (apache#35489) Partition columns in CTAS need to be compatible with the STRING type of external tables/TVF 1. For TVF queries with partition columns in the file path, map these partition columns to the `VARCHAR(65533)` type. (Previously, they were of `STRING` type.) 3. If you use the `STRING` type of an external table as a partition column, CTAS will automatically convert it to `VARCHAR(65535)` type."

…TRING type of external tables/TVF (#37161) bp: #35489

) Fix 2 bugs: 1. introduced from #35489 When the target table is unpartitioned with text type column from external table, NPE will be thrown. Only happen when using old planner. The new planner works well. 2. If first column is varchar type, the new planner will change it to string type and then failed because first column can not be string type.

…che#39744) Fix 2 bugs: 1. introduced from apache#35489 When the target table is unpartitioned with text type column from external table, NPE will be thrown. Only happen when using old planner. The new planner works well. 2. If first column is varchar type, the new planner will change it to string type and then failed because first column can not be string type.

) Fix 2 bugs: 1. introduced from #35489 When the target table is unpartitioned with text type column from external table, NPE will be thrown. Only happen when using old planner. The new planner works well. 2. If first column is varchar type, the new planner will change it to string type and then failed because first column can not be string type.

…TRING type of external tables/TVF (apache#35489) Partition columns in CTAS need to be compatible with the STRING type of external tables/TVF 1. For TVF queries with partition columns in the file path, map these partition columns to the `VARCHAR(65533)` type. (Previously, they were of `STRING` type.) 3. If you use the `STRING` type of an external table as a partition column, CTAS will automatically convert it to `VARCHAR(65535)` type."

…che#39744) Fix 2 bugs: 1. introduced from apache#35489 When the target table is unpartitioned with text type column from external table, NPE will be thrown. Only happen when using old planner. The new planner works well. 2. If first column is varchar type, the new planner will change it to string type and then failed because first column can not be string type.

morningman reviewed May 30, 2024
View reviewed changes

Comment threadfe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java Outdated

morningman added kind/behavior-changed dev/2.1.x labels May 31, 2024

morningman previously approved these changes May 31, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label May 31, 2024

morningman previously approved these changes Jun 25, 2024
View reviewed changes

wuwenchi reviewed Jun 26, 2024
View reviewed changes

Comment threadfe/fe-core/src/main/java/org/apache/doris/tablefunction/ExternalFileTableValuedFunction.java Outdated

fix 11
0fef775

BePPPower dismissed morningman’s stale review via 0fef775June 26, 2024 02:21

morningman approved these changes Jun 26, 2024
View reviewed changes

wuwenchi approved these changes Jun 26, 2024
View reviewed changes

morningman merged commit 23b0b7b into apache:masterJun 27, 2024

BePPPower mentioned this pull request Jul 2, 2024
[fix](tvf) Partition columns in CTAS need to be compatible with the STRING type of external tables/TVF #37161
Merged

morningman pushed a commit that referenced this pull request Jul 3, 2024
[fix](tvf) Partition columns in CTAS need to be compatible with the S…
4761090
…TRING type of external tables/TVF (#37161) bp: #35489

morningman added dev/2.1.5-merged and removed dev/2.1.x labels Jul 3, 2024

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

morningman mentioned this pull request Aug 21, 2024
[fix](ctas) fix NPE when ctas with old planner and varchar issue #39744
Merged

morningman removed the kind/need-document label Feb 11, 2026

Uh oh!

Conversation

BePPPower commented May 28, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

doris-robot commented May 28, 2024

Uh oh!

BePPPower commented May 28, 2024

Uh oh!

BePPPower commented May 28, 2024

Uh oh!

doris-robot commented May 28, 2024

Uh oh!

doris-robot commented May 28, 2024

Uh oh!

doris-robot commented May 28, 2024

Uh oh!

BePPPower commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

BePPPower commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

BePPPower commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

doris-robot commented May 29, 2024

Uh oh!

BePPPower commented May 30, 2024

Uh oh!

Uh oh!

BePPPower commented May 30, 2024

Uh oh!

BePPPower commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

doris-robot commented May 31, 2024

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented May 31, 2024

Uh oh!

doris-robot commented Jun 25, 2024

Uh oh!

BePPPower commented Jun 25, 2024

Uh oh!

Uh oh!

BePPPower commented Jun 26, 2024

Uh oh!

doris-robot commented Jun 26, 2024

Uh oh!

doris-robot commented Jun 26, 2024

Uh oh!

doris-robot commented Jun 26, 2024

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

BePPPower commented Jun 26, 2024

Uh oh!

BePPPower commented Jun 26, 2024

BePPPower commented May 28, 2024•
edited
Loading