[fix](nereids) fix generate column need add cast in stream load by feiniaofeiafei · Pull Request #49167 · apache/doris

feiniaofeiafei · 2025-03-17T13:22:43Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #35284

Problem Summary:
Generate column need add cast(casting slot to its own type) in stream load, because when loading data(stream load and other load), the slots reading from files are string type. So we need to cast it to its own type to avoid error.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-03-17T13:22:48Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

feiniaofeiafei · 2025-03-18T02:05:50Z

run buildall

doris-robot · 2025-03-18T02:38:36Z

TPC-H: Total hot run time: 32124 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a6c62b639c5548d627ec7dba2411ff70c80a9386, data reload: false
------ Round 1 ----------------------------------
q1	24677	5136	5046	5046
q2	2048	319	173	173
q3	10371	1302	710	710
q4	10225	1004	559	559
q5	8047	2454	2327	2327
q6	197	164	134	134
q7	931	765	607	607
q8	9324	1378	1065	1065
q9	4919	4641	4557	4557
q10	6826	2324	1884	1884
q11	496	277	251	251
q12	342	349	221	221
q13	17767	3738	3106	3106
q14	235	234	207	207
q15	529	474	478	474
q16	645	628	581	581
q17	574	865	373	373
q18	6860	6434	6171	6171
q19	1214	951	563	563
q20	329	330	198	198
q21	2852	2138	1955	1955
q22	1020	1031	962	962
Total cold run time: 110428 ms
Total hot run time: 32124 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5148	5111	5129	5111
q2	241	341	233	233
q3	2148	2688	2305	2305
q4	1408	1797	1399	1399
q5	4475	4141	4558	4141
q6	215	167	133	133
q7	2008	1978	1755	1755
q8	2652	2632	2675	2632
q9	7143	7183	7138	7138
q10	2992	3219	2751	2751
q11	567	522	492	492
q12	700	752	607	607
q13	3542	3839	3275	3275
q14	289	306	273	273
q15	526	482	486	482
q16	651	694	632	632
q17	1130	1616	1363	1363
q18	7755	7568	7494	7494
q19	816	841	960	841
q20	2006	2037	1861	1861
q21	5377	5082	4599	4599
q22	1072	1023	1010	1010
Total cold run time: 52861 ms
Total hot run time: 50527 ms

doris-robot · 2025-03-18T02:50:07Z

TPC-DS: Total hot run time: 185040 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a6c62b639c5548d627ec7dba2411ff70c80a9386, data reload: false
query1	1026	486	456	456
query2	6557	1976	1937	1937
query3	6790	220	216	216
query4	26528	23735	23209	23209
query5	4341	665	493	493
query6	310	208	193	193
query7	4625	493	287	287
query8	296	275	242	242
query9	8622	2517	2516	2516
query10	453	310	253	253
query11	15530	15153	14886	14886
query12	162	112	105	105
query13	1652	513	390	390
query14	9146	6953	6113	6113
query15	208	186	172	172
query16	7792	649	476	476
query17	1202	713	562	562
query18	2026	391	293	293
query19	187	171	145	145
query20	119	117	119	117
query21	208	128	104	104
query22	4038	4235	4205	4205
query23	33942	32887	33020	32887
query24	7717	2375	2393	2375
query25	520	462	407	407
query26	1220	272	154	154
query27	2121	492	328	328
query28	3869	2393	2380	2380
query29	708	550	419	419
query30	293	219	184	184
query31	940	871	791	791
query32	71	63	60	60
query33	553	370	308	308
query34	779	838	497	497
query35	780	818	727	727
query36	941	976	902	902
query37	120	129	76	76
query38	4136	4323	4074	4074
query39	1477	1396	1383	1383
query40	225	117	101	101
query41	53	52	49	49
query42	128	102	98	98
query43	507	511	480	480
query44	1267	800	768	768
query45	175	173	166	166
query46	823	1010	609	609
query47	1757	1792	1709	1709
query48	379	406	298	298
query49	785	522	392	392
query50	695	721	397	397
query51	4203	4163	4145	4145
query52	106	101	94	94
query53	238	258	182	182
query54	499	484	402	402
query55	78	81	82	81
query56	280	261	243	243
query57	1126	1127	1054	1054
query58	244	237	230	230
query59	2680	2881	2579	2579
query60	279	306	261	261
query61	122	122	119	119
query62	791	730	673	673
query63	233	190	186	186
query64	4279	995	669	669
query65	4494	4308	4340	4308
query66	1071	420	300	300
query67	15700	15373	15340	15340
query68	8088	866	487	487
query69	461	296	267	267
query70	1109	1123	1122	1122
query71	457	291	284	284
query72	5742	3565	3703	3565
query73	756	739	351	351
query74	9208	8960	8966	8960
query75	3687	3185	2700	2700
query76	3494	1171	741	741
query77	837	368	280	280
query78	9828	10145	9345	9345
query79	2438	818	596	596
query80	648	516	441	441
query81	500	261	224	224
query82	214	134	95	95
query83	170	171	149	149
query84	246	107	78	78
query85	834	388	435	388
query86	379	316	283	283
query87	4380	4387	4331	4331
query88	4041	2214	2212	2212
query89	406	312	278	278
query90	1874	217	220	217
query91	139	136	118	118
query92	85	61	58	58
query93	2121	1063	571	571
query94	640	419	310	310
query95	351	270	334	270
query96	488	552	279	279
query97	3259	3380	3327	3327
query98	237	204	200	200
query99	1331	1384	1258	1258
Total cold run time: 273670 ms
Total hot run time: 185040 ms

doris-robot · 2025-03-18T02:55:26Z

ClickBench: Total hot run time: 31.74 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a6c62b639c5548d627ec7dba2411ff70c80a9386, data reload: false
query1	0.04	0.04	0.04
query2	0.13	0.10	0.10
query3	0.25	0.19	0.19
query4	1.59	0.19	0.19
query5	0.60	0.60	0.59
query6	1.21	0.72	0.72
query7	0.02	0.02	0.02
query8	0.03	0.04	0.03
query9	0.59	0.52	0.53
query10	0.57	0.61	0.57
query11	0.16	0.10	0.11
query12	0.16	0.11	0.12
query13	0.62	0.61	0.61
query14	2.69	2.83	2.67
query15	0.92	0.86	0.85
query16	0.38	0.36	0.36
query17	1.02	1.02	1.06
query18	0.22	0.19	0.20
query19	1.93	2.04	1.88
query20	0.02	0.01	0.02
query21	15.38	0.91	0.56
query22	0.76	1.27	0.76
query23	14.74	1.39	0.64
query24	6.70	1.14	1.32
query25	0.51	0.30	0.10
query26	0.47	0.16	0.14
query27	0.05	0.05	0.05
query28	9.96	0.88	0.44
query29	12.54	4.05	3.33
query30	0.25	0.10	0.06
query31	2.82	0.60	0.39
query32	3.23	0.55	0.46
query33	3.04	3.16	3.05
query34	15.67	5.11	4.52
query35	4.52	4.52	4.54
query36	0.67	0.49	0.48
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.04	0.03	0.03
query40	0.17	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.96 s
Total hot run time: 31.74 s

morrySnow · 2025-03-20T09:11:08Z

+ Expression expressionForLoad = expression.rewriteDownShortCircuit(e -> {
+ if (e instanceof SlotReference && !(e.getDataType() instanceof CharacterType)) {
+ return new Cast(e, e.getDataType());
+ }
+ return e;
+ });


add comment to explain why need cast e to itself's type

feiniaofeiafei · 2025-03-20T10:02:13Z

run buildall

doris-robot · 2025-03-20T10:25:21Z

TPC-H: Total hot run time: 34082 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 09e59c1147a96d370f9fa41e5c9404c2f6b3bc12, data reload: false
------ Round 1 ----------------------------------
q1	24022	5209	5023	5023
q2	2049	293	171	171
q3	10500	1232	663	663
q4	10265	1010	513	513
q5	8852	2297	2405	2297
q6	266	166	134	134
q7	905	763	615	615
q8	9332	1300	1145	1145
q9	7474	5145	5107	5107
q10	6814	2301	1905	1905
q11	480	279	253	253
q12	335	349	218	218
q13	17786	3663	3118	3118
q14	234	226	206	206
q15	540	483	484	483
q16	625	615	604	604
q17	567	850	345	345
q18	7614	7388	7078	7078
q19	1383	942	558	558
q20	305	321	192	192
q21	3969	2669	2486	2486
q22	1068	1023	968	968
Total cold run time: 115385 ms
Total hot run time: 34082 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5230	5147	5134	5134
q2	226	319	226	226
q3	2155	2611	2297	2297
q4	1383	1817	1414	1414
q5	4555	4456	4474	4456
q6	209	167	127	127
q7	1988	1918	1748	1748
q8	2577	2563	2553	2553
q9	7096	7155	7073	7073
q10	2989	3183	2753	2753
q11	587	500	489	489
q12	708	745	609	609
q13	3521	3994	3226	3226
q14	281	296	275	275
q15	521	485	483	483
q16	652	698	635	635
q17	1104	1621	1335	1335
q18	7870	7547	7454	7454
q19	792	794	780	780
q20	1990	2031	1877	1877
q21	5220	4847	4653	4653
q22	1059	1042	999	999
Total cold run time: 52713 ms
Total hot run time: 50596 ms

doris-robot · 2025-03-20T10:36:56Z

TPC-DS: Total hot run time: 186288 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 09e59c1147a96d370f9fa41e5c9404c2f6b3bc12, data reload: false
query1	1003	488	465	465
query2	6586	1985	1897	1897
query3	6799	224	228	224
query4	26680	23698	22895	22895
query5	4352	690	530	530
query6	299	203	188	188
query7	4610	477	285	285
query8	288	252	223	223
query9	8614	2570	2553	2553
query10	436	301	250	250
query11	15716	15293	15236	15236
query12	178	106	106	106
query13	1657	501	385	385
query14	10516	6200	7030	6200
query15	209	195	171	171
query16	7258	627	491	491
query17	1200	736	567	567
query18	2003	412	299	299
query19	187	180	156	156
query20	117	120	120	120
query21	209	123	106	106
query22	4077	4354	4050	4050
query23	33777	32803	33097	32803
query24	7627	2365	2388	2365
query25	527	456	393	393
query26	1223	262	138	138
query27	2181	505	320	320
query28	3951	2419	2381	2381
query29	739	558	422	422
query30	280	212	186	186
query31	952	837	780	780
query32	74	62	72	62
query33	567	364	350	350
query34	785	844	502	502
query35	793	824	722	722
query36	950	996	870	870
query37	116	99	74	74
query38	4080	4128	3985	3985
query39	1442	1375	1390	1375
query40	211	117	102	102
query41	54	54	56	54
query42	118	104	100	100
query43	480	517	488	488
query44	1274	798	789	789
query45	178	174	169	169
query46	839	1010	615	615
query47	1760	1779	1685	1685
query48	371	408	295	295
query49	767	500	410	410
query50	680	719	411	411
query51	4202	4234	4121	4121
query52	105	102	100	100
query53	226	252	177	177
query54	485	488	414	414
query55	83	80	86	80
query56	294	265	272	265
query57	1145	1140	1096	1096
query58	253	239	236	236
query59	2599	2772	2568	2568
query60	277	276	254	254
query61	127	125	126	125
query62	791	717	645	645
query63	234	185	181	181
query64	4346	1092	758	758
query65	4494	4354	4380	4354
query66	1116	407	299	299
query67	15835	15431	15279	15279
query68	7795	861	502	502
query69	491	296	267	267
query70	1207	1064	1034	1034
query71	469	314	272	272
query72	5534	5123	5140	5123
query73	671	639	352	352
query74	8997	9154	8712	8712
query75	3393	3219	2720	2720
query76	3325	1184	739	739
query77	673	369	279	279
query78	9916	10181	9371	9371
query79	1709	810	556	556
query80	605	515	453	453
query81	510	257	227	227
query82	208	131	101	101
query83	186	190	168	168
query84	250	105	78	78
query85	808	419	385	385
query86	373	306	282	282
query87	4442	4667	4360	4360
query88	3271	2256	2243	2243
query89	381	321	282	282
query90	2003	205	208	205
query91	137	143	108	108
query92	76	60	56	56
query93	1708	1058	580	580
query94	663	414	312	312
query95	359	268	262	262
query96	481	552	277	277
query97	3265	3360	3271	3271
query98	229	201	210	201
query99	1320	1440	1315	1315
Total cold run time: 271857 ms
Total hot run time: 186288 ms

doris-robot · 2025-03-20T10:42:13Z

ClickBench: Total hot run time: 31.05 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 09e59c1147a96d370f9fa41e5c9404c2f6b3bc12, data reload: false
query1	0.04	0.03	0.03
query2	0.13	0.10	0.11
query3	0.24	0.19	0.20
query4	1.59	0.20	0.20
query5	0.59	0.58	0.57
query6	1.18	0.72	0.71
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.53	0.53
query10	0.58	0.59	0.57
query11	0.15	0.10	0.11
query12	0.14	0.11	0.12
query13	0.62	0.59	0.60
query14	2.69	2.70	2.79
query15	0.95	0.85	0.85
query16	0.38	0.38	0.37
query17	1.04	1.02	1.05
query18	0.21	0.20	0.19
query19	1.91	1.94	1.81
query20	0.01	0.01	0.02
query21	15.36	0.90	0.53
query22	0.76	1.22	0.67
query23	14.95	1.39	0.65
query24	6.64	1.78	0.62
query25	0.48	0.34	0.07
query26	0.66	0.16	0.14
query27	0.05	0.05	0.05
query28	8.92	0.88	0.45
query29	12.55	3.97	3.32
query30	0.25	0.10	0.06
query31	2.82	0.57	0.38
query32	3.24	0.54	0.48
query33	3.05	3.05	3.08
query34	15.81	5.14	4.57
query35	4.59	4.57	4.54
query36	0.67	0.48	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.4 s
Total hot run time: 31.05 s

morrySnow · 2025-04-01T10:10:10Z

run buildall

doris-robot · 2025-04-01T10:29:33Z

TPC-H: Total hot run time: 34077 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 35d36f1e3fff029c69e4350489744f61d7025091, data reload: false
------ Round 1 ----------------------------------
q1	25986	5039	5034	5034
q2	2058	281	176	176
q3	10412	1282	671	671
q4	10229	1028	542	542
q5	7524	2369	2323	2323
q6	191	161	131	131
q7	907	755	609	609
q8	9324	1262	1110	1110
q9	6843	5161	5095	5095
q10	6842	2328	1871	1871
q11	486	289	274	274
q12	355	350	219	219
q13	17758	3710	3092	3092
q14	225	240	215	215
q15	534	489	486	486
q16	641	610	597	597
q17	589	876	371	371
q18	7538	7237	7060	7060
q19	1228	952	558	558
q20	336	336	234	234
q21	4102	2706	2435	2435
q22	1030	1006	974	974
Total cold run time: 115138 ms
Total hot run time: 34077 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5137	5153	5125	5125
q2	237	322	230	230
q3	2146	2656	2308	2308
q4	1426	1812	1407	1407
q5	4465	4386	4416	4386
q6	226	170	133	133
q7	1988	1914	1812	1812
q8	2585	2492	2497	2492
q9	7196	7088	7101	7088
q10	2985	3125	2747	2747
q11	575	507	478	478
q12	700	802	623	623
q13	3494	3981	3288	3288
q14	289	287	289	287
q15	518	455	464	455
q16	645	694	673	673
q17	1170	1577	1360	1360
q18	7869	7280	7374	7280
q19	845	808	864	808
q20	1953	1978	1869	1869
q21	5305	4784	4818	4784
q22	1104	1033	1016	1016
Total cold run time: 52858 ms
Total hot run time: 50649 ms

doris-robot · 2025-04-01T10:41:29Z

TPC-DS: Total hot run time: 192277 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 35d36f1e3fff029c69e4350489744f61d7025091, data reload: false
query1	1315	1081	1076	1076
query2	6241	1990	1946	1946
query3	11002	4548	4494	4494
query4	25324	23677	22972	22972
query5	5507	652	473	473
query6	310	220	226	220
query7	4002	496	288	288
query8	307	261	239	239
query9	8523	2629	2610	2610
query10	553	317	276	276
query11	15571	15471	14716	14716
query12	160	115	106	106
query13	1763	553	399	399
query14	10832	6165	6259	6165
query15	233	193	166	166
query16	7637	678	477	477
query17	1469	758	590	590
query18	2050	409	307	307
query19	236	191	185	185
query20	132	121	126	121
query21	202	129	110	110
query22	4336	4497	4410	4410
query23	34193	33280	33159	33159
query24	8138	2389	2427	2389
query25	500	461	397	397
query26	1164	283	159	159
query27	2825	502	331	331
query28	4621	2474	2415	2415
query29	701	579	447	447
query30	277	227	190	190
query31	902	892	778	778
query32	77	68	63	63
query33	566	372	312	312
query34	803	892	512	512
query35	821	834	762	762
query36	983	1004	901	901
query37	120	106	74	74
query38	4207	4233	4141	4141
query39	1493	1467	1421	1421
query40	214	119	104	104
query41	54	54	51	51
query42	120	105	106	105
query43	511	484	488	484
query44	1328	831	827	827
query45	191	179	173	173
query46	850	1045	657	657
query47	1931	1947	1861	1861
query48	395	431	308	308
query49	777	544	453	453
query50	677	693	408	408
query51	4297	4353	4228	4228
query52	121	107	106	106
query53	242	254	192	192
query54	614	583	514	514
query55	79	83	83	83
query56	299	297	295	295
query57	1085	1150	1089	1089
query58	259	269	268	268
query59	2725	2875	2630	2630
query60	321	322	305	305
query61	133	130	167	130
query62	801	736	655	655
query63	234	192	189	189
query64	4162	1125	782	782
query65	4416	4248	4256	4248
query66	977	417	331	331
query67	15810	15435	15416	15416
query68	8399	901	534	534
query69	546	374	267	267
query70	1165	1104	1094	1094
query71	527	329	303	303
query72	5316	4687	4639	4639
query73	649	572	351	351
query74	8958	8875	8817	8817
query75	4085	3204	2688	2688
query76	3759	1291	761	761
query77	783	364	284	284
query78	10074	10092	9309	9309
query79	2837	817	571	571
query80	683	538	449	449
query81	491	259	222	222
query82	502	122	98	98
query83	301	247	234	234
query84	281	105	87	87
query85	816	350	312	312
query86	346	302	294	294
query87	4568	4596	4333	4333
query88	3822	2245	2230	2230
query89	406	332	291	291
query90	1975	216	217	216
query91	153	138	115	115
query92	80	61	61	61
query93	2401	960	582	582
query94	665	416	307	307
query95	375	292	285	285
query96	497	562	273	273
query97	3178	3208	3110	3110
query98	241	209	198	198
query99	1416	1372	1273	1273
Total cold run time: 284014 ms
Total hot run time: 192277 ms

doris-robot · 2025-04-01T10:46:46Z

ClickBench: Total hot run time: 30.58 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 35d36f1e3fff029c69e4350489744f61d7025091, data reload: false
query1	0.04	0.04	0.03
query2	0.12	0.10	0.10
query3	0.24	0.20	0.18
query4	1.60	0.19	0.20
query5	0.60	0.58	0.59
query6	1.19	0.71	0.72
query7	0.03	0.02	0.02
query8	0.04	0.04	0.04
query9	0.58	0.53	0.55
query10	0.58	0.60	0.56
query11	0.17	0.11	0.10
query12	0.14	0.12	0.11
query13	0.62	0.60	0.61
query14	2.84	2.67	2.70
query15	0.93	0.87	0.84
query16	0.39	0.38	0.38
query17	1.04	1.00	1.00
query18	0.22	0.20	0.20
query19	1.93	1.91	1.87
query20	0.02	0.02	0.01
query21	15.36	0.88	0.55
query22	0.77	1.07	0.64
query23	15.06	1.36	0.63
query24	8.04	0.66	0.27
query25	0.30	0.12	0.15
query26	0.58	0.17	0.13
query27	0.06	0.05	0.05
query28	8.84	0.93	0.45
query29	12.56	4.01	3.28
query30	0.25	0.10	0.07
query31	2.82	0.61	0.39
query32	3.23	0.55	0.48
query33	2.99	3.14	3.09
query34	15.54	5.07	4.45
query35	4.58	4.49	4.48
query36	0.67	0.50	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.03
query39	0.02	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 105.46 s
Total hot run time: 30.58 s

morrySnow · 2025-04-07T03:20:41Z

run buildall

doris-robot · 2025-04-07T03:58:13Z

TPC-H: Total hot run time: 33953 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9e445b8631f8ad8dcb9d51961d332cfd302225bd, data reload: false
------ Round 1 ----------------------------------
q1	25793	5290	4973	4973
q2	2061	293	175	175
q3	10397	1226	671	671
q4	10236	1025	527	527
q5	7541	2321	2364	2321
q6	183	161	134	134
q7	914	732	614	614
q8	9329	1297	1085	1085
q9	6844	5092	5077	5077
q10	6842	2321	1894	1894
q11	470	283	268	268
q12	343	353	215	215
q13	17790	3638	3071	3071
q14	228	235	214	214
q15	533	473	482	473
q16	642	613	573	573
q17	601	908	354	354
q18	7384	7092	7143	7092
q19	1599	947	556	556
q20	327	319	217	217
q21	3934	3446	2446	2446
q22	1046	1003	1007	1003
Total cold run time: 115037 ms
Total hot run time: 33953 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5201	5050	5054	5050
q2	243	325	231	231
q3	2120	2600	2309	2309
q4	1368	1776	1370	1370
q5	4455	4360	4360	4360
q6	213	166	126	126
q7	2003	1900	1783	1783
q8	2557	2553	2564	2553
q9	7208	7307	7008	7008
q10	2989	3190	2711	2711
q11	579	499	512	499
q12	681	780	637	637
q13	3522	3906	3267	3267
q14	296	321	278	278
q15	527	485	475	475
q16	636	691	664	664
q17	1162	1503	1421	1421
q18	7789	7541	7543	7541
q19	824	836	864	836
q20	1921	1969	1827	1827
q21	5259	4828	4847	4828
q22	1142	1082	1010	1010
Total cold run time: 52695 ms
Total hot run time: 50784 ms

doris-robot · 2025-04-07T04:10:08Z

TPC-DS: Total hot run time: 191908 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9e445b8631f8ad8dcb9d51961d332cfd302225bd, data reload: false
query1	1416	1051	1048	1048
query2	6045	1937	1957	1937
query3	11173	4662	4452	4452
query4	25415	23698	23117	23117
query5	5433	628	454	454
query6	312	221	194	194
query7	3981	487	282	282
query8	277	240	224	224
query9	8503	2571	2563	2563
query10	531	321	269	269
query11	15217	15314	14785	14785
query12	158	112	118	112
query13	1562	542	401	401
query14	9022	6048	6204	6048
query15	198	185	165	165
query16	7617	654	495	495
query17	1156	781	595	595
query18	2048	423	337	337
query19	205	197	169	169
query20	129	127	118	118
query21	213	129	115	115
query22	4585	4659	4344	4344
query23	33911	33188	33122	33122
query24	8440	2405	2385	2385
query25	501	468	407	407
query26	1168	272	155	155
query27	2883	527	334	334
query28	4459	2431	2403	2403
query29	692	555	455	455
query30	277	229	201	201
query31	924	863	780	780
query32	73	61	64	61
query33	538	369	348	348
query34	784	874	527	527
query35	800	845	770	770
query36	985	1023	910	910
query37	121	104	82	82
query38	4315	4258	4084	4084
query39	1531	1460	1454	1454
query40	215	119	109	109
query41	54	55	55	55
query42	119	105	108	105
query43	507	522	476	476
query44	1315	810	817	810
query45	181	179	170	170
query46	853	1044	652	652
query47	1879	1887	1761	1761
query48	398	426	310	310
query49	769	524	453	453
query50	653	705	400	400
query51	4242	4268	4182	4182
query52	107	109	98	98
query53	237	264	188	188
query54	598	607	518	518
query55	88	84	79	79
query56	320	296	307	296
query57	1177	1205	1110	1110
query58	267	275	267	267
query59	2949	2876	2750	2750
query60	350	324	314	314
query61	131	126	125	125
query62	751	742	692	692
query63	217	184	185	184
query64	3990	1017	753	753
query65	4349	4266	4327	4266
query66	902	410	315	315
query67	16435	15515	15387	15387
query68	9539	853	515	515
query69	509	333	258	258
query70	1202	1033	1096	1033
query71	466	318	279	279
query72	5169	4626	4699	4626
query73	732	598	357	357
query74	9049	8864	8758	8758
query75	4436	3185	2702	2702
query76	4288	1204	761	761
query77	1001	367	291	291
query78	9861	10084	9297	9297
query79	2719	802	566	566
query80	607	498	432	432
query81	462	265	218	218
query82	247	129	99	99
query83	270	252	238	238
query84	296	98	82	82
query85	779	432	299	299
query86	334	306	312	306
query87	4490	4447	4314	4314
query88	2797	2225	2232	2225
query89	428	318	276	276
query90	2037	211	218	211
query91	137	144	108	108
query92	74	58	61	58
query93	2079	951	577	577
query94	689	413	323	323
query95	375	288	283	283
query96	478	590	269	269
query97	3139	3387	3113	3113
query98	228	207	207	207
query99	1358	1413	1300	1300
Total cold run time: 281926 ms
Total hot run time: 191908 ms

doris-robot · 2025-04-07T04:15:28Z

ClickBench: Total hot run time: 31.42 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9e445b8631f8ad8dcb9d51961d332cfd302225bd, data reload: false
query1	0.03	0.03	0.03
query2	0.12	0.10	0.11
query3	0.24	0.20	0.19
query4	1.58	0.20	0.19
query5	0.59	0.56	0.58
query6	1.18	0.72	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.51
query10	0.57	0.61	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.60	0.60
query14	2.80	2.79	2.82
query15	0.93	0.84	0.83
query16	0.38	0.38	0.38
query17	1.06	1.04	1.03
query18	0.21	0.20	0.20
query19	1.96	1.96	1.83
query20	0.02	0.01	0.01
query21	15.38	0.92	0.57
query22	0.76	1.12	0.76
query23	14.89	1.38	0.64
query24	7.12	2.16	0.74
query25	0.50	0.27	0.12
query26	0.59	0.16	0.14
query27	0.06	0.05	0.05
query28	10.04	0.81	0.43
query29	12.55	3.99	3.34
query30	0.24	0.08	0.06
query31	2.83	0.56	0.39
query32	3.24	0.54	0.47
query33	3.02	3.10	3.07
query34	15.88	5.12	4.55
query35	4.52	4.55	4.49
query36	0.65	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.95 s
Total hot run time: 31.42 s

feiniaofeiafei · 2025-04-07T07:49:34Z

run feut

github-actions · 2025-04-14T08:33:20Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-04-14T08:33:24Z

PR approved by anyone and no changes requested.

### What problem does this PR solve? Related PR: #35284 Problem Summary: Generate column need add cast(casting slot to its own type) in stream load, because when loading data(stream load and other load), the slots reading from files are string type. So we need to cast it to its own type to avoid error.

…he#49167) ### What problem does this PR solve? Related PR: apache#35284 Problem Summary: Generate column need add cast(casting slot to its own type) in stream load, because when loading data(stream load and other load), the slots reading from files are string type. So we need to cast it to its own type to avoid error.

…m load #49167 (#50029) Cherry-picked from #49167 Co-authored-by: feiniaofeiafei <moailing@selectdb.com>

…he#49167) ### What problem does this PR solve? Related PR: apache#35284 Problem Summary: Generate column need add cast(casting slot to its own type) in stream load, because when loading data(stream load and other load), the slots reading from files are string type. So we need to cast it to its own type to avoid error.

…e#49167 (apache#3909)

feiniaofeiafei force-pushed the fix_generated_column branch from c1a6f62 to a6c62b6Compare March 17, 2025 13:23

dataroaring added dev/3.0.x usercase Important user case type label labels Mar 18, 2025

morrySnow reviewed Mar 20, 2025
View reviewed changes

feiniaofeiafei force-pushed the fix_generated_column branch 2 times, most recently from afcb04d to 09e59c1Compare March 20, 2025 10:02

morrySnow force-pushed the fix_generated_column branch from 09e59c1 to 35d36f1Compare March 31, 2025 03:26

feiniaofeiafei added 2 commits April 7, 2025 11:20

fix
c1ce275

add comment
9e445b8

morrySnow force-pushed the fix_generated_column branch from 35d36f1 to 9e445b8Compare April 7, 2025 03:20

starocean999 approved these changes Apr 14, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Apr 14, 2025

github-actionsBot added the reviewed label Apr 14, 2025

morrySnow approved these changes Apr 14, 2025
View reviewed changes

morrySnow merged commit 0afc483 into apache:masterApr 14, 2025

github-actionsBot mentioned this pull request Apr 14, 2025
branch-3.0: [fix](nereids) fix generate column need add cast in stream load #49167 #50029
Merged

dataroaring pushed a commit that referenced this pull request Apr 22, 2025
branch-3.0: [fix](nereids) fix generate column need add cast in strea…
ae32003
…m load #49167 (#50029) Cherry-picked from #49167 Co-authored-by: feiniaofeiafei <moailing@selectdb.com>

dataroaring added dev/3.0.6-merged and removed dev/3.0.x labels Apr 22, 2025

suxiaogang223 pushed a commit to suxiaogang223/doris that referenced this pull request Jul 15, 2025
[fix](nereids) fix generate column need add cast in stream loadapach…
496e838
…e#49167 (apache#3909)

Hastyshell pushed a commit to Hastyshell/doris that referenced this pull request Jul 21, 2025
[fix](nereids) fix generate column need add cast in stream loadapach…
d14badf
…e#49167 (apache#3909)

Uh oh!

Conversation

feiniaofeiafei commented Mar 17, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Mar 17, 2025

Uh oh!

feiniaofeiafei commented Mar 18, 2025

Uh oh!

doris-robot commented Mar 18, 2025

Uh oh!

doris-robot commented Mar 18, 2025

Uh oh!

doris-robot commented Mar 18, 2025

Uh oh!

morrySnowMar 20, 2025

Choose a reason for hiding this comment

Uh oh!

feiniaofeiafei commented Mar 20, 2025

Uh oh!

doris-robot commented Mar 20, 2025

Uh oh!

doris-robot commented Mar 20, 2025

Uh oh!

doris-robot commented Mar 20, 2025

Uh oh!

morrySnow commented Apr 1, 2025

Uh oh!

doris-robot commented Apr 1, 2025

Uh oh!

doris-robot commented Apr 1, 2025

Uh oh!

doris-robot commented Apr 1, 2025

Uh oh!

morrySnow commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

feiniaofeiafei commented Apr 7, 2025

Uh oh!

github-actionsBot commented Apr 14, 2025

Uh oh!

github-actionsBot commented Apr 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

feiniaofeiafei commented Mar 17, 2025•
edited
Loading