branch-3.1: [feat](nereids) Add session variable enable_insert_value_auto_cast for insert value truncate long string #55325 by github-actions[bot] · Pull Request #55414 · apache/doris

github-actions · 2025-08-28T07:14:05Z

Cherry-picked from #55325

…r insert value truncate long string (#55325) for below sql: ```sql create table t(a int, b varchar(2)); insert into t values(1, 'abcde'); ``` for doris 2.0, the insert command will throw error 'Insert has filtered data in strict mode', because length of value 'abcde' is bigger than varchar(2). for mysql, pg, the above sql will also throw error for the same reason. for doris version >= 2.1.0, it will not throw error, and just truncate 'abcde' to 'ab'. what's more, for stream load, no matter 2.0, or 2.1, ..., doris will stream load fail. but considering 2.1 have released for 1.5+ year, we don't want to make a behaviour change for 2.1, so add a session variable enable_insert_value_auto_cast, when insert value with longer string value, then will have: - if enable_insert_value_auto_cast = true (default), the longer string will be truncate and insert succ; - if enable_insert_value_auto_cast = false, enable_insert_strict = true(default), will throw exception 'Insert has filtered data in strict mode'; - if enable_insert_value_auto_cast = false, enable_insert_strict = false, then the longer string will be filtered, other rows will insert succ. relate PR: #52802

hello-stephen · 2025-08-28T07:14:12Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

hello-stephen · 2025-08-28T07:14:16Z

run buildall

doris-robot · 2025-08-28T07:37:04Z

TPC-H: Total hot run time: 32698 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c96589cc9fddf0209f02a73987f473eb9915dd8f, data reload: false
------ Round 1 ----------------------------------
q1	17600	5553	5391	5391
q2	2039	401	297	297
q3	11913	1271	762	762
q4	10541	882	472	472
q5	9670	2368	2145	2145
q6	184	163	130	130
q7	930	752	615	615
q8	9345	1445	1132	1132
q9	5204	4981	4901	4901
q10	6770	2300	1836	1836
q11	481	282	264	264
q12	336	358	208	208
q13	17795	3572	3017	3017
q14	232	218	205	205
q15	521	466	456	456
q16	410	432	370	370
q17	589	868	377	377
q18	6894	6504	6357	6357
q19	1195	962	563	563
q20	333	349	219	219
q21	3028	2191	1997	1997
q22	1068	1035	984	984
Total cold run time: 107078 ms
Total hot run time: 32698 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5517	5535	5485	5485
q2	245	335	238	238
q3	2262	2682	2335	2335
q4	1362	1831	1385	1385
q5	4453	5059	4975	4975
q6	162	164	128	128
q7	2118	1978	1816	1816
q8	2661	2850	2714	2714
q9	7272	7204	7212	7204
q10	3086	3261	2726	2726
q11	580	516	493	493
q12	667	762	645	645
q13	3446	3782	3120	3120
q14	299	322	288	288
q15	510	487	463	463
q16	446	480	443	443
q17	1241	1727	1276	1276
q18	7732	7462	7249	7249
q19	787	1040	1152	1040
q20	2016	2055	1899	1899
q21	5480	5037	4606	4606
q22	1087	1093	1058	1058
Total cold run time: 53429 ms
Total hot run time: 51586 ms

doris-robot · 2025-08-28T07:49:13Z

TPC-DS: Total hot run time: 191979 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c96589cc9fddf0209f02a73987f473eb9915dd8f, data reload: false
query1	969	419	424	419
query2	6187	1939	1870	1870
query3	8719	197	198	197
query4	33608	23601	23335	23335
query5	3582	612	457	457
query6	280	206	181	181
query7	4225	490	311	311
query8	305	245	230	230
query9	9488	2605	2585	2585
query10	475	336	261	261
query11	18424	15386	15218	15218
query12	163	112	105	105
query13	1567	549	412	412
query14	9194	7282	7139	7139
query15	224	198	180	180
query16	8074	650	468	468
query17	1581	779	590	590
query18	2148	436	332	332
query19	241	189	164	164
query20	133	123	124	123
query21	205	137	112	112
query22	4538	4633	4393	4393
query23	35442	34317	33971	33971
query24	7760	2680	2756	2680
query25	529	512	433	433
query26	1133	293	180	180
query27	2031	493	379	379
query28	5116	2249	2206	2206
query29	763	596	465	465
query30	244	193	167	167
query31	972	908	825	825
query32	90	63	60	60
query33	529	379	332	332
query34	725	863	574	574
query35	782	833	751	751
query36	1016	1043	949	949
query37	99	92	66	66
query38	4006	4030	3976	3976
query39	1560	1500	1488	1488
query40	212	122	104	104
query41	51	51	51	51
query42	130	117	109	109
query43	525	520	496	496
query44	1340	829	838	829
query45	183	176	174	174
query46	899	1064	686	686
query47	1966	2001	1942	1942
query48	426	438	358	358
query49	759	519	409	409
query50	677	682	433	433
query51	7361	7270	7219	7219
query52	101	105	97	97
query53	239	276	189	189
query54	550	602	463	463
query55	78	83	80	80
query56	286	305	268	268
query57	1284	1297	1222	1222
query58	244	221	235	221
query59	3068	3162	3118	3118
query60	310	293	281	281
query61	114	120	121	120
query62	791	762	695	695
query63	232	195	200	195
query64	4333	995	658	658
query65	3402	3290	3324	3290
query66	952	418	331	331
query67	16469	15819	15573	15573
query68	7470	814	532	532
query69	498	303	265	265
query70	1193	1135	1134	1134
query71	376	292	267	267
query72	5812	3639	3759	3639
query73	627	738	352	352
query74	10316	9376	8879	8879
query75	3180	3181	2665	2665
query76	2981	1171	796	796
query77	515	366	283	283
query78	10389	10531	9558	9558
query79	3107	874	594	594
query80	747	515	425	425
query81	514	251	216	216
query82	965	115	89	89
query83	160	158	147	147
query84	247	97	81	81
query85	783	379	299	299
query86	395	307	293	293
query87	4334	4265	4210	4210
query88	4880	2382	2374	2374
query89	403	335	291	291
query90	1790	189	202	189
query91	134	144	113	113
query92	68	55	52	52
query93	2756	892	540	540
query94	688	423	300	300
query95	346	281	273	273
query96	478	599	286	286
query97	3184	3368	3160	3160
query98	227	206	204	204
query99	1372	1401	1278	1278
Total cold run time: 295044 ms
Total hot run time: 191979 ms

doris-robot · 2025-08-28T07:54:41Z

ClickBench: Total hot run time: 29.34 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c96589cc9fddf0209f02a73987f473eb9915dd8f, data reload: false
query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.07	0.07
query4	1.60	0.11	0.11
query5	0.50	0.51	0.52
query6	1.15	0.73	0.74
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.57	0.49	0.50
query10	0.55	0.56	0.55
query11	0.14	0.10	0.10
query12	0.14	0.12	0.11
query13	0.62	0.60	0.60
query14	0.80	0.80	0.81
query15	0.85	0.83	0.83
query16	0.38	0.39	0.38
query17	1.08	1.02	1.00
query18	0.25	0.23	0.23
query19	1.96	1.79	1.86
query20	0.02	0.01	0.02
query21	15.37	0.95	0.59
query22	0.74	0.84	0.72
query23	15.00	1.45	0.54
query24	3.33	1.24	1.51
query25	0.20	0.22	0.13
query26	0.18	0.14	0.14
query27	0.05	0.05	0.05
query28	14.08	1.01	0.43
query29	12.59	3.97	3.24
query30	0.25	0.09	0.08
query31	2.81	0.61	0.39
query32	3.22	0.53	0.45
query33	3.02	3.06	3.04
query34	16.71	5.15	4.52
query35	4.58	4.57	4.57
query36	0.65	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.3 s
Total hot run time: 29.34 s

hello-stephen · 2025-08-28T08:30:12Z

FE UT Coverage Report

Increment line coverage 33.33% (2/6) 🎉
Increment coverage report
Complete coverage report

yujun777 · 2025-08-28T11:56:38Z

run cloud_p0

github-actionsBot requested a review from morrySnow as a code owner August 28, 2025 07:14

dataroaring closed this Aug 28, 2025

dataroaring reopened this Aug 28, 2025

morrySnow approved these changes Sep 4, 2025
View reviewed changes

morrySnow merged commit a909b83 into branch-3.1Sep 4, 2025
23 checks passed

morrySnow deleted the auto-pick-55325-branch-3.1 branch September 4, 2025 01:54

Navigation Menu

Uh oh!

branch-3.1: [feat](nereids) Add session variable enable_insert_value_auto_cast for insert value truncate long string #55325 - #55414

branch-3.1: [feat](nereids) Add session variable enable_insert_value_auto_cast for insert value truncate long string #55325#55414
morrySnow merged 1 commit into
branch-3.1from
auto-pick-55325-branch-3.1

github-actionsBot commented Aug 28, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

Uh oh!

yujun777 commented Aug 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

github-actionsBot commented Aug 28, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

FE UT Coverage Report

Uh oh!

yujun777 commented Aug 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants