[fix](be) Normalize v1 date string cast result by jacktengg · Pull Request #64575 · apache/doris

jacktengg · 2026-06-16T15:58:08Z

Problem Summary: Strict-mode string casts to v1 DATE can return early after parsing a compact datetime string or an hour-only time part. Those early returns skipped the common target-type normalization, so DATE results could keep non-zero time fields such as 2024-05-01 01:00:00 instead of 2024-05-01. Normalize the parsed value before both early returns so DATE clears the time fields and DATETIME keeps datetime semantics.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Problem Summary: Strict-mode string casts to v1 DATE can return early after parsing a compact datetime string or an hour-only time part. Those early returns skipped the common target-type normalization, so DATE results could keep non-zero time fields such as 2024-05-01 01:00:00 instead of 2024-05-01. Normalize the parsed value before both early returns so DATE clears the time fields and DATETIME keeps datetime semantics.

hello-stephen · 2026-06-16T15:58:13Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

jacktengg · 2026-06-16T15:58:44Z

/review

jacktengg · 2026-06-16T15:59:47Z

run buildall

github-actions

Review result: requesting changes for a test-coverage gap.

Critical checkpoints:

Goal: The PR intends to normalize v1 DATE string-cast results for compact datetime and hour-only strict-parser exits. The implementation now normalizes both changed exits, and non-strict callers also benefit because they delegate to this templated parser first.
Scope: The implementation change is small and focused.
Concurrency/lifecycle/config/compatibility/transactions/persistence/FE-BE protocol: Not involved in this PR.
Parallel paths: Checked the v1 DATE/DATETIME serde callers and strict/non-strict parser flow; I did not find another implementation path needing the same normalization.
Tests: The new BE unit test covers the hour-only v1 DATE case, but it does not cover the exact 14-digit YYYYMMDDHHMMSS branch, which is one of the two modified implementation branches.
Test results: I did not run the BE unit test locally. git diff --check passed.
Observability/performance/nullability: No new issue found; the change adds no new logging need, allocation path, or nullable-column behavior.
User focus: No additional user-provided review focus was specified.

hello-stephen · 2026-06-16T17:20:42Z

TPC-H: Total hot run time: 28792 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1bc41e29f8e9c72774aff2250007f1bffc579217, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17817	4033	3975	3975
q2	2004	303	180	180
q3	10382	1431	805	805
q4	4675	474	339	339
q5	7509	841	576	576
q6	178	166	134	134
q7	757	863	628	628
q8	9845	1561	1722	1561
q9	6399	4509	4509	4509
q10	6805	1779	1540	1540
q11	440	270	241	241
q12	640	441	286	286
q13	18182	3392	2705	2705
q14	264	257	241	241
q15	q16	779	772	704	704
q17	1024	891	933	891
q18	6895	5765	5512	5512
q19	1180	1255	1047	1047
q20	491	414	265	265
q21	5636	2538	2349	2349
q22	421	356	304	304
Total cold run time: 102323 ms
Total hot run time: 28792 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4335	4246	4245	4245
q2	321	359	225	225
q3	4608	4986	4384	4384
q4	2028	2131	1366	1366
q5	4454	4299	4301	4299
q6	229	174	125	125
q7	1700	1642	1909	1642
q8	2532	2171	2129	2129
q9	7869	7987	8064	7987
q10	4781	4739	4260	4260
q11	582	419	389	389
q12	737	745	537	537
q13	3453	3535	2965	2965
q14	303	332	308	308
q15	q16	709	734	713	713
q17	1339	1319	1344	1319
q18	8136	7204	6996	6996
q19	1120	1100	1113	1100
q20	2215	2213	1936	1936
q21	5242	4559	4399	4399
q22	521	463	393	393
Total cold run time: 57214 ms
Total hot run time: 51717 ms

hello-stephen · 2026-06-16T17:31:56Z

TPC-DS: Total hot run time: 175089 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1bc41e29f8e9c72774aff2250007f1bffc579217, data reload: false
query5	4333	612	478	478
query6	437	191	183	183
query7	4875	551	312	312
query8	373	212	194	194
query9	8730	4000	4013	4000
query10	440	306	259	259
query11	5919	2335	2153	2153
query12	154	103	101	101
query13	1282	597	418	418
query14	6364	5367	5014	5014
query14_1	4385	4359	4351	4351
query15	206	196	181	181
query16	986	459	459	459
query17	1122	714	584	584
query18	2441	485	349	349
query19	201	191	146	146
query20	110	106	108	106
query21	222	143	117	117
query22	13547	13577	13397	13397
query23	17424	16481	16141	16141
query23_1	16289	16355	16285	16285
query24	7542	1783	1318	1318
query24_1	1310	1321	1324	1321
query25	575	449	396	396
query26	1299	327	172	172
query27	2737	590	355	355
query28	4433	2050	2056	2050
query29	1096	617	498	498
query30	309	242	201	201
query31	1107	1076	956	956
query32	110	64	59	59
query33	514	318	262	262
query34	1218	1122	703	703
query35	740	769	676	676
query36	1374	1377	1254	1254
query37	152	102	88	88
query38	3213	3113	3065	3065
query39	938	916	885	885
query39_1	862	888	877	877
query40	211	117	97	97
query41	73	60	59	59
query42	93	90	96	90
query43	319	325	283	283
query44	1414	774	773	773
query45	189	181	175	175
query46	1084	1190	756	756
query47	2391	2346	2237	2237
query48	407	409	293	293
query49	626	463	346	346
query50	1046	350	264	264
query51	4405	4312	4253	4253
query52	84	86	75	75
query53	248	262	184	184
query54	280	210	200	200
query55	75	75	69	69
query56	234	216	205	205
query57	1449	1409	1306	1306
query58	235	211	204	204
query59	1571	1622	1448	1448
query60	292	243	224	224
query61	152	142	150	142
query62	693	645	586	586
query63	236	190	189	189
query64	2509	742	583	583
query65	4872	4777	4798	4777
query66	1802	453	389	389
query67	29019	29621	29622	29621
query68	3056	1468	967	967
query69	398	288	259	259
query70	1060	973	951	951
query71	288	225	197	197
query72	2975	2613	2275	2275
query73	860	776	440	440
query74	5098	4979	4754	4754
query75	2616	2580	2236	2236
query76	2348	1187	795	795
query77	351	367	282	282
query78	12426	12522	11855	11855
query79	1398	1208	741	741
query80	1264	456	376	376
query81	532	277	233	233
query82	641	157	121	121
query83	310	276	239	239
query84	268	143	115	115
query85	879	518	417	417
query86	430	316	306	306
query87	3375	3307	3209	3209
query88	3714	2798	2779	2779
query89	416	385	325	325
query90	1961	186	170	170
query91	173	160	135	135
query92	67	60	58	58
query93	1610	1396	872	872
query94	713	358	299	299
query95	668	473	349	349
query96	1054	803	338	338
query97	2726	2713	2555	2555
query98	213	204	201	201
query99	1165	1188	1036	1036
Total cold run time: 260732 ms
Total hot run time: 175089 ms

hello-stephen · 2026-06-16T17:36:47Z

ClickBench: Total hot run time: 25.23 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1bc41e29f8e9c72774aff2250007f1bffc579217, data reload: false
query1	0.00	0.00	0.00
query2	0.09	0.04	0.05
query3	0.26	0.14	0.13
query4	1.61	0.14	0.13
query5	0.25	0.24	0.22
query6	1.30	1.05	1.06
query7	0.05	0.01	0.00
query8	0.06	0.04	0.04
query9	0.37	0.34	0.31
query10	0.55	0.53	0.57
query11	0.20	0.14	0.14
query12	0.18	0.14	0.15
query13	0.48	0.47	0.47
query14	1.01	1.01	1.02
query15	0.60	0.58	0.60
query16	0.31	0.33	0.32
query17	1.09	1.09	1.15
query18	0.22	0.22	0.20
query19	1.96	1.83	1.88
query20	0.01	0.02	0.01
query21	15.42	0.21	0.14
query22	4.82	0.06	0.05
query23	16.10	0.32	0.12
query24	2.92	0.42	0.32
query25	0.11	0.04	0.04
query26	0.75	0.21	0.15
query27	0.04	0.04	0.03
query28	3.56	0.95	0.55
query29	12.47	4.31	3.49
query30	0.28	0.15	0.15
query31	2.76	0.59	0.31
query32	3.23	0.60	0.48
query33	3.14	3.28	3.21
query34	15.66	4.24	3.59
query35	3.48	3.56	3.55
query36	0.54	0.43	0.43
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.4 s
Total hot run time: 25.23 s

hello-stephen · 2026-06-16T19:27:51Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	74.03% (28328/38266)
Line Coverage	58.01% (308948/532599)
Region Coverage	54.96% (259410/471978)
Branch Coverage	56.23% (112396/199884)

github-actions · 2026-06-17T02:09:37Z

PR approved by at least one committer and no changes requested.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

jacktengg · 2026-06-17T02:27:15Z

/review

jacktengg · 2026-06-17T02:28:49Z

run buildall

github-actions

No blocking issues found in the current patch.

Checkpoint conclusions:

Goal/test: The change normalizes v1 DATE results before both affected early returns in CastToDateOrDatetime::from_string_strict_mode. The added BE unit test covers the hour-only path and the exact 14-digit YYYYMMDDHHMMSS path, including the case previously raised in the existing review thread.
Scope: The implementation is small and focused: two normalization calls plus targeted v1 DATE strict-mode coverage.
Parallel paths: The modified parser is the v1 DATE/DATETIME path. DATEV2/DATETIMEV2 have separate implementations; the v1 final path already normalizes, and the two early exits now match it.
Concurrency/lifecycle/config/compatibility/data writes/persistence/FE-BE protocol: Not applicable for this local cast parser change.
Error handling: Existing Status/strict-mode handling is unchanged; the added calls do not introduce silent continuation.
Performance/observability: No meaningful regression or new observability need; this is a constant-time normalization on early return paths.
Test results: I did not run BE UT locally. git diff --check passed for the PR files. The visible macOS BE UT failure is a runner setup issue (Java 25 reported where JDK 17 is required) before tests compile/run, not a failure of this patch.
User focus: No additional user-provided review focus was specified.

hello-stephen · 2026-06-17T03:52:26Z

BE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	54.38% (21311/39191)
Line Coverage	37.99% (203551/535750)
Region Coverage	34.02% (159849/469834)
Branch Coverage	35.02% (69908/199629)

hello-stephen · 2026-06-17T04:06:55Z

TPC-H: Total hot run time: 29169 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7c5967906277d9b1487c785174748249ca3198f8, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17769	4045	3965	3965
q2	2034	318	188	188
q3	10283	1504	791	791
q4	4678	460	338	338
q5	7627	847	569	569
q6	183	170	133	133
q7	768	829	630	630
q8	9323	1650	1687	1650
q9	5802	4553	4521	4521
q10	6617	1792	1537	1537
q11	427	268	238	238
q12	630	430	288	288
q13	18111	3593	2789	2789
q14	269	261	239	239
q15	q16	794	775	714	714
q17	990	990	849	849
q18	7663	5829	5678	5678
q19	1305	1225	1125	1125
q20	488	399	266	266
q21	5917	2705	2360	2360
q22	429	354	301	301
Total cold run time: 102107 ms
Total hot run time: 29169 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4332	4219	4269	4219
q2	328	357	231	231
q3	4620	4970	4459	4459
q4	2066	2136	1376	1376
q5	4430	4280	4296	4280
q6	227	173	124	124
q7	1712	1867	1833	1833
q8	2563	2187	2162	2162
q9	8177	8178	7972	7972
q10	4821	4764	4352	4352
q11	590	414	377	377
q12	752	764	535	535
q13	3333	3659	3034	3034
q14	289	315	268	268
q15	q16	700	726	637	637
q17	1340	1332	1459	1332
q18	7889	7267	7097	7097
q19	1197	1096	1109	1096
q20	2212	2217	1933	1933
q21	5269	4589	4432	4432
q22	506	452	419	419
Total cold run time: 57353 ms
Total hot run time: 52168 ms

hello-stephen · 2026-06-17T04:18:00Z

TPC-DS: Total hot run time: 176147 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7c5967906277d9b1487c785174748249ca3198f8, data reload: false
query5	4329	616	489	489
query6	443	186	169	169
query7	4806	545	314	314
query8	366	215	222	215
query9	8743	4074	4042	4042
query10	473	307	243	243
query11	5934	2352	2170	2170
query12	156	101	97	97
query13	1247	615	451	451
query14	6398	5386	5040	5040
query14_1	4404	4381	4385	4381
query15	208	201	178	178
query16	994	458	460	458
query17	943	724	600	600
query18	2448	491	353	353
query19	200	193	146	146
query20	111	112	108	108
query21	221	140	118	118
query22	13582	13497	13460	13460
query23	17257	16516	16223	16223
query23_1	16325	16257	16322	16257
query24	7471	1801	1311	1311
query24_1	1320	1282	1343	1282
query25	591	465	405	405
query26	1310	333	176	176
query27	2672	584	341	341
query28	4465	2043	2055	2043
query29	1085	632	512	512
query30	314	241	206	206
query31	1143	1094	945	945
query32	103	64	64	64
query33	535	325	263	263
query34	1170	1130	662	662
query35	775	765	672	672
query36	1377	1359	1255	1255
query37	149	104	87	87
query38	3225	3132	3045	3045
query39	941	910	902	902
query39_1	877	914	870	870
query40	212	117	97	97
query41	61	65	58	58
query42	95	92	94	92
query43	315	323	281	281
query44	1412	774	793	774
query45	191	188	176	176
query46	1051	1175	730	730
query47	2358	2378	2277	2277
query48	397	424	299	299
query49	642	468	348	348
query50	1040	372	257	257
query51	4348	4279	4223	4223
query52	88	86	75	75
query53	247	267	197	197
query54	258	214	191	191
query55	77	74	68	68
query56	231	229	209	209
query57	1430	1418	1304	1304
query58	230	212	207	207
query59	1590	1666	1424	1424
query60	275	246	228	228
query61	156	152	148	148
query62	683	649	588	588
query63	230	192	188	188
query64	2489	765	596	596
query65	4840	4770	4784	4770
query66	1788	450	331	331
query67	29745	29663	29486	29486
query68	3426	1577	995	995
query69	400	305	257	257
query70	1049	967	929	929
query71	285	233	225	225
query72	2957	2649	2353	2353
query73	851	762	428	428
query74	5096	4961	4768	4768
query75	2621	2611	2230	2230
query76	2310	1190	787	787
query77	354	368	274	274
query78	12387	12516	11930	11930
query79	1440	1173	801	801
query80	596	466	391	391
query81	456	276	239	239
query82	571	154	121	121
query83	348	276	251	251
query84	299	144	118	118
query85	856	524	417	417
query86	370	320	287	287
query87	3374	3331	3215	3215
query88	3700	2791	2798	2791
query89	426	388	336	336
query90	1948	182	181	181
query91	175	163	131	131
query92	60	60	57	57
query93	1405	1475	884	884
query94	524	358	318	318
query95	676	392	346	346
query96	1119	869	371	371
query97	2703	2693	2553	2553
query98	220	206	204	204
query99	1169	1174	1011	1011
Total cold run time: 260122 ms
Total hot run time: 176147 ms

hello-stephen · 2026-06-17T04:22:56Z

ClickBench: Total hot run time: 25.11 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7c5967906277d9b1487c785174748249ca3198f8, data reload: false
query1	0.00	0.00	0.01
query2	0.10	0.05	0.04
query3	0.25	0.14	0.14
query4	1.61	0.13	0.14
query5	0.24	0.23	0.23
query6	1.28	1.05	1.05
query7	0.04	0.01	0.01
query8	0.06	0.03	0.04
query9	0.36	0.32	0.31
query10	0.55	0.54	0.58
query11	0.19	0.14	0.14
query12	0.20	0.14	0.14
query13	0.47	0.47	0.47
query14	1.02	0.99	0.99
query15	0.60	0.60	0.59
query16	0.30	0.33	0.34
query17	1.10	1.07	1.10
query18	0.23	0.22	0.21
query19	2.01	1.90	1.95
query20	0.02	0.01	0.01
query21	15.46	0.20	0.13
query22	4.99	0.06	0.05
query23	16.12	0.32	0.11
query24	2.93	0.41	0.33
query25	0.11	0.05	0.05
query26	0.73	0.22	0.15
query27	0.04	0.04	0.04
query28	3.52	0.95	0.53
query29	12.50	4.42	3.46
query30	0.27	0.15	0.17
query31	2.78	0.58	0.31
query32	3.23	0.59	0.48
query33	3.15	3.26	3.20
query34	15.83	4.20	3.50
query35	3.56	3.51	3.53
query36	0.56	0.43	0.44
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.17	0.15
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 96.93 s
Total hot run time: 25.11 s

Fixed

hello-stephen · 2026-06-17T06:49:55Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	74.07% (28346/38267)
Line Coverage	58.03% (309081/532618)
Region Coverage	54.90% (259129/471979)
Branch Coverage	56.25% (112443/199887)

Problem Summary: Strict-mode string casts to v1 DATE can return early after parsing a compact datetime string or an hour-only time part. Those early returns skipped the common target-type normalization, so DATE results could keep non-zero time fields such as 2024-05-01 01:00:00 instead of 2024-05-01. Normalize the parsed value before both early returns so DATE clears the time fields and DATETIME keeps datetime semantics. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…4744) Cherry-picked from #64575 Co-authored-by: TengJianPing <tengjianping@selectdb.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actionsBot previously requested changes Jun 16, 2026
View reviewed changes

Comment threadbe/test/exprs/function/cast/cast_to_date_test.cpp

zclllyybb previously approved these changes Jun 17, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 17, 2026

Update be/test/exprs/function/cast/cast_to_date_test.cpp
7c59679
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

jacktengg dismissed zclllyybb’s stale review via 7c59679June 17, 2026 02:21

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Jun 17, 2026

github-actionsBot reviewed Jun 17, 2026
View reviewed changes

zclllyybb approved these changes Jun 17, 2026
View reviewed changes

zclllyybb merged commit 6879d61 into apache:masterJun 17, 2026
33 of 34 checks passed

yiguolei added the dev/4.1.x label Jun 23, 2026

github-actionsBot mentioned this pull request Jun 23, 2026
branch-4.1: [fix](be) Normalize v1 date string cast result #64575 #64744
Merged

yiguolei added dev/4.1.3-merged and removed dev/4.1.x labels Jun 24, 2026

yiguolei mentioned this pull request Jul 7, 2026
4.1.3 Release Note #65306
Closed

Uh oh!

Conversation

jacktengg commented Jun 16, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jun 16, 2026

Uh oh!

jacktengg commented Jun 16, 2026

Uh oh!

jacktengg commented Jun 16, 2026

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hello-stephen commented Jun 16, 2026

Uh oh!

hello-stephen commented Jun 16, 2026

Uh oh!

hello-stephen commented Jun 16, 2026

Uh oh!

hello-stephen commented Jun 16, 2026

BE Regression && UT Coverage Report

Uh oh!

github-actionsBot commented Jun 17, 2026

Uh oh!

jacktengg commented Jun 17, 2026

Uh oh!

jacktengg commented Jun 17, 2026

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

hello-stephen commented Jun 17, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Jun 17, 2026

Uh oh!

hello-stephen commented Jun 17, 2026

Uh oh!

hello-stephen commented Jun 17, 2026

Uh oh!

hello-stephen commented Jun 17, 2026

BE Regression && UT Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jacktengg commented Jun 16, 2026•
edited
Loading