[fix](nereids) fix compare with date like overflow by yujun777 · Pull Request #45868 · apache/doris

yujun777 · 2024-12-24T09:53:57Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

fix compare date like overflow, for example:

c_date < '9999-12-31 23:59:59' => c_date < '9999-12-31' + 1 day
c_datetime_0 < '9999-12-31 23:59:59.1' => c_datetime_0 < '9999-12-31 23:59:59' + 1 second

there two rewritten exprs will met an overflow case.

to fix this, when met overflow case, we don't rewrite the expression.

but what's more, these above two expression both can rewrite to 'TRUE'

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2024-12-24T09:54:02Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

yujun777 · 2024-12-24T11:29:26Z

run buildall

doris-robot · 2024-12-24T12:00:53Z

TPC-H: Total hot run time: 32813 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 45fb10352833d0bcc00d8c61f6bfd25b834d9558, data reload: false
------ Round 1 ----------------------------------
q1	17563	6227	6093	6093
q2	2058	295	180	180
q3	10414	1266	722	722
q4	10221	861	434	434
q5	7943	2225	2006	2006
q6	205	189	150	150
q7	922	745	630	630
q8	9240	1397	1162	1162
q9	5337	4962	4965	4962
q10	6773	2313	1879	1879
q11	480	273	270	270
q12	367	376	228	228
q13	17782	3603	3094	3094
q14	227	257	210	210
q15	561	507	502	502
q16	643	610	588	588
q17	583	856	322	322
q18	6999	6533	6318	6318
q19	1239	979	589	589
q20	319	325	190	190
q21	3117	2235	1969	1969
q22	361	327	315	315
Total cold run time: 103354 ms
Total hot run time: 32813 ms
----- Round 2, with runtime_filter_mode=off -----
q1	6226	6217	6215	6215
q2	237	330	240	240
q3	2221	2684	2309	2309
q4	1413	1814	1379	1379
q5	4376	4800	4869	4800
q6	184	174	147	147
q7	2115	1942	1846	1846
q8	2616	2820	2667	2667
q9	7309	7250	7203	7203
q10	3067	3337	2833	2833
q11	581	528	512	512
q12	652	743	594	594
q13	3473	3769	3096	3096
q14	302	313	288	288
q15	575	555	494	494
q16	648	685	641	641
q17	1244	1762	1256	1256
q18	7705	7317	7097	7097
q19	825	1135	1069	1069
q20	1945	2001	1828	1828
q21	5497	5223	4948	4948
q22	607	634	593	593
Total cold run time: 53818 ms
Total hot run time: 52055 ms

doris-robot · 2024-12-24T12:12:56Z

TPC-DS: Total hot run time: 188969 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 45fb10352833d0bcc00d8c61f6bfd25b834d9558, data reload: false
query1	954	373	382	373
query2	6516	2379	2238	2238
query3	6703	210	214	210
query4	33851	23696	23221	23221
query5	4301	595	467	467
query6	291	209	184	184
query7	4614	478	311	311
query8	306	248	233	233
query9	9622	2766	2747	2747
query10	483	302	255	255
query11	18318	15262	15153	15153
query12	154	106	110	106
query13	1674	533	410	410
query14	9888	7776	7028	7028
query15	267	201	185	185
query16	8663	607	451	451
query17	1608	755	598	598
query18	2091	387	304	304
query19	213	175	145	145
query20	116	112	105	105
query21	211	122	102	102
query22	4362	4361	4268	4268
query23	34423	33315	33044	33044
query24	6405	2384	2289	2289
query25	490	439	372	372
query26	1187	272	155	155
query27	2011	461	327	327
query28	5198	2472	2422	2422
query29	698	530	417	417
query30	232	180	153	153
query31	1024	914	842	842
query32	83	60	55	55
query33	514	358	291	291
query34	763	839	521	521
query35	808	788	714	714
query36	1001	1049	970	970
query37	118	95	71	71
query38	4111	4211	4112	4112
query39	1490	1427	1438	1427
query40	206	113	102	102
query41	47	47	45	45
query42	122	103	103	103
query43	515	530	482	482
query44	1300	803	804	803
query45	180	167	169	167
query46	844	1032	645	645
query47	1943	1965	1878	1878
query48	378	400	328	328
query49	780	466	386	386
query50	607	654	391	391
query51	7121	7161	6916	6916
query52	96	98	87	87
query53	218	256	178	178
query54	471	474	402	402
query55	83	80	76	76
query56	254	275	229	229
query57	1216	1188	1103	1103
query58	239	219	230	219
query59	3024	3185	2890	2890
query60	286	262	240	240
query61	116	111	117	111
query62	878	826	772	772
query63	225	187	186	186
query64	3950	1027	653	653
query65	3279	3181	3233	3181
query66	1050	412	309	309
query67	15910	15874	15682	15682
query68	9520	754	495	495
query69	495	289	253	253
query70	1222	1171	1111	1111
query71	445	326	255	255
query72	5838	3745	3898	3745
query73	674	751	373	373
query74	10148	9039	8888	8888
query75	4545	3135	2642	2642
query76	5547	1173	753	753
query77	993	367	270	270
query78	10176	10254	9314	9314
query79	3354	893	574	574
query80	712	510	432	432
query81	471	277	241	241
query82	628	152	126	126
query83	195	166	145	145
query84	282	88	68	68
query85	802	398	389	389
query86	353	287	303	287
query87	4670	4571	4276	4276
query88	3901	2245	2197	2197
query89	417	333	297	297
query90	1910	185	186	185
query91	134	132	104	104
query92	65	56	52	52
query93	1243	876	528	528
query94	656	411	289	289
query95	329	259	245	245
query96	484	603	280	280
query97	2784	2814	2689	2689
query98	216	204	200	200
query99	1702	1582	1426	1426
Total cold run time: 296011 ms
Total hot run time: 188969 ms

doris-robot · 2024-12-24T12:18:16Z

ClickBench: Total hot run time: 31.56 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 45fb10352833d0bcc00d8c61f6bfd25b834d9558, data reload: false
query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.06	0.07
query4	1.61	0.10	0.10
query5	0.43	0.41	0.40
query6	1.16	0.65	0.67
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.49	0.52
query10	0.55	0.59	0.57
query11	0.14	0.10	0.10
query12	0.14	0.12	0.12
query13	0.61	0.60	0.59
query14	2.71	2.81	2.83
query15	0.90	0.81	0.82
query16	0.40	0.39	0.39
query17	1.02	1.07	1.01
query18	0.24	0.19	0.20
query19	1.94	1.80	1.95
query20	0.02	0.01	0.01
query21	15.35	0.92	0.57
query22	0.75	0.96	0.74
query23	15.07	1.39	0.56
query24	2.61	1.61	1.17
query25	0.24	0.15	0.06
query26	0.30	0.15	0.14
query27	0.05	0.04	0.04
query28	13.62	1.49	1.05
query29	12.63	3.97	3.23
query30	0.25	0.09	0.06
query31	2.82	0.61	0.38
query32	3.23	0.55	0.46
query33	3.11	3.11	3.23
query34	16.65	5.15	4.50
query35	4.55	4.50	4.51
query36	0.66	0.48	0.48
query37	0.09	0.06	0.06
query38	0.05	0.05	0.03
query39	0.04	0.03	0.03
query40	0.16	0.13	0.13
query41	0.07	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.22 s
Total hot run time: 31.56 s

yujun777 · 2024-12-25T02:28:45Z

run buildall

github-actions · 2024-12-25T03:07:31Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-12-25T03:07:32Z

PR approved by anyone and no changes requested.

doris-robot · 2024-12-25T03:32:37Z

TPC-H: Total hot run time: 32926 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 56008d3278102d5fdb8c6dc0b70e8e724402a767, data reload: false
------ Round 1 ----------------------------------
q1	17740	6211	6092	6092
q2	2040	308	175	175
q3	10510	1288	748	748
q4	10215	861	429	429
q5	7644	2231	2009	2009
q6	210	180	149	149
q7	904	754	615	615
q8	9239	1405	1204	1204
q9	5290	4982	4996	4982
q10	6792	2336	1893	1893
q11	476	283	266	266
q12	359	361	225	225
q13	17762	3571	3008	3008
q14	261	239	216	216
q15	560	517	504	504
q16	646	615	589	589
q17	583	870	321	321
q18	7138	6466	6406	6406
q19	1899	975	579	579
q20	310	335	206	206
q21	2865	2273	1996	1996
q22	375	326	314	314
Total cold run time: 103818 ms
Total hot run time: 32926 ms
----- Round 2, with runtime_filter_mode=off -----
q1	6317	6285	6281	6281
q2	242	331	225	225
q3	2234	2669	2334	2334
q4	1392	1814	1338	1338
q5	4378	4820	4997	4820
q6	191	179	142	142
q7	2099	1977	1823	1823
q8	2603	2850	2691	2691
q9	7331	7174	7286	7174
q10	3039	3363	2814	2814
q11	575	531	500	500
q12	693	761	594	594
q13	3456	3738	3111	3111
q14	289	324	278	278
q15	560	509	520	509
q16	645	689	635	635
q17	1214	1762	1270	1270
q18	7597	7531	7050	7050
q19	815	1079	1097	1079
q20	1910	1968	1822	1822
q21	5682	5223	4942	4942
q22	616	608	574	574
Total cold run time: 53878 ms
Total hot run time: 52006 ms

doris-robot · 2024-12-25T03:44:50Z

TPC-DS: Total hot run time: 191774 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 56008d3278102d5fdb8c6dc0b70e8e724402a767, data reload: false
query1	977	376	363	363
query2	6516	2482	2363	2363
query3	6714	226	213	213
query4	33708	23743	23751	23743
query5	4323	612	469	469
query6	293	208	192	192
query7	4636	495	311	311
query8	298	234	226	226
query9	9525	2764	2743	2743
query10	450	319	247	247
query11	18070	15572	15647	15572
query12	175	109	102	102
query13	1666	563	417	417
query14	11817	7776	7849	7776
query15	244	187	190	187
query16	8169	639	456	456
query17	1570	755	585	585
query18	2136	422	307	307
query19	228	188	170	170
query20	126	111	114	111
query21	211	125	104	104
query22	4292	4226	4267	4226
query23	34716	33112	34472	33112
query24	6102	2284	2276	2276
query25	456	441	386	386
query26	1077	249	155	155
query27	2015	449	334	334
query28	5188	2490	2433	2433
query29	546	501	409	409
query30	228	180	152	152
query31	991	885	779	779
query32	93	65	62	62
query33	507	365	301	301
query34	770	836	508	508
query35	814	824	738	738
query36	995	1050	960	960
query37	113	91	71	71
query38	4175	4200	4177	4177
query39	1486	1488	1418	1418
query40	211	119	102	102
query41	47	44	44	44
query42	117	103	104	103
query43	525	549	497	497
query44	1311	799	798	798
query45	181	177	161	161
query46	855	1031	644	644
query47	1923	1933	1893	1893
query48	391	402	336	336
query49	795	473	412	412
query50	637	648	386	386
query51	7139	7229	7081	7081
query52	102	103	95	95
query53	225	259	184	184
query54	467	486	403	403
query55	89	78	79	78
query56	256	265	245	245
query57	1198	1183	1124	1124
query58	236	236	230	230
query59	2997	3198	3122	3122
query60	266	270	263	263
query61	113	136	114	114
query62	842	797	740	740
query63	223	189	197	189
query64	4563	1044	637	637
query65	3300	3200	3194	3194
query66	1046	422	310	310
query67	16058	15695	15546	15546
query68	9162	734	502	502
query69	473	286	256	256
query70	1214	1152	1109	1109
query71	446	286	257	257
query72	5813	3831	3859	3831
query73	656	756	373	373
query74	10210	9226	9016	9016
query75	4490	3161	2694	2694
query76	5249	1194	833	833
query77	837	352	281	281
query78	10077	10047	9383	9383
query79	6027	899	574	574
query80	736	521	439	439
query81	466	269	230	230
query82	351	155	129	129
query83	248	167	147	147
query84	288	96	70	70
query85	746	407	311	311
query86	345	322	306	306
query87	4484	4440	4432	4432
query88	3912	2222	2200	2200
query89	449	341	292	292
query90	2086	187	187	187
query91	130	134	108	108
query92	66	56	55	55
query93	3564	877	519	519
query94	649	396	292	292
query95	336	259	253	253
query96	493	608	281	281
query97	2796	2819	2698	2698
query98	224	199	200	199
query99	1640	1583	1432	1432
Total cold run time: 301192 ms
Total hot run time: 191774 ms

doris-robot · 2024-12-25T03:50:11Z

ClickBench: Total hot run time: 31.91 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 56008d3278102d5fdb8c6dc0b70e8e724402a767, data reload: false
query1	0.03	0.03	0.03
query2	0.08	0.03	0.03
query3	0.23	0.07	0.07
query4	1.64	0.10	0.10
query5	0.41	0.40	0.42
query6	1.15	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.50	0.50
query10	0.55	0.58	0.57
query11	0.14	0.10	0.10
query12	0.13	0.11	0.10
query13	0.61	0.61	0.60
query14	2.70	2.74	2.86
query15	0.89	0.83	0.83
query16	0.39	0.38	0.37
query17	1.03	1.09	1.07
query18	0.22	0.19	0.20
query19	1.85	1.76	2.03
query20	0.02	0.01	0.01
query21	15.36	0.92	0.57
query22	0.75	0.74	0.78
query23	15.24	1.45	0.59
query24	3.35	1.87	1.44
query25	0.13	0.09	0.14
query26	0.38	0.14	0.14
query27	0.06	0.05	0.04
query28	14.06	1.46	1.04
query29	12.59	3.96	3.24
query30	0.26	0.09	0.06
query31	2.85	0.61	0.38
query32	3.23	0.55	0.47
query33	3.05	3.08	3.12
query34	16.75	5.16	4.54
query35	4.51	4.53	4.54
query36	0.65	0.49	0.51
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.16	0.13	0.13
query41	0.09	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 106.41 s
Total hot run time: 31.91 s

…#45922) cherry pick from #45868

…#45929) cherry pick from #45868

fix compare with date like overflow
ba78004

add test
45fb103

fix msg
56008d3

morrySnow added dev/2.1.x dev/3.0.x p0_b labels Dec 25, 2024

morrySnow approved these changes Dec 25, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 25, 2024

github-actionsBot added the reviewed label Dec 25, 2024

starocean999 approved these changes Dec 25, 2024
View reviewed changes

starocean999 merged commit ac385e5 into apache:masterDec 25, 2024

github-actionsBot added dev/3.0.x-conflict dev/2.1.x-conflict labels Dec 25, 2024

This was referenced Dec 25, 2024
branch-3.0: [fix](nereids) fix compare with date like overflow #45868 #45922
Merged
branch-2.1: [fix](nereids) fix compare with date like overflow #45868 #45929
Merged

morrySnow pushed a commit that referenced this pull request Dec 25, 2024
branch-3.0: [fix](nereids) fix compare with date like overflow#45868(…
3374363
…#45922) cherry pick from #45868

morrySnow added dev/3.0.4-merged and removed dev/3.0.x dev/3.0.x-conflict labels Dec 25, 2024

morrySnow pushed a commit that referenced this pull request Dec 27, 2024
branch-2.1: [fix](nereids) fix compare with date like overflow#45868(…
60d3c1e
…#45929) cherry pick from #45868

morrySnow added dev/2.1.8-merged and removed dev/2.1.x dev/2.1.x-conflict labels Dec 27, 2024

yiguolei mentioned this pull request Jan 19, 2025
Release Note 2.1.8 #47198
Closed

Navigation Menu

Uh oh!

[fix](nereids) fix compare with date like overflow - #45868

[fix](nereids) fix compare with date like overflow#45868
starocean999 merged 3 commits into
apache:masterfrom
yujun777:fix-compare-with-date-like-overflow

yujun777 commented Dec 24, 2024•
edited
Loading

Uh oh!

hello-stephen commented Dec 24, 2024

Uh oh!

yujun777 commented Dec 24, 2024

Uh oh!

doris-robot commented Dec 24, 2024

Uh oh!

doris-robot commented Dec 24, 2024

Uh oh!

doris-robot commented Dec 24, 2024

Uh oh!

yujun777 commented Dec 25, 2024

Uh oh!

github-actionsBot commented Dec 25, 2024

Uh oh!

github-actionsBot commented Dec 25, 2024

Uh oh!

doris-robot commented Dec 25, 2024

Uh oh!

doris-robot commented Dec 25, 2024

Uh oh!

doris-robot commented Dec 25, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

yujun777 commented Dec 24, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Dec 24, 2024

Uh oh!

yujun777 commented Dec 24, 2024

Uh oh!

doris-robot commented Dec 24, 2024

Uh oh!

doris-robot commented Dec 24, 2024

Uh oh!

doris-robot commented Dec 24, 2024

Uh oh!

yujun777 commented Dec 25, 2024

Uh oh!

github-actionsBot commented Dec 25, 2024

Uh oh!

github-actionsBot commented Dec 25, 2024

Uh oh!

doris-robot commented Dec 25, 2024

Uh oh!

doris-robot commented Dec 25, 2024

Uh oh!

doris-robot commented Dec 25, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

yujun777 commented Dec 24, 2024•
edited
Loading