[fix](nereids)should not push down constant output expr in window expression by starocean999 · Pull Request #35948 · apache/doris

starocean999 · 2024-06-06T03:39:46Z

SELECT 0 AS x1, (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE 0 END) AS x2 FROM t

0 AS x1 was pushed down to bottom project. Then it will be used to normalize case when expr, and change it to
(CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE x1 END) AS x2

the literal 0 was replaced by x1 accidentally. This pr fix it by keep constant expr in output list

Issue Number: close #xxx

…ression

doris-robot · 2024-06-06T03:39:50Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

starocean999 · 2024-06-06T03:39:55Z

run buildall

doris-robot · 2024-06-06T04:16:18Z

TPC-H: Total hot run time: 41012 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6d3b1ce5cbf537bb40fe55992599501cb99ce9ba, data reload: false
------ Round 1 ----------------------------------
q1	17638	4327	4197	4197
q2	2031	195	199	195
q3	10448	1203	1210	1203
q4	10193	840	804	804
q5	7480	2705	2690	2690
q6	217	135	137	135
q7	942	621	614	614
q8	9262	2129	2095	2095
q9	9148	6686	6712	6686
q10	9676	3892	3838	3838
q11	442	236	257	236
q12	448	240	235	235
q13	17343	3252	3214	3214
q14	268	217	222	217
q15	521	467	457	457
q16	507	390	389	389
q17	1002	607	666	607
q18	8232	7752	7745	7745
q19	6933	1491	1410	1410
q20	636	338	340	338
q21	5104	3370	4084	3370
q22	403	337	340	337
Total cold run time: 118874 ms
Total hot run time: 41012 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4531	4371	4394	4371
q2	378	287	278	278
q3	3107	2985	2843	2843
q4	1895	1685	1632	1632
q5	5467	5444	5513	5444
q6	219	126	127	126
q7	2180	1847	1814	1814
q8	3228	3387	3370	3370
q9	8594	8647	8692	8647
q10	4059	3812	3687	3687
q11	580	496	490	490
q12	819	645	626	626
q13	15930	3144	3145	3144
q14	291	280	268	268
q15	521	486	486	486
q16	496	427	417	417
q17	1798	1484	1549	1484
q18	8009	7519	7275	7275
q19	1698	1552	1593	1552
q20	2069	1779	1783	1779
q21	7028	4669	4757	4669
q22	681	550	547	547
Total cold run time: 73578 ms
Total hot run time: 54949 ms

doris-robot · 2024-06-06T04:27:15Z

TPC-DS: Total hot run time: 170212 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6d3b1ce5cbf537bb40fe55992599501cb99ce9ba, data reload: false
query1	928	384	393	384
query2	6457	2580	2333	2333
query3	6649	205	208	205
query4	19455	17186	17400	17186
query5	4159	448	448	448
query6	251	156	156	156
query7	4590	293	291	291
query8	317	284	281	281
query9	8372	2417	2418	2417
query10	430	292	266	266
query11	10571	9985	9999	9985
query12	143	80	84	80
query13	1627	360	371	360
query14	10135	7590	7523	7523
query15	229	196	188	188
query16	7736	257	266	257
query17	1344	519	511	511
query18	1952	281	282	281
query19	202	154	151	151
query20	92	85	81	81
query21	212	138	152	138
query22	4371	4097	4107	4097
query23	33653	33051	32976	32976
query24	11266	2869	2811	2811
query25	629	370	352	352
query26	1397	150	154	150
query27	3017	312	321	312
query28	7502	2051	2039	2039
query29	933	623	586	586
query30	269	145	148	145
query31	935	717	726	717
query32	94	54	53	53
query33	748	276	272	272
query34	983	466	493	466
query35	736	607	627	607
query36	1079	930	883	883
query37	150	70	70	70
query38	2871	2755	2715	2715
query39	858	789	790	789
query40	214	123	122	122
query41	57	51	56	51
query42	115	100	95	95
query43	585	545	538	538
query44	1217	733	743	733
query45	197	155	160	155
query46	1067	712	712	712
query47	1847	1782	1772	1772
query48	361	298	292	292
query49	1074	397	408	397
query50	778	395	391	391
query51	6750	6774	6603	6603
query52	110	92	88	88
query53	356	283	310	283
query54	838	435	430	430
query55	74	75	74	74
query56	277	252	264	252
query57	1124	1034	1065	1034
query58	265	232	235	232
query59	3490	3244	2994	2994
query60	290	268	266	266
query61	88	84	83	83
query62	651	447	459	447
query63	322	286	286	286
query64	9962	2305	1821	1821
query65	3160	3085	3133	3085
query66	1068	348	336	336
query67	15302	14890	15095	14890
query68	4624	540	530	530
query69	543	447	346	346
query70	1143	1080	1105	1080
query71	393	279	272	272
query72	7351	5941	2863	2863
query73	739	321	323	321
query74	5750	5558	5516	5516
query75	3359	2703	2680	2680
query76	2543	946	882	882
query77	656	306	299	299
query78	10440	9967	9771	9771
query79	2728	508	510	508
query80	2498	485	477	477
query81	585	223	224	223
query82	1490	114	107	107
query83	316	180	176	176
query84	273	89	93	89
query85	1664	328	318	318
query86	489	315	324	315
query87	3302	3106	3069	3069
query88	3883	2516	2322	2322
query89	476	387	380	380
query90	1876	194	188	188
query91	128	96	96	96
query92	62	50	48	48
query93	2543	514	506	506
query94	1333	193	177	177
query95	405	326	310	310
query96	602	268	268	268
query97	3172	3080	3030	3030
query98	224	210	199	199
query99	1296	825	822	822
Total cold run time: 275867 ms
Total hot run time: 170212 ms

doris-robot · 2024-06-06T04:32:48Z

ClickBench: Total hot run time: 30.69 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6d3b1ce5cbf537bb40fe55992599501cb99ce9ba, data reload: false
query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.49	0.49
query6	1.12	0.71	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.50	0.49
query10	0.54	0.55	0.54
query11	0.15	0.11	0.11
query12	0.15	0.11	0.12
query13	0.59	0.58	0.62
query14	0.77	0.77	0.79
query15	0.83	0.81	0.80
query16	0.36	0.36	0.35
query17	0.99	0.96	0.99
query18	0.22	0.26	0.25
query19	1.89	1.81	1.80
query20	0.01	0.01	0.01
query21	15.66	0.64	0.63
query22	4.04	7.64	2.21
query23	18.24	1.37	1.25
query24	2.06	0.21	0.22
query25	0.15	0.08	0.08
query26	0.27	0.17	0.17
query27	0.09	0.08	0.08
query28	13.23	1.03	1.00
query29	13.93	3.32	3.23
query30	0.25	0.05	0.05
query31	2.85	0.39	0.38
query32	3.28	0.46	0.46
query33	2.89	2.87	2.93
query34	17.12	4.39	4.43
query35	4.50	4.47	4.58
query36	0.68	0.46	0.49
query37	0.17	0.15	0.14
query38	0.14	0.13	0.13
query39	0.04	0.03	0.04
query40	0.16	0.16	0.14
query41	0.08	0.05	0.04
query42	0.05	0.04	0.04
query43	0.04	0.03	0.04
Total cold run time: 110.67 s
Total hot run time: 30.69 s

github-actions · 2024-06-06T08:52:29Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-06-06T08:52:31Z

PR approved by anyone and no changes requested.

…ression (#35964) pick from master #35948 SELECT 0 AS x1, (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE 0 END) AS x2 FROM t 0 AS x1 was pushed down to bottom project. Then it will be used to normalize case when expr, and change it to (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE x1 END) AS x2 the literal 0 was replaced by x1 accidentally. This pr fix it by keep constant expr in output list

…ression (#35961) pick from master #35948 SELECT 0 AS x1, (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE 0 END) AS x2 FROM t 0 AS x1 was pushed down to bottom project. Then it will be used to normalize case when expr, and change it to (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE x1 END) AS x2 the literal 0 was replaced by x1 accidentally. This pr fix it by keep constant expr in output list

…ression (#35948) SELECT 0 AS x1, (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE 0 END) AS x2 FROM t 0 AS x1 was pushed down to bottom project. Then it will be used to normalize case when expr, and change it to (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE **x1** END) AS x2 the literal 0 was replaced by x1 accidentally. This pr fix it by keep constant expr in output list

…datatype too (#36055) this pr revert #35948 and fix the bug in correct way. The root cause is decimal literal's comparison only compare value, we should also compare their datatype as well as datetime literal. suppose a=0 with type decimal(2,1) b=0 with type decimal(3,1) we have: a.equals(b) returns false

…ression (apache#35961) pick from master apache#35948 SELECT 0 AS x1, (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE 0 END) AS x2 FROM t 0 AS x1 was pushed down to bottom project. Then it will be used to normalize case when expr, and change it to (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE x1 END) AS x2 the literal 0 was replaced by x1 accidentally. This pr fix it by keep constant expr in output list

…ression (apache#35948) SELECT 0 AS x1, (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE 0 END) AS x2 FROM t 0 AS x1 was pushed down to bottom project. Then it will be used to normalize case when expr, and change it to (CASE WHEN t.a IS NOT NULL THEN 1.0 / count(1) OVER (PARTITION BY t.b) ELSE **x1** END) AS x2 the literal 0 was replaced by x1 accidentally. This pr fix it by keep constant expr in output list

…datatype too (apache#36055) this pr revert apache#35948 and fix the bug in correct way. The root cause is decimal literal's comparison only compare value, we should also compare their datatype as well as datetime literal. suppose a=0 with type decimal(2,1) b=0 with type decimal(3,1) we have: a.equals(b) returns false

[fix](nereids)should not push down constant output expr in window exp…
6d3b1ce
…ression

starocean999 marked this pull request as ready for review June 6, 2024 08:19

morrySnow added dev/2.1.x dev/2.0.x usercase Important user case type label p0_w p0_b and removed p0_w labels Jun 6, 2024

morrySnow approved these changes Jun 6, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 6, 2024

github-actionsBot added the reviewed label Jun 6, 2024

morrySnow added dev/2.0.12-merged and removed dev/2.0.x labels Jun 6, 2024

924060929 approved these changes Jun 6, 2024
View reviewed changes

morrySnow merged commit fd84a59 into apache:masterJun 6, 2024

starocean999 mentioned this pull request Jun 7, 2024
[fix](nereids)decimal and datetime literal comparison should compare datatype too #36055
Merged

starocean999 removed the dev/2.1.x label Jun 11, 2024

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

Navigation Menu

Uh oh!

[fix](nereids)should not push down constant output expr in window expression - #35948

[fix](nereids)should not push down constant output expr in window expression#35948
morrySnow merged 1 commit into
apache:masterfrom
starocean999:nereids_1026

starocean999 commented Jun 6, 2024•
edited
Loading

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

starocean999 commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

github-actionsBot commented Jun 6, 2024

Uh oh!

github-actionsBot commented Jun 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

starocean999 commented Jun 6, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

starocean999 commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

github-actionsBot commented Jun 6, 2024

Uh oh!

github-actionsBot commented Jun 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

starocean999 commented Jun 6, 2024•
edited
Loading