Skip to content

[fix](function) avoid calculating sqrt of negative in agg function CORR - #39324

Merged
BiteTheDDDDt merged 2 commits into
apache:masterfrom
zclllyybb:corr
Aug 15, 2024
Merged

[fix](function) avoid calculating sqrt of negative in agg function CORR#39324
BiteTheDDDDt merged 2 commits into
apache:masterfrom
zclllyybb:corr

Conversation

@zclllyybb

@zclllyybbzclllyybb commented Aug 13, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close #xxx

before:

mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999as DOUBLE)) |
+---------------------------------------+
| NULL |
+---------------------------------------+1 row inset (0.53 sec)

--- actually here's a silent nan by float exception. now sure in all platform.

after:

mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999as DOUBLE)) |
+---------------------------------------+
| 0 |
+---------------------------------------+1 row inset (0.13 sec)

--- is stable

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybbzclllyybb changed the title 1[fix](function) avoid calculating sqrt of negative in agg function CORRAug 13, 2024
@zclllyybb

zclllyybb commented Aug 13, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Aug 13, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 39845 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 171f057bd09d2cf626b0640a34b25980f59588f5, data reload: false
------ Round 1 ----------------------------------
q1	18132	4440	4377	4377
q2	2038	178	173	173
q3	10511	1173	1059	1059
q4	10157	667	672	667
q5	7735	2770	2760	2760
q6	223	137	135	135
q7	951	587	584	584
q8	9318	1987	2032	1987
q9	8941	6532	6557	6532
q10	7005	2211	2187	2187
q11	465	247	245	245
q12	390	217	218	217
q13	18635	2962	2949	2949
q14	267	250	229	229
q15	538	489	490	489
q16	488	407	386	386
q17	964	704	675	675
q18	8200	7485	7571	7485
q19	7582	1046	1067	1046
q20	703	323	315	315
q21	5872	4397	4336	4336
q22	1102	1045	1012	1012
Total cold run time: 120217 ms
Total hot run time: 39845 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4593	4267	4241	4241
q2	373	261	272	261
q3	2823	2583	2613	2583
q4	1900	1588	1575	1575
q5	5353	5369	5327	5327
q6	218	130	127	127
q7	2032	1659	1660	1659
q8	3161	3327	3315	3315
q9	8343	8271	8254	8254
q10	3417	3175	3154	3154
q11	597	511	479	479
q12	778	589	593	589
q13	17544	2973	2965	2965
q14	307	285	266	266
q15	536	482	471	471
q16	470	410	407	407
q17	1793	1487	1484	1484
q18	7718	7546	7489	7489
q19	1687	1572	1623	1572
q20	2066	1819	1766	1766
q21	5315	5011	5021	5011
q22	1076	1047	1009	1009
Total cold run time: 72100 ms
Total hot run time: 54004 ms

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 185304 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 171f057bd09d2cf626b0640a34b25980f59588f5, data reload: false
query1	915	378	382	378
query2	6453	2005	1863	1863
query3	6645	204	220	204
query4	33877	23197	23146	23146
query5	4177	489	490	489
query6	271	153	158	153
query7	4594	289	294	289
query8	253	208	198	198
query9	8514	2442	2423	2423
query10	437	273	266	266
query11	17740	15018	14950	14950
query12	160	104	100	100
query13	1638	367	364	364
query14	10297	7039	7614	7039
query15	222	170	170	170
query16	7663	517	489	489
query17	1594	582	546	546
query18	1954	301	294	294
query19	194	152	148	148
query20	115	112	107	107
query21	213	102	104	102
query22	4274	4097	3981	3981
query23	33848	33332	33202	33202
query24	12009	2915	2746	2746
query25	688	397	396	396
query26	1779	160	158	158
query27	2870	280	276	276
query28	7551	2059	2032	2032
query29	1054	432	422	422
query30	306	153	152	152
query31	973	739	774	739
query32	97	55	56	55
query33	750	289	291	289
query34	943	462	477	462
query35	844	702	720	702
query36	1111	934	916	916
query37	163	84	79	79
query38	3954	3879	3806	3806
query39	1437	1382	1387	1382
query40	283	121	120	120
query41	51	46	45	45
query42	116	100	100	100
query43	515	478	477	477
query44	1198	736	734	734
query45	201	163	164	163
query46	1132	721	723	721
query47	1843	1770	1772	1770
query48	363	294	292	292
query49	1194	422	433	422
query50	811	409	407	407
query51	6884	6619	6681	6619
query52	107	87	94	87
query53	262	187	178	178
query54	941	464	446	446
query55	76	82	78	78
query56	295	260	258	258
query57	1162	1083	1059	1059
query58	249	233	232	232
query59	3118	2765	2906	2765
query60	328	291	276	276
query61	118	117	220	117
query62	828	633	670	633
query63	218	181	179	179
query64	10631	2303	1764	1764
query65	3193	3120	3137	3120
query66	1362	332	344	332
query67	15591	14998	14851	14851
query68	7943	564	566	564
query69	679	400	315	315
query70	1194	1087	1048	1048
query71	535	265	270	265
query72	8145	2268	2008	2008
query73	805	323	326	323
query74	9127	8834	8807	8807
query75	5116	2755	2722	2722
query76	5122	1017	959	959
query77	804	308	304	304
query78	9744	9137	8986	8986
query79	8074	526	528	526
query80	1006	495	492	492
query81	576	230	228	228
query82	841	136	134	134
query83	329	142	145	142
query84	269	77	76	76
query85	1396	321	277	277
query86	449	298	296	296
query87	4514	4180	4193	4180
query88	5217	2407	2428	2407
query89	532	286	289	286
query90	1949	195	191	191
query91	122	94	95	94
query92	66	50	54	50
query93	6008	539	539	539
query94	874	285	293	285
query95	367	257	259	257
query96	606	274	272	272
query97	3195	3067	3039	3039
query98	222	203	202	202
query99	1539	1273	1272	1272
Total cold run time: 323012 ms
Total hot run time: 185304 ms

@zclllyybb

zclllyybb commented Aug 13, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Aug 13, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 39712 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7ff30f07090981528520ee14f0e3e0193c9e5958, data reload: false
------ Round 1 ----------------------------------
q1	18478	6668	4284	4284
q2	2023	192	183	183
q3	10576	1147	1130	1130
q4	10275	666	663	663
q5	7745	2852	2744	2744
q6	227	138	137	137
q7	952	593	585	585
q8	9324	2063	2016	2016
q9	8619	6539	6517	6517
q10	7018	2147	2258	2147
q11	457	254	267	254
q12	403	222	219	219
q13	17759	2995	3012	2995
q14	278	239	235	235
q15	528	481	492	481
q16	510	403	390	390
q17	981	721	767	721
q18	8240	7587	7398	7398
q19	5668	1058	969	969
q20	668	339	322	322
q21	6334	4320	4605	4320
q22	1110	1021	1002	1002
Total cold run time: 118173 ms
Total hot run time: 39712 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4548	4296	4241	4241
q2	364	276	270	270
q3	2819	2663	2629	2629
q4	1852	1596	1576	1576
q5	5381	5368	5362	5362
q6	218	127	130	127
q7	2020	1654	1670	1654
q8	3175	3345	3323	3323
q9	8359	8376	8370	8370
q10	3400	3166	3171	3166
q11	586	501	496	496
q12	763	641	600	600
q13	16342	3013	3005	3005
q14	313	283	291	283
q15	518	479	479	479
q16	467	420	420	420
q17	1757	1507	1483	1483
q18	7581	7578	7539	7539
q19	1688	1567	1585	1567
q20	2004	1804	1770	1770
q21	5268	5090	5041	5041
q22	1105	1019	985	985
Total cold run time: 70528 ms
Total hot run time: 54386 ms

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 184825 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7ff30f07090981528520ee14f0e3e0193c9e5958, data reload: false
query1	921	372	372	372
query2	6454	1976	1871	1871
query3	6648	223	210	210
query4	34200	23237	22980	22980
query5	4201	496	476	476
query6	264	154	158	154
query7	4592	287	285	285
query8	256	201	214	201
query9	8517	2450	2429	2429
query10	427	278	262	262
query11	17813	15015	14908	14908
query12	149	100	98	98
query13	1627	366	361	361
query14	9948	7616	6449	6449
query15	212	164	167	164
query16	7763	483	456	456
query17	1573	581	557	557
query18	1991	291	286	286
query19	191	142	144	142
query20	113	105	117	105
query21	208	105	105	105
query22	4262	3996	3983	3983
query23	33978	33177	33284	33177
query24	12039	2853	2825	2825
query25	704	404	394	394
query26	1817	158	160	158
query27	2890	271	278	271
query28	7573	2041	2040	2040
query29	1141	426	434	426
query30	317	154	159	154
query31	1019	749	765	749
query32	99	54	54	54
query33	759	291	285	285
query34	945	462	469	462
query35	835	742	727	727
query36	1066	937	935	935
query37	178	81	83	81
query38	4001	3823	3815	3815
query39	1438	1378	1439	1378
query40	271	120	116	116
query41	50	46	46	46
query42	118	93	99	93
query43	502	463	473	463
query44	1236	735	735	735
query45	195	164	170	164
query46	1120	721	770	721
query47	1841	1783	1750	1750
query48	367	298	294	294
query49	1199	433	425	425
query50	811	415	408	408
query51	6807	6753	6742	6742
query52	99	102	94	94
query53	256	182	181	181
query54	957	464	449	449
query55	80	79	75	75
query56	273	254	253	253
query57	1140	1052	1103	1052
query58	252	252	250	250
query59	2985	2896	2832	2832
query60	295	270	274	270
query61	116	209	92	92
query62	833	640	654	640
query63	214	179	179	179
query64	10531	2275	1733	1733
query65	3189	3141	3157	3141
query66	1383	335	327	327
query67	15699	14872	14803	14803
query68	8901	558	567	558
query69	702	392	304	304
query70	1343	1123	1146	1123
query71	547	278	276	276
query72	7979	2238	2039	2039
query73	2219	335	324	324
query74	9177	8794	8770	8770
query75	5156	2707	2699	2699
query76	5407	1006	1020	1006
query77	759	293	304	293
query78	10538	9204	9052	9052
query79	9646	523	524	523
query80	1099	497	497	497
query81	597	224	225	224
query82	542	142	133	133
query83	327	145	147	145
query84	273	78	79	78
query85	1304	263	259	259
query86	418	306	294	294
query87	4513	4315	4250	4250
query88	4745	2405	2455	2405
query89	500	286	303	286
query90	2098	207	200	200
query91	131	96	92	92
query92	67	50	48	48
query93	6262	546	542	542
query94	1055	296	279	279
query95	366	256	260	256
query96	617	278	271	271
query97	3178	3026	3043	3026
query98	230	199	207	199
query99	1604	1270	1261	1261
Total cold run time: 328248 ms
Total hot run time: 184825 ms

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.98 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7ff30f07090981528520ee14f0e3e0193c9e5958, data reload: false
query1	0.04	0.05	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.08	0.07
query5	0.51	0.50	0.50
query6	1.14	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.54	0.53	0.56
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.61	0.62	0.58
query14	0.76	0.81	0.77
query15	0.84	0.82	0.82
query16	0.37	0.35	0.36
query17	0.96	0.99	1.00
query18	0.23	0.22	0.23
query19	1.84	1.69	1.67
query20	0.01	0.02	0.01
query21	15.39	0.74	0.64
query22	3.72	7.39	2.29
query23	18.27	1.38	1.29
query24	2.13	0.23	0.22
query25	0.16	0.08	0.09
query26	0.31	0.21	0.21
query27	0.46	0.23	0.22
query28	13.26	1.01	1.01
query29	12.66	3.24	3.29
query30	0.24	0.04	0.04
query31	2.90	0.41	0.40
query32	3.25	0.49	0.48
query33	2.97	2.90	2.94
query34	17.15	4.32	4.35
query35	4.46	4.40	4.40
query36	0.65	0.47	0.47
query37	0.18	0.16	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.13	0.12
query41	0.09	0.06	0.06
query42	0.06	0.05	0.05
query43	0.05	0.05	0.05
Total cold run time: 109.49 s
Total hot run time: 30.98 s

@zclllyybb

zclllyybb commented Aug 13, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Aug 13, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 40249 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55c9d667984aafd5a7fea39805d791716cab45f4, data reload: false
------ Round 1 ----------------------------------
q1	17612	4355	4282	4282
q2	2021	177	171	171
q3	10591	1286	1104	1104
q4	11508	785	752	752
q5	7864	2893	2812	2812
q6	229	145	145	145
q7	999	625	618	618
q8	9554	2048	2057	2048
q9	8710	6501	6555	6501
q10	7081	2187	2191	2187
q11	485	255	264	255
q12	409	231	227	227
q13	18753	2996	3007	2996
q14	284	235	232	232
q15	526	489	493	489
q16	533	393	394	393
q17	987	671	669	669
q18	8167	7492	7467	7467
q19	7179	992	972	972
q20	718	334	318	318
q21	5294	4676	4615	4615
q22	1125	1041	996	996
Total cold run time: 120629 ms
Total hot run time: 40249 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4650	4256	4277	4256
q2	373	267	272	267
q3	2864	2592	2628	2592
q4	1914	1636	1588	1588
q5	5362	5385	5341	5341
q6	222	133	132	132
q7	2033	1694	1662	1662
q8	3166	3322	3316	3316
q9	8449	8392	8431	8392
q10	3375	3116	3170	3116
q11	598	539	492	492
q12	776	640	622	622
q13	17463	3006	3009	3006
q14	307	279	278	278
q15	518	476	488	476
q16	492	410	424	410
q17	1802	1477	1510	1477
q18	7708	7535	7341	7341
q19	1691	1422	1542	1422
q20	2011	1804	1804	1804
q21	5057	5085	5137	5085
q22	1109	986	999	986
Total cold run time: 71940 ms
Total hot run time: 54061 ms

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 185835 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55c9d667984aafd5a7fea39805d791716cab45f4, data reload: false
query1	925	378	374	374
query2	6456	1911	1905	1905
query3	6642	205	217	205
query4	33635	23266	23037	23037
query5	4220	482	528	482
query6	271	185	160	160
query7	4593	290	297	290
query8	240	190	190	190
query9	8690	2464	2418	2418
query10	435	262	258	258
query11	17709	14850	15070	14850
query12	156	99	105	99
query13	1627	371	362	362
query14	10222	7560	7762	7560
query15	222	172	166	166
query16	7697	485	485	485
query17	1556	549	564	549
query18	1981	285	279	279
query19	193	135	140	135
query20	111	104	102	102
query21	209	104	96	96
query22	4234	3980	4014	3980
query23	33952	33130	33087	33087
query24	12171	2889	2881	2881
query25	675	379	375	375
query26	1847	155	154	154
query27	2871	276	268	268
query28	7721	2044	2018	2018
query29	1095	409	431	409
query30	311	168	152	152
query31	981	752	751	751
query32	95	52	53	52
query33	758	284	294	284
query34	938	460	468	460
query35	851	711	711	711
query36	1083	928	926	926
query37	277	82	80	80
query38	3993	3803	3919	3803
query39	1426	1391	1382	1382
query40	274	118	112	112
query41	47	45	40	40
query42	113	93	96	93
query43	487	490	463	463
query44	1254	737	732	732
query45	196	161	167	161
query46	1118	739	751	739
query47	1837	1792	1765	1765
query48	360	300	289	289
query49	1219	425	410	410
query50	802	412	410	410
query51	6814	6810	6772	6772
query52	97	86	96	86
query53	258	185	176	176
query54	985	447	444	444
query55	79	74	73	73
query56	269	249	253	249
query57	1199	1039	1024	1024
query58	233	222	233	222
query59	2979	2677	2771	2677
query60	302	261	318	261
query61	97	94	94	94
query62	854	649	660	649
query63	218	182	179	179
query64	10618	2287	1748	1748
query65	3526	3137	3153	3137
query66	1408	334	342	334
query67	15118	15037	14816	14816
query68	4608	555	560	555
query69	422	270	283	270
query70	1159	1167	1114	1114
query71	406	279	268	268
query72	7069	2346	2031	2031
query73	766	326	331	326
query74	9221	8702	8758	8702
query75	3458	2690	2649	2649
query76	2789	1001	960	960
query77	452	307	321	307
query78	10092	9513	10728	9513
query79	3093	524	521	521
query80	1014	499	516	499
query81	581	227	224	224
query82	731	140	132	132
query83	237	152	161	152
query84	302	81	81	81
query85	1629	271	274	271
query86	520	308	322	308
query87	4385	4134	4119	4119
query88	3715	2416	2416	2416
query89	394	290	289	289
query90	1793	199	197	197
query91	127	96	96	96
query92	68	49	51	49
query93	1499	544	541	541
query94	900	290	286	286
query95	363	265	259	259
query96	599	282	278	278
query97	3228	3043	3067	3043
query98	220	203	204	203
query99	1612	1257	1246	1246
Total cold run time: 303282 ms
Total hot run time: 185835 ms

@doris-robot

doris-robot commented Aug 13, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.51 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 55c9d667984aafd5a7fea39805d791716cab45f4, data reload: false
query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.08	0.08
query5	0.51	0.50	0.48
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.47	0.48
query10	0.53	0.55	0.54
query11	0.16	0.12	0.11
query12	0.15	0.13	0.12
query13	0.59	0.60	0.58
query14	0.76	0.78	0.79
query15	0.88	0.83	0.81
query16	0.36	0.38	0.37
query17	1.06	1.05	0.94
query18	0.23	0.22	0.22
query19	1.90	1.80	1.75
query20	0.02	0.01	0.01
query21	15.40	0.72	0.65
query22	4.45	7.90	1.73
query23	18.27	1.41	1.30
query24	2.09	0.24	0.22
query25	0.16	0.10	0.08
query26	0.29	0.22	0.21
query27	0.46	0.23	0.23
query28	13.31	1.01	0.99
query29	12.58	3.27	3.23
query30	0.24	0.05	0.05
query31	2.89	0.40	0.39
query32	3.26	0.49	0.48
query33	2.92	2.96	2.95
query34	17.01	4.45	4.35
query35	4.40	4.42	4.42
query36	0.66	0.47	0.47
query37	0.18	0.15	0.16
query38	0.15	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.14	0.13
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 110.04 s
Total hot run time: 30.51 s

HappenLee
HappenLee previously approved these changes Aug 14, 2024
View reviewed changes

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented Aug 14, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 14, 2024
@github-actions

github-actionsBot commented Aug 14, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@zclllyybb

zclllyybb commented Aug 14, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actionsgithub-actionsBot removed the approved Indicates a PR has been approved by one committer. label Aug 14, 2024
@doris-robot

doris-robot commented Aug 14, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 37311 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8890c61865bd276564e4998f6b5a57f969b99f21, data reload: false
------ Round 1 ----------------------------------
q1	17603	4343	4307	4307
q2	2020	178	180	178
q3	11885	1262	1102	1102
q4	10534	738	757	738
q5	7778	2796	2789	2789
q6	220	138	139	138
q7	946	593	606	593
q8	9539	2044	2086	2044
q9	8681	6521	6514	6514
q10	7021	2114	2256	2114
q11	445	244	236	236
q12	392	219	214	214
q13	18941	2986	2950	2950
q14	295	245	238	238
q15	514	481	467	467
q16	499	393	388	388
q17	961	648	697	648
q18	7265	6712	6754	6712
q19	5979	972	956	956
q20	685	333	347	333
q21	3901	2633	2789	2633
q22	1126	1019	1021	1019
Total cold run time: 117230 ms
Total hot run time: 37311 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4448	4317	4239	4239
q2	360	264	258	258
q3	2847	2655	2593	2593
q4	1924	1665	1666	1665
q5	5694	5723	5654	5654
q6	231	135	133	133
q7	2158	1726	1740	1726
q8	3306	3446	3459	3446
q9	8757	8742	8822	8742
q10	3602	3218	3272	3218
q11	606	523	527	523
q12	777	612	588	588
q13	17140	3191	3158	3158
q14	332	287	303	287
q15	532	493	505	493
q16	492	463	448	448
q17	1834	1516	1536	1516
q18	8100	7968	7803	7803
q19	1792	1653	1570	1570
q20	2222	1911	1883	1883
q21	8845	5383	5272	5272
q22	1123	1023	1022	1022
Total cold run time: 77122 ms
Total hot run time: 56237 ms

@doris-robot

doris-robot commented Aug 14, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 189731 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8890c61865bd276564e4998f6b5a57f969b99f21, data reload: false
query1	1256	893	855	855
query2	6446	1899	1850	1850
query3	10626	3819	3797	3797
query4	58611	23889	23289	23289
query5	5944	490	503	490
query6	495	173	175	173
query7	6486	305	288	288
query8	318	206	204	204
query9	9076	2453	2434	2434
query10	498	279	254	254
query11	15842	14996	14949	14949
query12	161	104	93	93
query13	1607	384	371	371
query14	11690	7124	7092	7092
query15	236	166	173	166
query16	7722	510	508	508
query17	1183	594	588	588
query18	2092	316	306	306
query19	286	164	160	160
query20	122	117	113	113
query21	208	116	106	106
query22	4612	4317	4263	4263
query23	34560	33504	33111	33111
query24	5616	2876	2943	2876
query25	552	396	409	396
query26	703	157	158	157
query27	1804	271	275	271
query28	3694	2058	2046	2046
query29	724	420	423	420
query30	227	161	158	158
query31	976	750	748	748
query32	84	55	64	55
query33	515	296	283	283
query34	852	456	465	456
query35	841	710	696	696
query36	1056	906	952	906
query37	138	81	83	81
query38	4054	3847	3842	3842
query39	1461	1407	1395	1395
query40	199	116	115	115
query41	44	45	42	42
query42	115	97	94	94
query43	509	466	469	466
query44	1079	739	740	739
query45	199	168	164	164
query46	1093	714	732	714
query47	1857	1774	1796	1774
query48	368	293	290	290
query49	770	422	429	422
query50	801	404	408	404
query51	6778	6683	6735	6683
query52	101	93	96	93
query53	257	187	180	180
query54	575	457	555	457
query55	77	72	72	72
query56	261	238	245	238
query57	1141	1087	1051	1051
query58	220	227	293	227
query59	2904	2667	2766	2667
query60	298	261	268	261
query61	97	93	95	93
query62	770	635	642	635
query63	222	183	178	178
query64	4091	2211	1751	1751
query65	3212	3167	3145	3145
query66	715	332	328	328
query67	15648	15070	14828	14828
query68	8379	563	547	547
query69	701	387	300	300
query70	1245	1130	1140	1130
query71	544	277	271	271
query72	7662	2257	2006	2006
query73	2098	325	321	321
query74	9098	8679	8898	8679
query75	4941	2659	2688	2659
query76	4941	1019	1075	1019
query77	751	305	303	303
query78	9698	9222	8931	8931
query79	10020	535	541	535
query80	1155	497	489	489
query81	580	226	223	223
query82	761	137	133	133
query83	368	146	144	144
query84	265	78	75	75
query85	948	317	276	276
query86	359	311	293	293
query87	4345	4212	4206	4206
query88	4351	2293	2292	2292
query89	506	294	290	290
query90	2285	204	199	199
query91	123	98	92	92
query92	63	50	52	50
query93	5352	548	548	548
query94	1150	307	316	307
query95	353	262	259	259
query96	622	267	270	267
query97	3230	3060	3068	3060
query98	236	204	200	200
query99	1681	1274	1270	1270
Total cold run time: 336586 ms
Total hot run time: 189731 ms

@doris-robot

doris-robot commented Aug 14, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.32 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8890c61865bd276564e4998f6b5a57f969b99f21, data reload: false
query1	0.04	0.04	0.04
query2	0.08	0.04	0.03
query3	0.22	0.04	0.05
query4	1.70	0.08	0.07
query5	0.50	0.50	0.49
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.48	0.49
query10	0.54	0.54	0.55
query11	0.16	0.12	0.12
query12	0.15	0.12	0.13
query13	0.61	0.62	0.59
query14	0.76	0.79	0.77
query15	0.86	0.82	0.82
query16	0.37	0.37	0.36
query17	0.95	1.01	0.98
query18	0.23	0.23	0.22
query19	1.81	1.70	1.75
query20	0.01	0.01	0.01
query21	15.38	0.76	0.66
query22	4.18	6.52	2.58
query23	18.28	1.46	1.23
query24	1.85	0.26	0.22
query25	0.15	0.09	0.08
query26	0.28	0.22	0.21
query27	0.46	0.23	0.22
query28	13.34	1.02	1.00
query29	12.58	3.38	3.33
query30	0.24	0.05	0.05
query31	2.89	0.40	0.39
query32	3.26	0.49	0.48
query33	2.97	2.97	2.98
query34	17.07	4.40	4.35
query35	4.41	4.37	4.42
query36	0.67	0.49	0.46
query37	0.18	0.16	0.16
query38	0.16	0.14	0.15
query39	0.04	0.04	0.03
query40	0.15	0.12	0.12
query41	0.10	0.04	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.49 s
Total hot run time: 31.32 s

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 15, 2024
@github-actions

github-actionsBot commented Aug 15, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@BiteTheDDDDt
BiteTheDDDDt merged commit 76a887b into apache:masterAug 15, 2024
dataroaring pushed a commit that referenced this pull request Aug 17, 2024
@zclllyybb@dataroaring
[fix](function) avoid calculating sqrt of negative in agg function CO…
3ca955c
…RR (#39324)
## Proposed changes
Issue Number: close #xxx
before:
```sql
mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999 as DOUBLE)) |
+---------------------------------------+
| NULL |
+---------------------------------------+
1 row in set (0.53 sec)
```
--- actually here's a silent nan by float exception. now sure in all
platform.
after:
```sql
mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999 as DOUBLE)) |
+---------------------------------------+
| 0 |
+---------------------------------------+
1 row in set (0.13 sec)
```
--- is stable
@zclllyybb
zclllyybb deleted the corr branch August 23, 2024 10:01
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Aug 23, 2024
@zclllyybb
[fix](function) avoid calculating sqrt of negative in agg function CO…
1df5a9a
…RR (apache#39324)
Issue Number: close #xxx
before:
```sql
mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999 as DOUBLE)) |
+---------------------------------------+
| NULL |
+---------------------------------------+
1 row in set (0.53 sec)
```
--- actually here's a silent nan by float exception. now sure in all
platform.
after:
```sql
mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999 as DOUBLE)) |
+---------------------------------------+
| 0 |
+---------------------------------------+
1 row in set (0.13 sec)
```
--- is stable
@zclllyybbzclllyybb mentioned this pull request Aug 23, 2024
Merged
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Aug 23, 2024
@zclllyybb
[fix](function) avoid calculating sqrt of negative in agg function CO…
1177d27
…RR (apache#39324)
Issue Number: close #xxx
before:
```sql
mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999 as DOUBLE)) |
+---------------------------------------+
| NULL |
+---------------------------------------+
1 row in set (0.53 sec)
```
--- actually here's a silent nan by float exception. now sure in all
platform.
after:
```sql
mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999 as DOUBLE)) |
+---------------------------------------+
| 0 |
+---------------------------------------+
1 row in set (0.13 sec)
```
--- is stable
yiguolei pushed a commit that referenced this pull request Aug 24, 2024
@zclllyybb
[branch-2.1] pick some prs (#39860)
460605a
## Proposed changes
Issue Number: close #xxx
#38385 optimize parsing datetime
#38978 make stream load failure
message more clear and disable some error's stacktrace by default
#39255 fix random function coredump
#39324 fix function corr
inconsistency with doc
#39449 check auto partitoin nullity
when creating partition
#39695 make
DynamicPartitionScheduler immediately know interval's change
#39754 Add some partition expr check
on creating table
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@zclllyybb@HappenLee
[fix](function) avoid calculating sqrt of negative in agg function CO…
5b08a63
…RR (apache#39324)
## Proposed changes
Issue Number: close #xxx
before:
```sql
mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999 as DOUBLE)) |
+---------------------------------------+
| NULL |
+---------------------------------------+
1 row in set (0.53 sec)
```
--- actually here's a silent nan by float exception. now sure in all
platform.
after:
```sql
mysql [sqlfunctest]>SELECT CORR(data, 89.999999) FROM DOUBLEDATA_NOT_EMPTY_NULLABLE;
+---------------------------------------+
| corr(data, cast(89.999999 as DOUBLE)) |
+---------------------------------------+
| 0 |
+---------------------------------------+
1 row in set (0.13 sec)
```
--- is stable
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDtBiteTheDDDDtBiteTheDDDDt approved these changes
@MryangeMryangeMryange approved these changes
@HappenLeeHappenLeeHappenLee left review comments

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.6-mergeddev/3.0.2-mergeddoingreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zclllyybb@doris-robot@BiteTheDDDDt@HappenLee@Mryange@yiguolei@dataroaring