[fix](Nereids) create double literal when create decimal literal failed by morrySnow · Pull Request #28959 · apache/doris

morrySnow · 2023-12-25T06:12:56Z

Proposed changes

remove float and double literal toString and getStringValue introduced by
PR [Bug](materialized-view) fix divide double can not match mv #23504 and PR [feature](nereids)support subquery in select list #23271
These functions lead to wrong cast result of double and float literal
fix compute signature for datetimev2 always produce scale 6
fix stats calculator failed when generate node stats with two same column
constant fold on fe failed when cast double to integral

TODO
after fix the first problem, some mv matching not work well

test_dup_mv_div
test_dup_mv_json
test_tcu

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

morrySnow · 2023-12-25T06:13:14Z

run buildall

doris-robot · 2023-12-25T06:46:27Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 48c2e8d2b7589b3fb97860f9eae1d0d12a334a56, data reload: false
run tpch-sf100 query with default conf and session variables
q1	4680	4450	4396	4396
q2	377	177	159	159
q3	1450	1246	1246	1246
q4	1118	952	892	892
q5	3185	3177	3173	3173
q6	250	130	128	128
q7	1024	494	483	483
q8	2228	2243	2198	2198
q9	6708	6732	6698	6698
q10	3223	3289	3253	3253
q11	311	185	182	182
q12	354	207	207	207
q13	4568	3784	3804	3784
q14	244	211	215	211
q15	581	528	528	528
q16	434	383	387	383
q17	1024	587	498	498
q18	7005	6988	6785	6785
q19	1552	1421	1476	1421
q20	582	798	294	294
q21	3080	2661	2635	2635
q22	354	282	281	281
Total cold run time: 44332 ms
Total hot run time: 39835 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4334	4368	4346	4346
q2	271	170	168	168
q3	3508	3503	3502	3502
q4	2402	2380	2381	2380
q5	5719	5715	5715	5715
q6	242	125	125	125
q7	2372	1896	1873	1873
q8	3526	3541	3524	3524
q9	9020	8961	9046	8961
q10	3896	4009	3995	3995
q11	481	372	370	370
q12	768	632	621	621
q13	4290	3569	3588	3569
q14	286	253	264	253
q15	567	522	521	521
q16	503	463	452	452
q17	1902	1860	1854	1854
q18	8573	7918	8056	7918
q19	1753	1756	1777	1756
q20	2256	1944	1941	1941
q21	6514	6170	6165	6165
q22	504	432	440	432
Total cold run time: 63687 ms
Total hot run time: 60441 ms

doris-robot · 2023-12-25T06:59:03Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.81 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17183769042 Bytes

morrySnow · 2023-12-26T03:33:16Z

run buildall

doris-robot · 2023-12-26T04:18:47Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.37 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 37.1 seconds inserted 10000000 Rows, about 269K ops/s
storage size: 17183756501 Bytes

doris-robot · 2023-12-26T05:38:04Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 48c2e8d2b7589b3fb97860f9eae1d0d12a334a56, data reload: false
run tpch-sf100 query with default conf and session variables
q1	4783	4485	4458	4458
q2	367	156	150	150
q3	1472	1322	1191	1191
q4	1155	1001	916	916
q5	3201	3190	3174	3174
q6	262	137	134	134
q7	1040	499	505	499
q8	2313	2252	2251	2251
q9	6710	6719	6714	6714
q10	3210	3280	3294	3280
q11	338	218	217	217
q12	365	213	216	213
q13	4573	3831	3815	3815
q14	252	215	214	214
q15	580	529	527	527
q16	448	386	389	386
q17	1054	797	607	607
q18	7115	6764	6850	6764
q19	1583	1591	1589	1589
q20	573	318	298	298
q21	3243	2787	2775	2775
q22	378	303	309	303
Total cold run time: 45015 ms
Total hot run time: 40475 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4411	4391	4421	4391
q2	276	170	174	170
q3	3531	3522	3510	3510
q4	2438	2420	2411	2411
q5	5735	5737	5726	5726
q6	248	126	126	126
q7	2365	1876	1856	1856
q8	3627	3622	3606	3606
q9	8995	9011	9003	9003
q10	3931	4006	4027	4006
q11	503	367	380	367
q12	780	603	620	603
q13	4329	3560	3562	3560
q14	282	263	248	248
q15	575	532	522	522
q16	510	454	468	454
q17	1984	1957	1973	1957
q18	8734	8172	8315	8172
q19	1834	1821	1840	1821
q20	2277	1960	1950	1950
q21	6616	6286	6240	6240
q22	563	465	474	465
Total cold run time: 64544 ms
Total hot run time: 61164 ms

morrySnow · 2023-12-26T08:48:19Z

run buildall

doris-robot · 2023-12-26T09:12:56Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 2b95457e447ff6d05d976c916e6245d2ffab8b11, data reload: false
run tpch-sf100 query with default conf and session variables
q1	4826	4493	4478	4478
q2	375	140	158	140
q3	1469	1330	1260	1260
q4	1153	1003	961	961
q5	3183	3160	3191	3160
q6	266	132	136	132
q7	1053	483	500	483
q8	2260	2272	2240	2240
q9	6726	6687	6700	6687
q10	3187	3308	3295	3295
q11	327	201	218	201
q12	359	212	213	212
q13	4557	3832	3799	3799
q14	248	217	216	216
q15	572	522	520	520
q16	452	386	398	386
q17	1033	768	605	605
q18	7147	6839	7649	6839
q19	1582	1588	1590	1588
q20	1057	307	316	307
q21	3263	2791	2765	2765
q22	364	305	303	303
Total cold run time: 45459 ms
Total hot run time: 40577 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4412	4429	4424	4424
q2	272	171	179	171
q3	3532	3526	3510	3510
q4	2450	2416	2428	2416
q5	5746	5728	5756	5728
q6	249	127	127	127
q7	2380	1850	1870	1850
q8	3626	3629	3644	3629
q9	9088	8994	8979	8979
q10	3915	4037	4036	4036
q11	495	384	352	352
q12	790	613	602	602
q13	4280	3604	3591	3591
q14	291	248	249	248
q15	574	523	518	518
q16	497	467	450	450
q17	1995	1963	1973	1963
q18	8732	8122	8275	8122
q19	1834	1813	1821	1813
q20	2284	1950	1947	1947
q21	6660	6278	6248	6248
q22	553	469	472	469
Total cold run time: 64655 ms
Total hot run time: 61193 ms

doris-robot · 2023-12-26T09:25:20Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.57 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 37.6 seconds inserted 10000000 Rows, about 265K ops/s
storage size: 17183958261 Bytes

morrySnow · 2023-12-28T03:44:10Z

run buildall

doris-robot · 2023-12-28T05:15:40Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.47 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17183707196 Bytes

doris-robot · 2023-12-28T05:34:49Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 1614960e7d467b18df4687ef7adfe50318019ff5, data reload: false
run tpch-sf100 query with default conf and session variables
q1	5002	4661	4669	4661
q2	362	153	159	153
q3	1468	1346	1257	1257
q4	1130	933	875	875
q5	3174	3165	3168	3165
q6	253	133	128	128
q7	1036	489	522	489
q8	2261	2264	2238	2238
q9	6713	6673	6693	6673
q10	3200	3276	3268	3268
q11	334	216	213	213
q12	351	206	208	206
q13	4188	3428	3433	3428
q14	241	211	212	211
q15	573	518	530	518
q16	450	387	383	383
q17	1041	784	549	549
q18	7132	6771	6908	6771
q19	1618	1629	1665	1629
q20	602	319	314	314
q21	3196	2723	2715	2715
q22	377	300	307	300
Total cold run time: 44702 ms
Total hot run time: 40144 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4575	4616	4570	4570
q2	270	165	165	165
q3	3386	3364	3357	3357
q4	2232	2201	2214	2201
q5	5729	5715	5749	5715
q6	242	118	120	118
q7	2385	1837	1862	1837
q8	3614	3618	3612	3612
q9	9038	9031	9012	9012
q10	3798	3880	3902	3880
q11	484	368	380	368
q12	774	589	592	589
q13	3906	3167	3163	3163
q14	283	250	249	249
q15	580	517	511	511
q16	499	462	459	459
q17	1972	1971	1960	1960
q18	8770	8327	8339	8327
q19	1744	1750	1770	1750
q20	2256	1954	1940	1940
q21	6138	5755	5742	5742
q22	551	448	470	448
Total cold run time: 63226 ms
Total hot run time: 59973 ms

morrySnow · 2023-12-28T06:23:05Z

run buildall

doris-robot · 2023-12-28T07:02:14Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 3f7492fb7d020842a34cc3304715b5b94085a2db, data reload: false
run tpch-sf100 query with default conf and session variables
q1	4998	4629	4629	4629
q2	370	151	160	151
q3	1457	1289	1248	1248
q4	1124	952	918	918
q5	3195	3162	3194	3162
q6	250	125	128	125
q7	1063	497	504	497
q8	2263	2258	2262	2258
q9	6749	6699	6717	6699
q10	3206	3252	3271	3252
q11	338	226	213	213
q12	356	208	212	208
q13	4153	3397	3447	3397
q14	240	215	208	208
q15	574	523	524	523
q16	438	382	390	382
q17	1050	762	557	557
q18	7075	6920	6844	6844
q19	1627	1633	1641	1633
q20	538	326	305	305
q21	3209	2719	2697	2697
q22	372	314	321	314
Total cold run time: 44645 ms
Total hot run time: 40220 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4570	4573	4573	4573
q2	271	164	176	164
q3	3378	3371	3362	3362
q4	2221	2202	2203	2202
q5	5716	5715	5720	5715
q6	238	119	118	118
q7	2379	1886	1881	1881
q8	3610	3615	3607	3607
q9	9008	8962	8981	8962
q10	3796	3886	3895	3886
q11	499	372	371	371
q12	763	608	630	608
q13	3907	3187	3155	3155
q14	288	262	268	262
q15	564	519	519	519
q16	498	436	454	436
q17	1963	1973	1948	1948
q18	8751	8145	8221	8145
q19	1760	1768	1749	1749
q20	2260	1930	1922	1922
q21	6153	5760	5783	5760
q22	539	456	461	456
Total cold run time: 63132 ms
Total hot run time: 59801 ms

doris-robot · 2023-12-28T07:15:07Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.93 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17183981707 Bytes

morrySnow · 2023-12-28T12:22:30Z

run buildall

doris-robot · 2023-12-28T13:06:39Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.8 seconds
stream load tsv: 568 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.4 seconds inserted 10000000 Rows, about 340K ops/s
storage size: 17183831242 Bytes

doris-robot · 2023-12-28T13:46:58Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 86fe51a532b789c144c51773462a371bb2d56e55, data reload: false
run tpch-sf100 query with default conf and session variables
q1	4994	4631	4645	4631
q2	359	159	156	156
q3	1472	1220	1177	1177
q4	1129	949	873	873
q5	3176	3161	3194	3161
q6	251	129	125	125
q7	1060	488	495	488
q8	2277	2295	2249	2249
q9	6738	6714	6724	6714
q10	3171	3287	3243	3243
q11	334	217	202	202
q12	351	210	209	209
q13	4166	3378	3407	3378
q14	248	217	214	214
q15	577	523	529	523
q16	448	386	381	381
q17	1051	786	631	631
q18	7108	6740	6805	6740
q19	1634	1638	1634	1634
q20	537	325	296	296
q21	3134	2753	2733	2733
q22	367	290	304	290
Total cold run time: 44582 ms
Total hot run time: 40048 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4569	4581	4581	4581
q2	275	173	169	169
q3	3383	3373	3376	3373
q4	2232	2211	2203	2203
q5	5733	5716	5745	5716
q6	240	119	118	118
q7	2360	1876	1847	1847
q8	3604	3629	3621	3621
q9	9093	9059	9038	9038
q10	3795	3887	3908	3887
q11	498	365	370	365
q12	764	608	605	605
q13	3913	3190	3152	3152
q14	284	252	257	252
q15	567	515	510	510
q16	496	450	450	450
q17	1972	1955	1948	1948
q18	8664	8282	8342	8282
q19	1748	1752	1754	1752
q20	2312	1925	1919	1919
q21	6150	5741	5736	5736
q22	538	448	465	448
Total cold run time: 63190 ms
Total hot run time: 59972 ms

morrySnow · 2023-12-29T06:20:16Z

run buildall

github-actions · 2023-12-29T06:27:24Z

clang-tidy review says "All clean, LGTM! 👍"

morrySnow · 2024-01-05T07:18:33Z

run buildall

doris-robot · 2024-01-05T07:37:51Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit dba3fd83ad6b4c78173302af3d7eaa132f73ad54, data reload: false
------ Round 1 ----------------------------------
q1	18132	5279	5156	5156
q2	2023	156	148	148
q3	10613	1094	1118	1094
q4	10231	751	864	751
q5	7793	2942	2958	2942
q6	213	132	136	132
q7	928	517	551	517
q8	9279	2029	2030	2029
q9	6881	6473	6406	6406
q10	8242	3025	3117	3025
q11	430	230	208	208
q12	391	241	240	240
q13	18017	3609	3624	3609
q14	238	206	212	206
q15	560	515	516	515
q16	464	420	396	396
q17	973	447	531	447
q18	7501	6764	6751	6751
q19	1592	1353	1332	1332
q20	716	357	348	348
q21	2825	2361	2411	2361
q22	374	331	320	320
Total cold run time: 108416 ms
Total hot run time: 38933 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5134	5175	5131	5131
q2	335	240	243	240
q3	3299	3302	3297	3297
q4	2134	2027	2070	2027
q5	5776	5803	5775	5775
q6	219	130	131	130
q7	2319	1929	1918	1918
q8	3369	3456	3462	3456
q9	8808	8759	8728	8728
q10	3797	3852	3861	3852
q11	566	473	483	473
q12	799	641	640	640
q13	7401	3159	3211	3159
q14	288	284	273	273
q15	580	514	514	514
q16	575	491	492	491
q17	1962	1740	1745	1740
q18	8663	8407	8334	8334
q19	1637	1592	1597	1592
q20	2205	1982	1957	1957
q21	5627	5285	5279	5279
q22	528	488	487	487
Total cold run time: 66021 ms
Total hot run time: 59493 ms

doris-robot · 2024-01-08T05:22:13Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.39 seconds
stream load tsv: 580 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.9 seconds inserted 10000000 Rows, about 358K ops/s
storage size: 17184184917 Bytes

remove float and double literal toString and getStringValue introduced by PR apache#23504 and PR apache#23271 These functions lead to wrong cast result of double and float literal fix string cast to datetimev2 error introduced by PR apache#26827 we should cast to exactly scale of datetimev2 when cast string to it

morrySnow · 2024-01-08T07:14:12Z

run buildall

doris-robot · 2024-01-08T07:14:25Z

TPC-H: Total hot run time: 38271 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 082e2a1440bfce6dddfc54c0cbf3af1fbe8f2055, data reload: false
------ Round 1 ----------------------------------
q1	17710	4921	4905	4905
q2	2015	152	141	141
q3	10588	1102	1090	1090
q4	10474	776	813	776
q5	7800	2925	2921	2921
q6	203	124	120	120
q7	918	500	493	493
q8	9257	1973	1983	1973
q9	6425	6426	6308	6308
q10	8200	2998	3043	2998
q11	438	220	219	219
q12	378	231	221	221
q13	17997	3392	3342	3342
q14	240	212	212	212
q15	543	508	503	503
q16	443	423	405	405
q17	945	700	596	596
q18	7317	6622	6589	6589
q19	1580	1520	1523	1520
q20	745	318	299	299
q21	2736	2340	2371	2340
q22	343	300	309	300
Total cold run time: 107295 ms
Total hot run time: 38271 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4899	4917	4857	4857
q2	336	231	271	231
q3	3263	3291	3270	3270
q4	2123	2016	2056	2016
q5	5779	5776	5760	5760
q6	201	117	117	117
q7	2316	1882	1930	1882
q8	3365	3418	3486	3418
q9	8697	8719	8731	8719
q10	3762	3775	3837	3775
q11	560	438	455	438
q12	790	621	620	620
q13	4096	3159	3171	3159
q14	292	255	265	255
q15	543	504	501	501
q16	522	471	497	471
q17	1883	1866	1844	1844
q18	8600	8367	8390	8367
q19	1589	1618	1623	1618
q20	2188	1939	1962	1939
q21	5506	5173	5170	5170
q22	500	437	433	433
Total cold run time: 61810 ms
Total hot run time: 58860 ms

doris-robot · 2024-01-08T07:34:03Z

TPC-H: Total hot run time: 38857 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 483998a575420aec7ef14abdf1ad881d0d2a288b, data reload: false
------ Round 1 ----------------------------------
q1	17657	5011	4976	4976
q2	2024	159	147	147
q3	10721	1120	1168	1120
q4	10542	782	842	782
q5	7813	2977	2981	2977
q6	203	124	121	121
q7	924	513	500	500
q8	9298	2003	2007	2003
q9	6480	6451	6397	6397
q10	8326	3078	3046	3046
q11	438	232	216	216
q12	380	228	227	227
q13	18092	3352	3365	3352
q14	237	215	202	202
q15	552	513	502	502
q16	484	387	395	387
q17	952	723	669	669
q18	7416	6765	6708	6708
q19	1557	1533	1536	1533
q20	735	303	321	303
q21	2746	2375	2378	2375
q22	352	323	314	314
Total cold run time: 107929 ms
Total hot run time: 38857 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4942	4899	4903	4899
q2	335	238	247	238
q3	3306	3318	3272	3272
q4	2109	1997	2006	1997
q5	5816	5796	5793	5793
q6	205	115	114	114
q7	2334	1864	1927	1864
q8	3380	3470	3466	3466
q9	8857	8799	8758	8758
q10	3815	3862	3872	3862
q11	558	441	438	438
q12	809	632	661	632
q13	6190	3168	3184	3168
q14	305	276	264	264
q15	547	504	512	504
q16	535	477	475	475
q17	1887	1864	1866	1864
q18	8549	8395	8340	8340
q19	1608	1645	1643	1643
q20	2183	1959	1934	1934
q21	5574	5195	5171	5171
q22	503	444	469	444
Total cold run time: 64347 ms
Total hot run time: 59140 ms

doris-robot · 2024-01-08T07:45:41Z

TPC-DS: Total hot run time: 184384 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 483998a575420aec7ef14abdf1ad881d0d2a288b, data reload: false
run tpcds-sf100 query with default conf and session variables
query1	910	344	338	338
query2	4193	2003	1984	1984
query3	5534	230	227	227
query4	25900	22459	22466	22459
query5	5377	643	612	612
query6	342	207	217	207
query7	4926	312	307	307
query8	242	231	220	220
query9	7680	2988	3014	2988
query10	557	354	329	329
query11	16331	15735	15737	15735
query12	164	108	95	95
query13	1491	353	344	344
query14	11792	7610	7686	7610
query15	274	250	218	218
query16	6210	348	338	338
query17	1398	522	524	522
query18	1843	329	302	302
query19	254	160	160	160
query20	111	104	98	98
query21	171	94	94	94
query22	5156	4922	4781	4781
query23	32244	31090	31738	31090
query24	11601	2927	2951	2927
query25	604	362	370	362
query26	1574	172	174	172
query27	2779	313	310	310
query28	6314	2190	2188	2188
query29	1407	426	419	419
query30	281	144	149	144
query31	1031	827	808	808
query32	138	86	79	79
query33	808	351	310	310
query34	859	465	465	465
query35	922	884	854	854
query36	1353	1362	1298	1298
query37	182	97	86	86
query38	3494	3302	3371	3302
query39	1359	1283	1306	1283
query40	284	90	91	90
query41	38	36	37	36
query42	111	113	105	105
query43	555	520	506	506
query44	1205	799	772	772
query45	225	204	213	204
query46	1060	710	728	710
query47	1776	1621	1650	1621
query48	377	294	296	294
query49	1055	329	315	315
query50	733	331	358	331
query51	5411	5349	5424	5349
query52	115	94	103	94
query53	227	168	167	167
query54	1105	677	723	677
query55	111	104	98	98
query56	270	242	232	232
query57	1031	1001	1013	1001
query58	305	269	288	269
query59	3040	2780	2753	2753
query60	327	304	296	296
query61	149	151	137	137
query62	600	476	467	467
query63	194	179	189	179
query64	4972	1801	1742	1742
query65	3417	3351	3344	3344
query66	1287	375	373	373
query67	15766	15278	15650	15278
query68	13171	546	545	545
query69	625	328	318	318
query70	1787	1578	1556	1556
query71	601	258	293	258
query72	5763	3519	3527	3519
query73	2883	336	333	333
query74	7011	6549	6496	6496
query75	5364	2271	2284	2271
query76	6282	1212	1233	1212
query77	739	313	336	313
query78	9040	8632	8722	8632
query79	1033	526	549	526
query80	541	366	348	348
query81	460	219	219	219
query82	221	110	103	103
query83	193	139	140	139
query84	262	59	58	58
query85	926	284	274	274
query86	442	400	415	400
query87	3626	3489	3402	3402
query88	2950	2478	2473	2473
query89	360	285	275	275
query90	2030	249	256	249
query91	121	93	100	93
query92	84	81	78	78
query93	1292	439	497	439
query94	949	251	251	251
query95	583	507	473	473
query96	643	352	345	345
query97	4361	4235	4199	4199
query98	257	216	211	211
query99	1145	900	871	871
Total cold run time: 289297 ms
Total hot run time: 184384 ms

doris-robot · 2024-01-08T07:59:10Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.52 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.9 seconds inserted 10000000 Rows, about 358K ops/s
storage size: 17184127897 Bytes

morrySnow · 2024-01-09T03:20:19Z

run buildall

doris-robot · 2024-01-09T03:48:45Z

TPC-H: Total hot run time: 38245 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 483998a575420aec7ef14abdf1ad881d0d2a288b, data reload: false
------ Round 1 ----------------------------------
q1	17704	4932	4887	4887
q2	2007	152	142	142
q3	10601	1096	1163	1096
q4	10270	794	799	794
q5	7778	2977	2911	2911
q6	203	122	124	122
q7	907	506	501	501
q8	9277	1964	1993	1964
q9	6448	6375	6350	6350
q10	8237	3049	2990	2990
q11	436	211	213	211
q12	383	228	227	227
q13	17997	3389	3379	3379
q14	240	216	211	211
q15	543	508	500	500
q16	456	403	409	403
q17	946	690	538	538
q18	7188	6658	6596	6596
q19	1560	1518	1527	1518
q20	700	338	298	298
q21	2717	2387	2313	2313
q22	358	294	306	294
Total cold run time: 106956 ms
Total hot run time: 38245 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4944	4911	4894	4894
q2	331	233	252	233
q3	3276	3285	3261	3261
q4	2112	1995	1989	1989
q5	5792	5746	5747	5746
q6	201	115	117	115
q7	2327	1887	1884	1884
q8	3366	3474	3457	3457
q9	8774	8700	8693	8693
q10	3776	3836	3855	3836
q11	529	453	439	439
q12	794	637	611	611
q13	8881	3168	3180	3168
q14	291	261	271	261
q15	540	509	505	505
q16	539	475	528	475
q17	1877	1848	1851	1848
q18	8615	8334	8280	8280
q19	1581	1643	1626	1626
q20	2186	1935	1951	1935
q21	5617	5189	5135	5135
q22	516	473	440	440
Total cold run time: 66865 ms
Total hot run time: 58831 ms

doris-robot · 2024-01-09T04:00:31Z

TPC-DS: Total hot run time: 183738 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 483998a575420aec7ef14abdf1ad881d0d2a288b, data reload: false
run tpcds-sf100 query with default conf and session variables
query1	945	339	328	328
query2	6556	2040	1921	1921
query3	6733	226	231	226
query4	25914	22468	22458	22458
query5	5561	661	620	620
query6	335	207	216	207
query7	4634	321	305	305
query8	258	251	222	222
query9	8827	2896	2903	2896
query10	556	331	335	331
query11	16185	15614	15803	15614
query12	163	105	104	104
query13	1688	340	333	333
query14	13530	7498	7525	7498
query15	277	247	221	221
query16	6470	336	315	315
query17	1716	521	522	521
query18	1962	312	315	312
query19	243	166	172	166
query20	114	106	104	104
query21	194	97	95	95
query22	4899	4948	4881	4881
query23	32497	30880	30976	30880
query24	12019	2934	2917	2917
query25	615	371	359	359
query26	1853	173	180	173
query27	2807	322	317	317
query28	6939	2080	2089	2080
query29	2036	434	410	410
query30	288	144	149	144
query31	1028	810	813	810
query32	142	89	81	81
query33	854	331	347	331
query34	938	474	472	472
query35	956	896	842	842
query36	1407	1306	1269	1269
query37	211	85	103	85
query38	3467	3344	3343	3343
query39	1369	1301	1311	1301
query40	301	89	91	89
query41	40	36	35	35
query42	117	109	115	109
query43	557	512	530	512
query44	1168	807	817	807
query45	223	203	206	203
query46	1062	707	703	703
query47	1710	1653	1605	1605
query48	374	291	288	288
query49	1214	307	316	307
query50	791	319	367	319
query51	5386	5350	5286	5286
query52	120	96	101	96
query53	231	165	168	165
query54	1518	711	653	653
query55	107	97	101	97
query56	264	258	243	243
query57	1024	1027	959	959
query58	320	284	294	284
query59	2988	2885	2836	2836
query60	316	306	304	304
query61	145	153	156	153
query62	594	490	451	451
query63	206	171	169	169
query64	6007	1773	1772	1772
query65	3428	3306	3342	3306
query66	1389	392	379	379
query67	15672	15320	15298	15298
query68	12898	538	556	538
query69	611	319	332	319
query70	1817	1560	1623	1560
query71	601	295	276	276
query72	5406	3509	3515	3509
query73	2880	348	342	342
query74	6978	6571	6535	6535
query75	5121	2301	2270	2270
query76	6246	1068	1267	1068
query77	732	318	326	318
query78	9080	8546	8644	8546
query79	1048	538	527	527
query80	538	341	345	341
query81	463	221	221	221
query82	256	108	108	108
query83	166	141	140	140
query84	259	59	58	58
query85	975	280	275	275
query86	442	453	412	412
query87	3637	3463	3442	3442
query88	3037	2529	2546	2529
query89	370	284	276	276
query90	1857	239	264	239
query91	123	94	98	94
query92	88	79	73	73
query93	1125	432	417	417
query94	861	248	223	223
query95	562	515	501	501
query96	653	339	342	339
query97	4382	4223	4249	4223
query98	250	220	207	207
query99	1166	892	899	892
Total cold run time: 299086 ms
Total hot run time: 183738 ms

doris-robot · 2024-01-09T04:05:57Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.97 seconds
stream load tsv: 568 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.4 seconds inserted 10000000 Rows, about 364K ops/s
storage size: 17184206260 Bytes

morrySnow · 2024-01-09T07:28:07Z

run p0

morrySnow · 2024-01-09T07:28:15Z

run pipelinex_p0

github-actions · 2024-01-09T08:27:01Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-01-09T08:27:04Z

PR approved by anyone and no changes requested.

…ed (#28959) FIX 1. remove float and double literal toString and getStringValue introduced by PR #23504 and PR #23271 These functions lead to wrong cast result of double and float literal 2. fix compute signature for datetimev2 always produce scale 6 3. fix stats calculator failed when generate node stats with two same column 4. constant fold on fe failed when cast double to integral TODO after fix the first problem, some mv matching not work well, fix them later - test_dup_mv_div - test_dup_mv_json - test_tcu

…ed (apache#28959) FIX 1. remove float and double literal toString and getStringValue introduced by PR apache#23504 and PR apache#23271 These functions lead to wrong cast result of double and float literal 2. fix compute signature for datetimev2 always produce scale 6 3. fix stats calculator failed when generate node stats with two same column 4. constant fold on fe failed when cast double to integral TODO after fix the first problem, some mv matching not work well, fix them later - test_dup_mv_div - test_dup_mv_json - test_tcu

morrySnow force-pushed the decimal_double branch from 48c2e8d to 2b95457Compare December 26, 2023 08:48

morrySnow force-pushed the decimal_double branch from 2b95457 to 1614960Compare December 27, 2023 06:37

morrySnow force-pushed the decimal_double branch from 1614960 to 3f7492fCompare December 28, 2023 06:22

morrySnow force-pushed the decimal_double branch 2 times, most recently from 3bb1b72 to 86fe51aCompare December 28, 2023 09:03

morrySnow force-pushed the decimal_double branch from 86fe51a to e286733Compare December 29, 2023 06:20

morrySnow marked this pull request as draft January 5, 2024 02:49

morrySnow force-pushed the decimal_double branch from e286733 to dba3fd8Compare January 5, 2024 07:17

morrySnow force-pushed the decimal_double branch from 082e2a1 to 483998aCompare January 8, 2024 07:13

morrySnow marked this pull request as ready for review January 8, 2024 07:14

keanji-x approved these changes Jan 9, 2024
View reviewed changes

englefly approved these changes Jan 9, 2024
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Jan 9, 2024

morrySnow merged commit 6a4cb51 into apache:masterJan 9, 2024

starocean999 approved these changes Jan 9, 2024
View reviewed changes

morrySnow deleted the decimal_double branch January 9, 2024 09:18

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

LiBinfeng-01 mentioned this pull request Mar 26, 2025
[fix](Nereids) fix double literal to string literal cast problem #49416
Merged
16 tasks

seawinde mentioned this pull request Apr 10, 2025
[fix](mv) Add sync mv test that had been commented #49970
Merged
16 tasks

Uh oh!

Conversation

morrySnow commented Dec 25, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

morrySnow commented Dec 25, 2023

Uh oh!

doris-robot commented Dec 25, 2023

Uh oh!

doris-robot commented Dec 25, 2023

Uh oh!

morrySnow commented Dec 26, 2023

Uh oh!

doris-robot commented Dec 26, 2023

Uh oh!

doris-robot commented Dec 26, 2023

Uh oh!

morrySnow commented Dec 26, 2023

Uh oh!

doris-robot commented Dec 26, 2023

Uh oh!

doris-robot commented Dec 26, 2023

Uh oh!

morrySnow commented Dec 28, 2023

Uh oh!

doris-robot commented Dec 28, 2023

Uh oh!

doris-robot commented Dec 28, 2023

Uh oh!

morrySnow commented Dec 28, 2023

Uh oh!

doris-robot commented Dec 28, 2023

Uh oh!

doris-robot commented Dec 28, 2023

Uh oh!

morrySnow commented Dec 28, 2023

Uh oh!

doris-robot commented Dec 28, 2023

Uh oh!

doris-robot commented Dec 28, 2023

Uh oh!

morrySnow commented Dec 29, 2023

Uh oh!

github-actionsBot commented Dec 29, 2023

Uh oh!

morrySnow commented Jan 5, 2024

Uh oh!

doris-robot commented Jan 5, 2024

Uh oh!

doris-robot commented Jan 8, 2024

Uh oh!

morrySnow commented Jan 8, 2024

Uh oh!

doris-robot commented Jan 8, 2024

Uh oh!

doris-robot commented Jan 8, 2024

Uh oh!

doris-robot commented Jan 8, 2024

Uh oh!

doris-robot commented Jan 8, 2024

Uh oh!

morrySnow commented Jan 9, 2024

Uh oh!

doris-robot commented Jan 9, 2024

Uh oh!

doris-robot commented Jan 9, 2024

Uh oh!

doris-robot commented Jan 9, 2024

Uh oh!

morrySnow commented Jan 9, 2024

Uh oh!

morrySnow commented Jan 9, 2024

Uh oh!

github-actionsBot commented Jan 9, 2024

Uh oh!

github-actionsBot commented Jan 9, 2024

Uh oh!

Reviewers

morrySnow commented Dec 25, 2023•
edited
Loading