[Fix](nereids) fix rule merge_aggregate when has project by feiniaofeiafei · Pull Request #33892 · apache/doris

feiniaofeiafei · 2024-04-19T08:35:27Z

This pr fix 2 problem of merge_aggregate:
1.The map innerAggExprIdToAggFunc key is the inner agg output agg func exprid, value is the Agg func. When a LogicalProject is between 2 agg, the outeragg func child refer to the project output, I forget to do the rewrite to make the outeragg func child refer to the inneragg output. This can lead to an incorrect judgment when determining whether the outer agg func child exists in the inner agg func outputs. This pr fix this bug.
2.When merging 2 sum/min/max, the outer agg group by key is empty, and the inner agg group by key is not empty, the nullable attribute need to be adjusted. Because the sum without group by key is always nullable, the sum with group by key nullable attribute determined by the sum child nullable attribute. This pr fix this bug.

doris-robot · 2024-04-19T08:35:31Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

feiniaofeiafei · 2024-04-19T08:35:36Z

run buildall

doris-robot · 2024-04-19T09:14:26Z

TPC-H: Total hot run time: 38418 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 44a2b1f24b5b60ad11b110ac56098acc13b768cf, data reload: false
------ Round 1 ----------------------------------
q1	17615	4362	4203	4203
q2	2011	187	181	181
q3	10459	1151	1157	1151
q4	10188	781	832	781
q5	7513	2657	2619	2619
q6	214	130	129	129
q7	1005	618	577	577
q8	9239	2063	2014	2014
q9	7502	6558	6561	6558
q10	8562	3509	3535	3509
q11	459	240	226	226
q12	433	223	208	208
q13	17769	2943	2930	2930
q14	260	223	238	223
q15	512	482	480	480
q16	508	395	376	376
q17	953	643	747	643
q18	7351	6830	6745	6745
q19	3864	1480	1478	1478
q20	640	319	305	305
q21	3572	2800	2774	2774
q22	358	308	318	308
Total cold run time: 110987 ms
Total hot run time: 38418 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4277	4230	4168	4168
q2	384	276	275	275
q3	2953	2778	2782	2778
q4	1850	1575	1609	1575
q5	5311	5267	5300	5267
q6	203	122	121	121
q7	2259	1888	1885	1885
q8	3203	3318	3306	3306
q9	8520	8607	8554	8554
q10	4064	3859	3830	3830
q11	609	524	518	518
q12	804	600	631	600
q13	16326	3235	3289	3235
q14	311	284	307	284
q15	523	486	474	474
q16	487	449	448	448
q17	1808	1542	1557	1542
q18	8138	7990	7983	7983
q19	1643	1540	1584	1540
q20	2031	1875	1824	1824
q21	5126	4963	4974	4963
q22	581	474	492	474
Total cold run time: 71411 ms
Total hot run time: 55644 ms

doris-robot · 2024-04-19T09:26:02Z

TPC-DS: Total hot run time: 183979 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 44a2b1f24b5b60ad11b110ac56098acc13b768cf, data reload: false
query1	887	377	351	351
query2	6221	2648	2436	2436
query3	6665	214	202	202
query4	22681	21265	21234	21234
query5	4141	399	404	399
query6	274	177	180	177
query7	4578	296	282	282
query8	238	195	189	189
query9	8496	2339	2292	2292
query10	432	237	244	237
query11	14749	14212	14227	14212
query12	139	89	84	84
query13	1644	385	368	368
query14	9157	7780	7562	7562
query15	281	172	183	172
query16	8182	251	254	251
query17	1935	555	539	539
query18	2099	270	265	265
query19	257	146	144	144
query20	90	88	83	83
query21	196	127	119	119
query22	5084	4909	5000	4909
query23	34359	33237	33212	33212
query24	11125	3052	3025	3025
query25	617	382	389	382
query26	717	152	151	151
query27	2351	365	371	365
query28	6179	2065	2081	2065
query29	864	618	614	614
query30	300	178	172	172
query31	1024	773	777	773
query32	96	51	52	51
query33	765	246	248	246
query34	1126	473	491	473
query35	860	723	686	686
query36	1080	905	940	905
query37	120	73	72	72
query38	3476	3361	3314	3314
query39	1090	1034	1030	1030
query40	177	130	133	130
query41	48	42	46	42
query42	105	97	96	96
query43	588	532	556	532
query44	1290	748	742	742
query45	298	275	281	275
query46	1113	743	724	724
query47	2022	1924	1954	1924
query48	381	309	307	307
query49	830	400	406	400
query50	786	391	401	391
query51	6819	6759	6736	6736
query52	96	92	89	89
query53	340	277	280	277
query54	294	221	227	221
query55	79	77	76	76
query56	253	219	222	219
query57	1237	1119	1097	1097
query58	237	191	195	191
query59	3504	3172	3210	3172
query60	273	222	224	222
query61	87	87	85	85
query62	591	443	416	416
query63	297	273	272	272
query64	4678	4006	3199	3199
query65	3025	3019	3041	3019
query66	752	330	315	315
query67	15243	15082	14894	14894
query68	5886	539	539	539
query69	530	303	296	296
query70	1217	1169	1194	1169
query71	1436	1266	1258	1258
query72	6475	2684	2436	2436
query73	728	314	313	313
query74	6909	6553	6423	6423
query75	3524	2604	2647	2604
query76	3766	958	968	958
query77	592	265	262	262
query78	11034	10148	10319	10148
query79	7780	520	519	519
query80	1561	435	439	435
query81	520	241	246	241
query82	1605	98	94	94
query83	260	166	162	162
query84	270	81	82	81
query85	1387	263	260	260
query86	468	267	309	267
query87	3500	3236	3254	3236
query88	5097	2297	2312	2297
query89	503	373	364	364
query90	1867	186	177	177
query91	125	97	97	97
query92	54	45	47	45
query93	6265	516	506	506
query94	1034	181	175	175
query95	385	298	315	298
query96	600	259	255	255
query97	3146	2920	2899	2899
query98	232	216	218	216
query99	1211	873	843	843
Total cold run time: 289985 ms
Total hot run time: 183979 ms

doris-robot · 2024-04-19T09:31:34Z

ClickBench: Total hot run time: 30.23 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 44a2b1f24b5b60ad11b110ac56098acc13b768cf, data reload: false
query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.23	0.06	0.05
query4	1.66	0.07	0.08
query5	0.50	0.49	0.50
query6	1.47	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.54	0.56	0.55
query11	0.16	0.11	0.10
query12	0.15	0.11	0.12
query13	0.61	0.58	0.59
query14	0.75	0.79	0.76
query15	0.82	0.79	0.79
query16	0.36	0.36	0.37
query17	0.97	1.01	0.97
query18	0.22	0.24	0.24
query19	1.84	1.72	1.74
query20	0.02	0.01	0.02
query21	15.40	0.66	0.65
query22	4.21	7.70	1.72
query23	18.26	1.33	1.28
query24	1.68	0.26	0.22
query25	0.14	0.09	0.08
query26	0.26	0.16	0.16
query27	0.07	0.08	0.07
query28	13.34	1.01	0.98
query29	12.61	3.25	3.25
query30	0.27	0.07	0.06
query31	2.85	0.38	0.38
query32	3.27	0.46	0.46
query33	2.79	2.84	2.82
query34	17.16	4.43	4.53
query35	4.51	4.50	4.62
query36	0.64	0.50	0.46
query37	0.20	0.15	0.14
query38	0.15	0.14	0.14
query39	0.04	0.03	0.04
query40	0.18	0.14	0.14
query41	0.11	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.29 s
Total hot run time: 30.23 s

doris-robot · 2024-04-19T09:34:40Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 44a2b1f24b5b60ad11b110ac56098acc13b768cf with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select: 13.4 seconds inserted 10000000 Rows, about 746K ops/s

feiniaofeiafei · 2024-04-19T10:06:15Z

run buildall

doris-robot · 2024-04-19T10:47:49Z

TPC-H: Total hot run time: 38192 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 74dbcfa4a8f5d7185738bfb98333c394e9558af1, data reload: false
------ Round 1 ----------------------------------
q1	17626	4267	4194	4194
q2	2010	195	181	181
q3	10443	1158	1182	1158
q4	10191	721	791	721
q5	7529	2694	2643	2643
q6	217	131	132	131
q7	1003	590	590	590
q8	9226	2051	2019	2019
q9	7344	6562	6493	6493
q10	8416	3517	3690	3517
q11	432	240	218	218
q12	468	216	215	215
q13	17763	2948	2933	2933
q14	266	230	231	230
q15	521	486	481	481
q16	527	384	373	373
q17	943	624	721	624
q18	7364	6649	6740	6649
q19	6471	1511	1525	1511
q20	640	312	299	299
q21	3424	2712	2765	2712
q22	364	300	304	300
Total cold run time: 113188 ms
Total hot run time: 38192 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4299	4194	4214	4194
q2	377	270	260	260
q3	2966	2715	2708	2708
q4	1877	1602	1504	1504
q5	5321	5350	5276	5276
q6	210	122	121	121
q7	2215	1862	1850	1850
q8	3168	3329	3326	3326
q9	8586	8538	8551	8538
q10	4128	3920	3944	3920
q11	623	485	509	485
q12	791	628	633	628
q13	16459	3168	3130	3130
q14	329	283	280	280
q15	516	514	480	480
q16	505	461	443	443
q17	1807	1550	1500	1500
q18	7973	7870	7795	7795
q19	1701	1546	1512	1512
q20	2025	1836	1822	1822
q21	5126	4826	4890	4826
q22	553	491	475	475
Total cold run time: 71555 ms
Total hot run time: 55073 ms

doris-robot · 2024-04-19T10:59:29Z

TPC-DS: Total hot run time: 185253 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 74dbcfa4a8f5d7185738bfb98333c394e9558af1, data reload: false
query1	896	363	350	350
query2	6193	2587	2450	2450
query3	6654	210	209	209
query4	22858	21336	21271	21271
query5	4128	402	402	402
query6	268	177	168	168
query7	4587	288	280	280
query8	237	192	183	183
query9	8341	2312	2274	2274
query10	412	245	269	245
query11	14732	14098	14095	14095
query12	136	88	86	86
query13	1641	368	369	368
query14	9348	7528	7777	7528
query15	257	176	184	176
query16	8194	262	254	254
query17	1935	571	539	539
query18	2117	275	271	271
query19	290	151	146	146
query20	91	85	80	80
query21	193	123	123	123
query22	5087	4866	4843	4843
query23	33842	33086	33112	33086
query24	10970	3054	3005	3005
query25	576	374	387	374
query26	702	161	164	161
query27	2374	373	362	362
query28	5821	2032	2059	2032
query29	895	621	643	621
query30	321	175	177	175
query31	1036	776	746	746
query32	95	58	58	58
query33	682	247	249	247
query34	980	486	497	486
query35	840	698	764	698
query36	1069	902	930	902
query37	110	70	71	70
query38	3551	3412	3364	3364
query39	1651	1582	1591	1582
query40	172	130	125	125
query41	47	42	42	42
query42	104	95	105	95
query43	585	563	579	563
query44	1096	766	768	766
query45	270	265	273	265
query46	1096	765	699	699
query47	2057	1959	1931	1931
query48	370	305	302	302
query49	890	386	395	386
query50	782	403	393	393
query51	6816	6822	6803	6803
query52	108	84	95	84
query53	351	284	278	278
query54	301	239	238	238
query55	79	74	83	74
query56	249	231	242	231
query57	1329	1238	1215	1215
query58	214	205	206	205
query59	3559	3138	3189	3138
query60	247	233	223	223
query61	90	85	83	83
query62	603	444	424	424
query63	302	272	267	267
query64	4728	3758	3787	3758
query65	3098	3027	3034	3027
query66	731	321	321	321
query67	15387	14893	15283	14893
query68	7589	544	540	540
query69	547	302	302	302
query70	1206	1115	1178	1115
query71	1478	1262	1257	1257
query72	6750	2594	2424	2424
query73	752	313	312	312
query74	6927	6485	6409	6409
query75	4090	2639	2615	2615
query76	5048	1032	1030	1030
query77	616	259	307	259
query78	11084	10182	10396	10182
query79	8017	532	519	519
query80	1285	450	446	446
query81	498	247	250	247
query82	891	96	93	93
query83	204	170	165	165
query84	261	85	83	83
query85	1188	273	261	261
query86	448	312	310	310
query87	3442	3243	3227	3227
query88	5059	2323	2325	2323
query89	539	375	371	371
query90	1926	181	183	181
query91	145	100	97	97
query92	68	49	52	49
query93	6125	512	505	505
query94	963	177	176	176
query95	391	303	298	298
query96	615	262	263	262
query97	3173	2983	2977	2977
query98	231	220	225	220
query99	1218	859	868	859
Total cold run time: 292545 ms
Total hot run time: 185253 ms

doris-robot · 2024-04-19T11:04:58Z

ClickBench: Total hot run time: 31.19 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 74dbcfa4a8f5d7185738bfb98333c394e9558af1, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.11	0.08
query5	0.50	0.49	0.49
query6	1.46	0.72	0.74
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.48	0.50
query10	0.55	0.55	0.54
query11	0.15	0.11	0.11
query12	0.13	0.12	0.13
query13	0.62	0.59	0.59
query14	0.76	0.78	0.76
query15	0.82	0.80	0.80
query16	0.35	0.35	0.36
query17	0.96	0.96	1.00
query18	0.21	0.25	0.23
query19	1.85	1.71	1.68
query20	0.01	0.01	0.01
query21	15.40	0.66	0.65
query22	3.95	6.67	2.64
query23	18.31	1.31	1.28
query24	1.43	0.32	0.29
query25	0.14	0.08	0.09
query26	0.26	0.17	0.16
query27	0.08	0.07	0.09
query28	13.35	1.00	0.98
query29	12.55	3.25	3.26
query30	0.26	0.08	0.06
query31	2.86	0.37	0.38
query32	3.27	0.47	0.46
query33	2.82	2.79	2.90
query34	17.05	4.43	4.50
query35	4.52	4.59	4.54
query36	0.66	0.48	0.45
query37	0.19	0.15	0.15
query38	0.16	0.14	0.15
query39	0.04	0.03	0.04
query40	0.18	0.14	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 108.68 s
Total hot run time: 31.19 s

doris-robot · 2024-04-19T11:08:04Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 74dbcfa4a8f5d7185738bfb98333c394e9558af1 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select: 13.9 seconds inserted 10000000 Rows, about 719K ops/s

feiniaofeiafei · 2024-04-22T03:49:44Z

run buildall

doris-robot · 2024-04-22T04:23:20Z

TPC-H: Total hot run time: 39290 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6bb4604891eb45d156f2193279a791af9518c8d2, data reload: false
------ Round 1 ----------------------------------
q1	18519	4560	4426	4426
q2	2602	191	191	191
q3	11435	1223	1190	1190
q4	11216	797	865	797
q5	7758	2748	2708	2708
q6	225	140	139	139
q7	1050	657	626	626
q8	9410	2141	2085	2085
q9	7582	6818	6734	6734
q10	8973	3629	3642	3629
q11	457	239	232	232
q12	420	226	223	223
q13	18091	2929	2984	2929
q14	274	233	240	233
q15	532	476	480	476
q16	501	380	380	380
q17	978	710	682	682
q18	7440	6798	6654	6654
q19	3290	1549	1551	1549
q20	652	317	302	302
q21	3428	2858	2801	2801
q22	357	313	304	304
Total cold run time: 115190 ms
Total hot run time: 39290 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4344	4248	4276	4248
q2	369	273	259	259
q3	2994	2700	2775	2700
q4	1899	1588	1591	1588
q5	5329	5333	5317	5317
q6	209	126	129	126
q7	2283	1972	1903	1903
q8	3222	3328	3344	3328
q9	8591	8631	8652	8631
q10	3923	3723	3692	3692
q11	588	496	496	496
q12	748	600	598	598
q13	16380	2945	2947	2945
q14	305	265	271	265
q15	509	484	477	477
q16	476	423	426	423
q17	1783	1463	1468	1463
q18	7492	7679	7444	7444
q19	1662	1536	1559	1536
q20	1966	1747	1744	1744
q21	4991	4844	4779	4779
q22	546	468	461	461
Total cold run time: 70609 ms
Total hot run time: 54423 ms

doris-robot · 2024-04-22T04:34:55Z

TPC-DS: Total hot run time: 184504 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6bb4604891eb45d156f2193279a791af9518c8d2, data reload: false
query1	919	368	355	355
query2	7350	2373	2362	2362
query3	6666	217	214	214
query4	24149	21401	21328	21328
query5	4119	410	417	410
query6	262	175	176	175
query7	4578	290	279	279
query8	242	185	178	178
query9	8347	2336	2319	2319
query10	591	259	253	253
query11	14782	14229	14204	14204
query12	140	89	85	85
query13	1634	352	356	352
query14	9485	7852	7608	7608
query15	266	180	183	180
query16	8183	259	254	254
query17	1945	588	541	541
query18	2103	273	270	270
query19	275	145	153	145
query20	92	83	86	83
query21	202	126	126	126
query22	5036	4835	4800	4800
query23	34029	33203	33033	33033
query24	11879	2998	2931	2931
query25	649	366	371	366
query26	1749	146	148	146
query27	3127	299	316	299
query28	7750	1999	1991	1991
query29	1009	616	631	616
query30	279	171	173	171
query31	987	729	720	720
query32	96	51	53	51
query33	767	257	240	240
query34	1104	465	485	465
query35	808	689	720	689
query36	1006	905	902	902
query37	286	67	70	67
query38	3481	3192	3186	3186
query39	1584	1553	1534	1534
query40	278	124	125	124
query41	47	44	42	42
query42	105	92	97	92
query43	579	515	532	515
query44	1194	722	722	722
query45	265	263	246	246
query46	1077	733	736	733
query47	1937	1849	1874	1849
query48	359	298	320	298
query49	1202	386	394	386
query50	760	379	379	379
query51	6682	6662	6654	6654
query52	106	86	89	86
query53	350	287	279	279
query54	317	221	223	221
query55	78	70	69	69
query56	242	221	228	221
query57	1253	1129	1133	1129
query58	227	193	206	193
query59	3481	3244	3120	3120
query60	248	237	236	236
query61	116	89	87	87
query62	639	440	449	440
query63	299	273	277	273
query64	6251	3847	3915	3847
query65	3156	3039	2998	2998
query66	1389	346	337	337
query67	15449	14834	14974	14834
query68	5118	530	548	530
query69	471	291	299	291
query70	1165	1174	1190	1174
query71	1390	1262	1263	1262
query72	6350	2658	2420	2420
query73	707	318	320	318
query74	6827	6486	6517	6486
query75	3369	2609	2616	2609
query76	2945	968	1005	968
query77	433	260	265	260
query78	11008	10341	10208	10208
query79	2581	513	508	508
query80	2019	432	470	432
query81	544	237	243	237
query82	768	95	98	95
query83	275	162	173	162
query84	267	85	83	83
query85	2071	290	277	277
query86	483	275	300	275
query87	3495	3278	3318	3278
query88	3977	2333	2333	2333
query89	455	372	369	369
query90	1978	181	183	181
query91	128	99	97	97
query92	63	49	47	47
query93	3588	517	506	506
query94	1218	181	176	176
query95	386	315	293	293
query96	598	269	263	263
query97	3149	2977	2952	2952
query98	233	224	224	224
query99	1206	856	852	852
Total cold run time: 289307 ms
Total hot run time: 184504 ms

doris-robot · 2024-04-22T04:40:27Z

ClickBench: Total hot run time: 30.67 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6bb4604891eb45d156f2193279a791af9518c8d2, data reload: false
query1	0.03	0.03	0.04
query2	0.08	0.04	0.03
query3	0.23	0.05	0.06
query4	1.66	0.08	0.08
query5	0.50	0.49	0.50
query6	1.45	0.72	0.71
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.54	0.51	0.49
query10	0.54	0.56	0.55
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.58	0.58
query14	0.78	0.77	0.78
query15	0.82	0.82	0.80
query16	0.37	0.36	0.37
query17	0.95	0.96	1.03
query18	0.20	0.27	0.22
query19	1.89	1.68	1.67
query20	0.02	0.01	0.01
query21	15.42	0.67	0.66
query22	4.12	6.69	2.28
query23	18.31	1.37	1.29
query24	1.56	0.31	0.22
query25	0.15	0.08	0.08
query26	0.25	0.17	0.16
query27	0.08	0.07	0.07
query28	13.35	1.00	0.98
query29	12.64	3.31	3.27
query30	0.26	0.07	0.06
query31	2.83	0.39	0.38
query32	3.27	0.46	0.46
query33	2.78	2.80	2.83
query34	17.18	4.38	4.37
query35	4.46	4.46	4.57
query36	0.64	0.46	0.46
query37	0.19	0.16	0.15
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.17	0.14	0.13
query41	0.09	0.05	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.06 s
Total hot run time: 30.67 s

feiniaofeiafei · 2024-04-22T06:05:48Z

run buildall

doris-robot · 2024-04-22T06:43:27Z

TPC-H: Total hot run time: 38296 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f5ea21ec6d01e21335296b92c70e78e5fbf71234, data reload: false
------ Round 1 ----------------------------------
q1	17612	4292	4227	4227
q2	2016	193	196	193
q3	10435	1183	1224	1183
q4	10193	749	706	706
q5	7538	2688	2603	2603
q6	216	133	136	133
q7	993	595	572	572
q8	9224	2048	2014	2014
q9	7500	6552	6522	6522
q10	8666	3534	3524	3524
q11	444	236	254	236
q12	495	228	219	219
q13	18710	2913	2926	2913
q14	264	227	237	227
q15	530	490	490	490
q16	524	388	373	373
q17	941	696	725	696
q18	7324	6666	6713	6666
q19	4951	1472	1504	1472
q20	638	332	298	298
q21	3504	2724	2717	2717
q22	367	315	312	312
Total cold run time: 113085 ms
Total hot run time: 38296 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4359	4234	4234	4234
q2	380	270	268	268
q3	2923	2762	2738	2738
q4	1839	1570	1537	1537
q5	5326	5356	5296	5296
q6	211	124	124	124
q7	2227	1847	1880	1847
q8	3158	3320	3315	3315
q9	8535	8512	8532	8512
q10	4109	3889	3908	3889
q11	620	520	492	492
q12	810	638	644	638
q13	17438	3201	3109	3109
q14	334	276	291	276
q15	529	478	479	478
q16	507	447	467	447
q17	1776	1518	1522	1518
q18	8188	7937	7891	7891
q19	1651	1547	1543	1543
q20	2085	1868	1802	1802
q21	5204	4968	4992	4968
q22	554	483	464	464
Total cold run time: 72763 ms
Total hot run time: 55386 ms

doris-robot · 2024-04-22T06:55:15Z

TPC-DS: Total hot run time: 185222 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f5ea21ec6d01e21335296b92c70e78e5fbf71234, data reload: false
query1	892	382	363	363
query2	6378	2753	2320	2320
query3	6648	197	195	195
query4	23516	21205	21212	21205
query5	4170	402	408	402
query6	280	182	181	181
query7	4595	303	289	289
query8	249	198	188	188
query9	8478	2326	2297	2297
query10	405	236	253	236
query11	14663	14213	14163	14163
query12	131	94	85	85
query13	1636	364	353	353
query14	9575	7548	7529	7529
query15	258	182	182	182
query16	8109	269	263	263
query17	1907	571	563	563
query18	2049	290	292	290
query19	229	156	150	150
query20	93	85	87	85
query21	204	127	125	125
query22	4940	4841	4799	4799
query23	34214	33050	33277	33050
query24	10601	3148	3058	3058
query25	626	414	413	413
query26	716	177	163	163
query27	2437	371	370	370
query28	6078	2051	2090	2051
query29	871	641	634	634
query30	296	185	182	182
query31	991	751	760	751
query32	97	59	54	54
query33	663	263	271	263
query34	897	497	514	497
query35	853	733	706	706
query36	1060	957	960	957
query37	114	70	74	70
query38	3532	3432	3316	3316
query39	1688	1576	1550	1550
query40	166	127	121	121
query41	48	42	43	42
query42	106	100	102	100
query43	615	537	554	537
query44	1140	740	745	740
query45	285	272	286	272
query46	1091	751	737	737
query47	1985	1979	1963	1963
query48	391	292	309	292
query49	847	400	399	399
query50	766	386	401	386
query51	7006	6810	6770	6770
query52	94	93	87	87
query53	343	275	279	275
query54	300	245	224	224
query55	73	72	70	70
query56	231	217	219	217
query57	1225	1124	1130	1124
query58	222	198	194	194
query59	3407	3266	3084	3084
query60	243	233	223	223
query61	86	83	85	83
query62	568	444	451	444
query63	302	279	285	279
query64	4933	3984	3666	3666
query65	3054	3001	3036	3001
query66	736	327	330	327
query67	15709	15011	15134	15011
query68	8554	545	547	545
query69	554	301	302	301
query70	1204	1167	1175	1167
query71	1475	1265	1262	1262
query72	6677	2596	2453	2453
query73	735	316	315	315
query74	6735	6492	6449	6449
query75	4153	2696	2623	2623
query76	5113	978	967	967
query77	649	288	261	261
query78	11076	10464	10366	10366
query79	11354	534	506	506
query80	2062	465	429	429
query81	517	242	237	237
query82	833	96	103	96
query83	209	161	160	160
query84	258	80	80	80
query85	1310	257	256	256
query86	486	306	301	301
query87	3514	3231	3309	3231
query88	5020	2315	2290	2290
query89	523	385	364	364
query90	2009	178	182	178
query91	131	94	96	94
query92	63	47	46	46
query93	7332	508	502	502
query94	1187	183	178	178
query95	394	290	303	290
query96	605	256	257	256
query97	3105	2955	2937	2937
query98	241	222	212	212
query99	1255	854	872	854
Total cold run time: 300444 ms
Total hot run time: 185222 ms

doris-robot · 2024-04-22T07:00:45Z

ClickBench: Total hot run time: 29.96 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f5ea21ec6d01e21335296b92c70e78e5fbf71234, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.23	0.05	0.06
query4	1.68	0.07	0.07
query5	0.50	0.50	0.49
query6	1.48	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.55	0.56	0.57
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.60	0.59	0.58
query14	0.76	0.77	0.77
query15	0.83	0.80	0.81
query16	0.34	0.37	0.36
query17	0.94	0.93	1.02
query18	0.22	0.24	0.22
query19	1.74	1.68	1.65
query20	0.01	0.02	0.01
query21	15.42	0.66	0.64
query22	4.54	7.54	1.93
query23	18.29	1.25	1.20
query24	1.86	0.21	0.24
query25	0.14	0.08	0.08
query26	0.26	0.16	0.16
query27	0.07	0.08	0.07
query28	13.36	0.99	0.98
query29	12.59	3.27	3.19
query30	0.26	0.07	0.05
query31	2.85	0.38	0.37
query32	3.30	0.46	0.45
query33	2.86	2.78	2.81
query34	17.18	4.36	4.38
query35	4.44	4.47	4.42
query36	0.63	0.46	0.48
query37	0.19	0.16	0.15
query38	0.15	0.15	0.14
query39	0.05	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.06	0.04
query42	0.06	0.04	0.05
query43	0.04	0.03	0.04
Total cold run time: 109.72 s
Total hot run time: 29.96 s

feiniaofeiafei · 2024-04-23T02:22:03Z

run buildall

morrySnow · 2024-04-24T07:46:14Z

 topDown(new EliminateGroupBy(),
- new MergeAggregate())
+ new MergeAggregate(),
+ new AdjustAggregateNullableForEmptySet())


add comment here to explain why need execute AdjustAggregateNullableForEmptySet after MergeAggregate

feiniaofeiafei · 2024-04-24T08:33:49Z

run buildall

feiniaofeiafei · 2024-04-24T12:49:09Z

run p0

github-actions · 2024-04-25T11:18:40Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-04-25T11:18:43Z

PR approved by anyone and no changes requested.

starocean999 marked this pull request as draft April 19, 2024 09:15

feiniaofeiafei force-pushed the merge_agg_bug branch from 6bb4604 to f5ea21eCompare April 22, 2024 06:05

feiniaofeiafei added 3 commits April 23, 2024 10:21

[Fix](nereids) fix rule merge_aggregate when has project
d0f4544

[Fix](nereids) fix rule merge_aggregate when has project
30e4493

adjust nullable after merge aggregate
f95e424

feiniaofeiafei force-pushed the merge_agg_bug branch from f5ea21e to f95e424Compare April 23, 2024 02:21

feiniaofeiafei marked this pull request as ready for review April 23, 2024 06:09

morrySnow reviewed Apr 24, 2024
View reviewed changes

add comment
429ee56

morrySnow approved these changes Apr 25, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Apr 25, 2024

github-actionsBot added the reviewed label Apr 25, 2024

starocean999 approved these changes Apr 26, 2024
View reviewed changes

starocean999 merged commit f34114d into apache:masterApr 26, 2024

yiguolei pushed a commit that referenced this pull request Apr 26, 2024
[Fix](nereids) fix rule merge_aggregate when has project (#33892)
b41a533

dataroaring pushed a commit that referenced this pull request Apr 27, 2024
[Fix](nereids) fix rule merge_aggregate when has project (#33892)
4e6af32

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
[Fix](nereids) fix rule merge_aggregate when has project (apache#33892)
a43a501

Uh oh!

Conversation

feiniaofeiafei commented Apr 19, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

feiniaofeiafei commented Apr 19, 2024

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

feiniaofeiafei commented Apr 19, 2024

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

doris-robot commented Apr 19, 2024

Uh oh!

feiniaofeiafei commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

feiniaofeiafei commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

feiniaofeiafei commented Apr 23, 2024

Uh oh!

morrySnowApr 24, 2024

Choose a reason for hiding this comment

Uh oh!

feiniaofeiafei commented Apr 24, 2024

Uh oh!

feiniaofeiafei commented Apr 24, 2024

Uh oh!

github-actionsBot commented Apr 25, 2024

Uh oh!

github-actionsBot commented Apr 25, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feiniaofeiafei commented Apr 19, 2024•
edited
Loading