[fix](eager-agg) Handle duplicate aggregate functions pushed through projects by feiniaofeiafei · Pull Request #66531 · apache/doris

feiniaofeiafei · 2026-08-06T06:30:58Z

What problem does this PR solve?

Related PR: #63690

Problem Summary:

Eager aggregation pushdown may fail when different aggregate functions become
the same expression after passing through a Project. report error:

2026-08-06 04:26:54,285 INFO (mysql-nio-pool-14|325) [PushDownAggregation.visitLogicalAggregate():280] PushDownAggregation failed: Cannot invoke "org.apache.doris.nereids.trees.expressions.NamedExpression.toSlot()" because "namedExpression" is null
at org.apache.doris.nereids.rules.rewrite.eageraggregation.EagerAggRewriter.visitLogicalProject(EagerAggRewriter.java:718)
at org.apache.doris.nereids.rules.rewrite.eageraggregation.EagerAggRewriter.visitLogicalProject(EagerAggRewriter.java:90)
at org.apache.doris.nereids.trees.plans.logical.LogicalProject.accept(LogicalProject.java:160)
at org.apache.doris.nereids.rules.rewrite.eageraggregation.EagerAggRewriter.visitLogicalUnion(EagerAggRewriter.java:582)
at org.apache.doris.nereids.rules.rewrite.eageraggregation.EagerAggRewriter.visitLogicalUnion(EagerAggRewriter.java:90)
at org.apache.doris.nereids.trees.plans.logical.LogicalUnion.accept(LogicalUnion.java:155)

For example:

Aggregate: SUM(x)#4, SUM(y)#5
Union All
Project: 0 AS x, 0 AS y
Join

After pushing the aggregates through the Project, both functions becomeSUM(0):

functions: [SUM(0), SUM(0)]
aliasMap: SUM(0) -> #5

Because aliasMap uses expression equality, only one entry is retained.
The Project still tries to read both #4 and #5 from BilateralState,
causing a null lookup.

This PR deduplicates the child aggregate and records the ExprId mapping:

child aggregate: SUM(0) -> #8
ExprId mapping: #4 -> #8, #5 -> #8

The Project then restores both required outputs:

slot#8 AS slot#4
slot#8 AS slot#5

When no aggregate functions are merged, the original ExprIds are reused to
avoid unnecessary aliases.

The same fix also covers cases such as:

Project: a#1 AS x, a#1 AS y

where SUM(x) and SUM(y) both become SUM(a#1) after pushdown.

Release note

None

Check List (For Author)

Test
- Regression test
  - query_p0/eager_agg/bilateral_eager_agg
  - Covers two aggregate functions that become the same function after
    Project pushdown.
- Unit Test
- Manual test
- No need to test or manual test.
Behavior changed:
- Yes.
  - Prevents eager aggregation pushdown from failing when multiple
    aggregate functions become identical after Project rewriting.
  - eager_aggregation_mode=1 can force eligible pushdown after a UNION.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-08-06T06:31:04Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

feiniaofeiafei · 2026-08-06T06:31:08Z

run buildall

hello-stephen · 2026-08-06T09:08:23Z

FE Regression Coverage Report

Increment line coverage 2.97% (3/101) 🎉
Increment coverage report
Complete coverage report

feiniaofeiafei · 2026-08-06T13:24:22Z

run buildall

hello-stephen · 2026-08-06T14:27:11Z

TPC-H: Total hot run time: 28827 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b3d4f0d64a87166d3fa65af2c2cf00f5b476f930, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17636	4058	3950	3950
q2	2048	321	210	210
q3	10253	1389	795	795
q4	4679	467	334	334
q5	7542	826	565	565
q6	181	166	135	135
q7	821	804	600	600
q8	9328	1580	1584	1580
q9	5317	4137	4047	4047
q10	6728	1622	1357	1357
q11	504	350	327	327
q12	718	586	451	451
q13	18102	3290	2755	2755
q14	264	263	239	239
q15	q16	736	739	666	666
q17	1004	953	1004	953
q18	6497	5612	5582	5582
q19	1367	1271	1034	1034
q20	791	668	559	559
q21	5885	2563	2387	2387
q22	443	354	301	301
Total cold run time: 100844 ms
Total hot run time: 28827 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4339	4249	4243	4243
q2	269	319	210	210
q3	4625	4892	4393	4393
q4	2169	2259	1410	1410
q5	4212	4103	4131	4103
q6	223	168	131	131
q7	1693	1576	1394	1394
q8	2713	2208	2116	2116
q9	7334	7537	7533	7533
q10	4293	4257	3903	3903
q11	568	423	387	387
q12	732	768	497	497
q13	3162	3502	2891	2891
q14	293	297	276	276
q15	q16	713	715	634	634
q17	1298	1262	1283	1262
q18	12109	11053	11825	11053
q19	1144	1148	1228	1148
q20	2248	2212	1948	1948
q21	5608	4817	4901	4817
q22	543	489	432	432
Total cold run time: 60288 ms
Total hot run time: 54781 ms

hello-stephen · 2026-08-06T14:37:50Z

TPC-DS: Total hot run time: 166510 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b3d4f0d64a87166d3fa65af2c2cf00f5b476f930, data reload: false
query5	4324	584	446	446
query6	453	228	211	211
query7	4870	560	334	334
query8	321	159	148	148
query9	8773	4022	4012	4012
query10	448	365	287	287
query11	5811	2169	1957	1957
query12	151	100	96	96
query13	1281	570	476	476
query14	6073	4246	3981	3981
query14_1	3788	3758	3787	3758
query15	205	197	175	175
query16	991	477	453	453
query17	950	704	569	569
query18	2488	446	326	326
query19	200	180	146	146
query20	103	99	99	99
query21	231	154	130	130
query22	12951	13018	12808	12808
query23	15776	15096	14567	14567
query23_1	15352	15142	15088	15088
query24	7678	1686	1234	1234
query24_1	1177	1214	1216	1214
query25	510	402	340	340
query26	1324	345	233	233
query27	2592	575	391	391
query28	4550	2002	2023	2002
query29	1041	587	456	456
query30	338	259	217	217
query31	1165	1113	1037	1037
query32	117	59	59	59
query33	508	313	235	235
query34	1185	1114	635	635
query35	714	721	634	634
query36	795	780	709	709
query37	154	107	85	85
query38	1813	1790	1668	1668
query39	815	826	798	798
query39_1	805	773	792	773
query40	243	159	133	133
query41	61	60	61	60
query42	92	88	88	88
query43	314	319	265	265
query44	1412	767	766	766
query45	179	169	158	158
query46	1048	1180	729	729
query47	1555	1573	1448	1448
query48	420	396	316	316
query49	572	398	283	283
query50	1078	416	331	331
query51	10468	10628	10547	10547
query52	85	86	78	78
query53	269	276	196	196
query54	278	226	219	219
query55	75	74	65	65
query56	311	277	289	277
query57	1017	1025	938	938
query58	286	248	250	248
query59	1487	1558	1366	1366
query60	316	266	254	254
query61	145	143	147	143
query62	402	318	263	263
query63	228	188	192	188
query64	2933	1150	977	977
query65	3879	3809	3822	3809
query66	1846	480	365	365
query67	28061	28134	27982	27982
query68	3270	1643	971	971
query69	412	301	274	274
query70	859	792	840	792
query71	386	358	319	319
query72	3219	2573	2225	2225
query73	849	821	431	431
query74	4615	4474	4278	4278
query75	2383	2334	1979	1979
query76	2340	1140	731	731
query77	330	355	268	268
query78	11219	11058	10489	10489
query79	1355	1108	740	740
query80	834	544	448	448
query81	481	329	284	284
query82	626	175	132	132
query83	376	320	287	287
query84	317	157	128	128
query85	967	616	516	516
query86	360	227	218	218
query87	1958	1974	1821	1821
query88	3671	2789	2789	2789
query89	398	330	281	281
query90	1861	196	187	187
query91	198	185	164	164
query92	60	58	53	53
query93	1672	1615	1027	1027
query94	614	365	309	309
query95	775	582	461	461
query96	1005	827	355	355
query97	2459	2483	2351	2351
query98	191	186	175	175
query99	744	729	618	618
Total cold run time: 252793 ms
Total hot run time: 166510 ms

hello-stephen · 2026-08-06T14:42:45Z

ClickBench: Total hot run time: 23.7 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b3d4f0d64a87166d3fa65af2c2cf00f5b476f930, data reload: false
query1	0.00	0.00	0.00
query2	0.09	0.04	0.05
query3	0.26	0.13	0.13
query4	1.60	0.14	0.14
query5	0.24	0.21	0.22
query6	1.16	0.84	0.80
query7	0.03	0.01	0.01
query8	0.05	0.03	0.04
query9	0.36	0.30	0.32
query10	0.54	0.57	0.56
query11	0.19	0.14	0.13
query12	0.18	0.15	0.14
query13	0.47	0.45	0.47
query14	1.02	0.99	0.98
query15	0.61	0.58	0.59
query16	0.31	0.32	0.31
query17	1.07	1.06	1.08
query18	0.21	0.20	0.19
query19	2.04	1.99	1.93
query20	0.01	0.02	0.02
query21	15.41	0.19	0.13
query22	4.99	0.06	0.05
query23	16.16	0.31	0.12
query24	2.90	0.40	0.33
query25	0.11	0.05	0.04
query26	0.73	0.20	0.16
query27	0.05	0.03	0.04
query28	3.52	0.81	0.36
query29	12.49	4.03	3.16
query30	0.27	0.15	0.15
query31	2.77	0.56	0.31
query32	3.22	0.58	0.50
query33	3.11	3.12	3.16
query34	15.51	3.89	3.28
query35	3.21	3.21	3.19
query36	0.56	0.43	0.44
query37	0.09	0.07	0.06
query38	0.06	0.03	0.03
query39	0.03	0.03	0.03
query40	0.17	0.15	0.13
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 95.97 s
Total hot run time: 23.7 s

hello-stephen · 2026-08-06T16:11:52Z

FE Regression Coverage Report

Increment line coverage 12.50% (43/344) 🎉
Increment coverage report
Complete coverage report

englefly · 2026-08-08T00:29:39Z

 }
+ boolean needDifferentExprId = aggFunctions.size() != context.getAliasMap().size();


这一轮遍历可以省略, 判断去重可以在 L429 这个 for 循环里一次完成

feiniaofeiafei · 2026-08-10T03:26:38Z

run buildall

hello-stephen · 2026-08-10T04:58:15Z

FE UT Coverage Report

Increment line coverage 54.35% (25/46) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-08-10T06:05:56Z

FE Regression Coverage Report

Increment line coverage 6.52% (46/706) 🎉
Increment coverage report
Complete coverage report

feiniaofeiafei · 2026-08-10T08:09:46Z

run performance

feiniaofeiafei · 2026-08-10T08:21:50Z

run performance

feiniaofeiafei · 2026-08-10T09:30:51Z

run performance

feiniaofeiafei · 2026-08-10T09:56:32Z

run performance

feiniaofeiafei · 2026-08-10T11:21:26Z

run performance

hello-stephen · 2026-08-10T12:43:58Z

TPC-H: Total hot run time: 29318 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c386135e62e4db7e0eb159bd28de948a4bbced6f, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17626	4006	3996	3996
q2	2052	325	199	199
q3	10317	1434	814	814
q4	4683	472	350	350
q5	7496	838	550	550
q6	195	177	142	142
q7	755	824	596	596
q8	9323	1526	1565	1526
q9	5296	4090	4096	4090
q10	6769	1631	1350	1350
q11	497	352	335	335
q12	745	595	458	458
q13	18163	3232	2760	2760
q14	262	255	243	243
q15	q16	729	731	659	659
q17	1042	1009	949	949
q18	6553	5646	5557	5557
q19	1312	1210	1054	1054
q20	836	671	605	605
q21	6467	2867	2768	2768
q22	469	377	317	317
Total cold run time: 101587 ms
Total hot run time: 29318 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5050	4736	4816	4736
q2	287	323	212	212
q3	4967	5202	4728	4728
q4	2250	2295	1433	1433
q5	4759	4481	4417	4417
q6	243	188	139	139
q7	1905	1766	1472	1472
q8	2360	2089	2069	2069
q9	7166	6792	6693	6693
q10	4229	4193	3818	3818
q11	513	375	345	345
q12	692	706	494	494
q13	2998	3276	2776	2776
q14	279	286	242	242
q15	q16	660	682	610	610
q17	1264	1225	1235	1225
q18	12036	11032	11746	11032
q19	1107	1037	1035	1035
q20	2197	2185	1912	1912
q21	5326	4485	4635	4485
q22	536	447	414	414
Total cold run time: 60824 ms
Total hot run time: 54287 ms

hello-stephen · 2026-08-10T12:54:16Z

TPC-DS: Total hot run time: 159105 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c386135e62e4db7e0eb159bd28de948a4bbced6f, data reload: false
query5	4331	597	479	479
query6	482	228	229	228
query7	4844	573	343	343
query8	326	174	150	150
query9	8784	4060	4098	4060
query10	509	353	321	321
query11	5826	2230	2077	2077
query12	154	103	101	101
query13	1256	614	435	435
query14	6084	4468	4117	4117
query14_1	3913	3961	3963	3961
query15	205	196	184	184
query16	1018	523	457	457
query17	938	688	540	540
query18	2417	460	330	330
query19	204	195	143	143
query20	105	100	99	99
query21	232	157	137	137
query22	13101	13010	12875	12875
query23	15770	15061	14502	14502
query23_1	14591	14756	14581	14581
query24	7598	1721	1230	1230
query24_1	1248	1260	1278	1260
query25	527	433	352	352
query26	1310	366	214	214
query27	2590	610	382	382
query28	4584	2058	2026	2026
query29	1023	661	479	479
query30	345	269	226	226
query31	1199	1123	1063	1063
query32	116	63	61	61
query33	522	303	242	242
query34	1184	1116	666	666
query35	731	746	641	641
query36	763	779	704	704
query37	157	105	96	96
query38	1836	1786	1706	1706
query39	826	826	788	788
query39_1	771	785	784	784
query40	259	163	143	143
query41	69	64	67	64
query42	96	98	94	94
query43	327	326	283	283
query44	1460	779	773	773
query45	186	186	168	168
query46	1079	1191	746	746
query47	1560	1545	1444	1444
query48	404	441	282	282
query49	578	417	299	299
query50	1084	439	346	346
query51	10612	10411	10756	10411
query52	91	87	77	77
query53	260	295	195	195
query54	293	231	231	231
query55	78	75	66	66
query56	310	299	294	294
query57	1022	996	915	915
query58	298	277	257	257
query59	1575	1617	1396	1396
query60	321	312	273	273
query61	180	175	183	175
query62	411	329	281	281
query63	254	209	203	203
query64	3002	1178	991	991
query65	3883	3768	3832	3768
query66	1861	477	401	401
query67	20014	20041	19978	19978
query68	3304	1673	1035	1035
query69	424	329	291	291
query70	878	813	818	813
query71	377	354	337	337
query72	3191	2669	2332	2332
query73	863	769	416	416
query74	4646	4496	4300	4300
query75	2384	2358	1993	1993
query76	2327	1194	795	795
query77	348	373	283	283
query78	11283	11142	10513	10513
query79	1417	1142	758	758
query80	1132	541	472	472
query81	514	332	287	287
query82	620	176	139	139
query83	452	325	310	310
query84	328	164	132	132
query85	988	656	511	511
query86	375	245	232	232
query87	1959	1947	1856	1856
query88	3726	2827	2807	2807
query89	399	327	288	288
query90	1835	206	201	201
query91	202	191	161	161
query92	66	62	56	56
query93	1651	1547	973	973
query94	633	361	306	306
query95	807	598	512	512
query96	1105	803	371	371
query97	2495	2478	2349	2349
query98	202	191	185	185
query99	737	729	609	609
Total cold run time: 245761 ms
Total hot run time: 159105 ms

hello-stephen · 2026-08-10T12:59:13Z

ClickBench: Total hot run time: 23.91 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c386135e62e4db7e0eb159bd28de948a4bbced6f, data reload: false
query1	0.01	0.00	0.01
query2	0.09	0.05	0.05
query3	0.25	0.13	0.13
query4	1.61	0.15	0.13
query5	0.24	0.22	0.21
query6	1.16	0.84	0.81
query7	0.04	0.01	0.00
query8	0.05	0.03	0.04
query9	0.37	0.31	0.32
query10	0.55	0.57	0.57
query11	0.19	0.14	0.13
query12	0.18	0.14	0.14
query13	0.46	0.47	0.46
query14	1.00	1.01	0.98
query15	0.60	0.59	0.58
query16	0.33	0.32	0.32
query17	1.08	1.08	1.08
query18	0.21	0.20	0.20
query19	1.98	2.01	1.92
query20	0.02	0.01	0.02
query21	15.45	0.18	0.15
query22	5.02	0.05	0.06
query23	16.12	0.32	0.12
query24	2.96	0.41	0.33
query25	0.11	0.05	0.04
query26	0.72	0.20	0.16
query27	0.04	0.03	0.04
query28	3.52	0.74	0.36
query29	12.52	4.04	3.19
query30	0.27	0.16	0.17
query31	2.77	0.58	0.31
query32	3.23	0.59	0.49
query33	3.18	3.24	3.20
query34	15.54	3.90	3.27
query35	3.23	3.24	3.20
query36	0.54	0.45	0.43
query37	0.09	0.07	0.07
query38	0.06	0.04	0.03
query39	0.03	0.03	0.03
query40	0.18	0.15	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 96.17 s
Total hot run time: 23.91 s

feiniaofeiafei · 2026-08-11T10:33:58Z

run buildall

hello-stephen · 2026-08-11T11:28:38Z

TPC-H: Total hot run time: 29000 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 72aa4d60c98f8f1edbe4015bdc8295d3d1a185f0, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17765	4009	3972	3972
q2	2064	330	207	207
q3	10290	1432	821	821
q4	4675	471	343	343
q5	7514	833	579	579
q6	183	168	135	135
q7	739	794	600	600
q8	9319	1544	1514	1514
q9	5295	4063	4027	4027
q10	6728	1630	1354	1354
q11	508	374	324	324
q12	727	573	454	454
q13	18048	3223	2736	2736
q14	263	260	241	241
q15	q16	742	738	665	665
q17	1035	1058	921	921
q18	6628	5633	5577	5577
q19	1165	1280	1026	1026
q20	766	657	632	632
q21	6080	2915	2553	2553
q22	453	365	319	319
Total cold run time: 100987 ms
Total hot run time: 29000 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4878	4642	4642	4642
q2	291	324	211	211
q3	4870	5265	4721	4721
q4	2174	2221	1418	1418
q5	4478	4628	4344	4344
q6	241	176	129	129
q7	1840	1728	1460	1460
q8	2349	2014	1992	1992
q9	7066	6854	6656	6656
q10	4243	4211	3772	3772
q11	504	367	341	341
q12	703	710	485	485
q13	2942	3246	2720	2720
q14	272	276	244	244
q15	q16	650	676	589	589
q17	1234	1211	1205	1205
q18	12127	10992	11733	10992
q19	1103	1085	1071	1071
q20	2207	2178	1889	1889
q21	5192	4468	4507	4468
q22	509	442	397	397
Total cold run time: 59873 ms
Total hot run time: 53746 ms

hello-stephen · 2026-08-11T11:38:51Z

TPC-DS: Total hot run time: 157959 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 72aa4d60c98f8f1edbe4015bdc8295d3d1a185f0, data reload: false
query5	4298	581	440	440
query6	458	215	201	201
query7	4843	582	347	347
query8	324	176	147	147
query9	8813	4034	4008	4008
query10	504	363	315	315
query11	5836	2191	2017	2017
query12	158	100	91	91
query13	1249	583	424	424
query14	6062	4235	3954	3954
query14_1	3760	3793	3781	3781
query15	207	193	174	174
query16	978	459	476	459
query17	910	684	550	550
query18	2430	471	348	348
query19	202	192	141	141
query20	103	100	104	100
query21	224	150	131	131
query22	13027	12992	12787	12787
query23	16248	15008	14561	14561
query23_1	14551	14624	14792	14624
query24	7482	1686	1227	1227
query24_1	1234	1243	1220	1220
query25	517	419	344	344
query26	1307	356	203	203
query27	2595	621	384	384
query28	4590	2009	1990	1990
query29	1095	606	472	472
query30	342	262	219	219
query31	1176	1112	1052	1052
query32	108	62	62	62
query33	510	302	232	232
query34	1212	1099	631	631
query35	722	736	628	628
query36	778	777	748	748
query37	170	118	101	101
query38	1826	1778	1668	1668
query39	830	815	789	789
query39_1	800	774	809	774
query40	249	165	139	139
query41	75	78	80	78
query42	94	93	91	91
query43	314	315	274	274
query44	1410	779	763	763
query45	184	174	168	168
query46	1026	1148	739	739
query47	1557	1591	1497	1497
query48	403	399	286	286
query49	591	432	307	307
query50	1099	434	342	342
query51	10686	10696	10598	10598
query52	93	89	75	75
query53	254	289	202	202
query54	289	249	249	249
query55	75	75	69	69
query56	337	309	302	302
query57	1070	1024	969	969
query58	297	264	261	261
query59	1559	1591	1371	1371
query60	346	262	245	245
query61	151	152	150	150
query62	389	318	268	268
query63	238	197	198	197
query64	2845	1003	930	930
query65	3880	3805	3802	3802
query66	1835	461	360	360
query67	20085	20016	19994	19994
query68	3105	1473	900	900
query69	407	306	252	252
query70	872	803	780	780
query71	375	335	312	312
query72	2946	2649	2366	2366
query73	825	837	462	462
query74	4594	4511	4310	4310
query75	2351	2313	1974	1974
query76	2336	1125	774	774
query77	332	400	277	277
query78	10939	11009	10600	10600
query79	1240	1128	725	725
query80	618	546	442	442
query81	464	328	289	289
query82	276	164	135	135
query83	405	323	293	293
query84	324	164	130	130
query85	918	636	501	501
query86	280	232	220	220
query87	1954	1967	1824	1824
query88	3688	2778	2749	2749
query89	372	316	276	276
query90	2043	195	196	195
query91	200	197	165	165
query92	64	58	57	57
query93	1501	1470	1023	1023
query94	526	343	306	306
query95	781	588	479	479
query96	1066	782	344	344
query97	2504	2448	2332	2332
query98	192	190	182	182
query99	727	724	647	647
Total cold run time: 243400 ms
Total hot run time: 157959 ms

hello-stephen · 2026-08-11T11:43:47Z

ClickBench: Total hot run time: 23.63 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 72aa4d60c98f8f1edbe4015bdc8295d3d1a185f0, data reload: false
query1	0.00	0.00	0.01
query2	0.10	0.05	0.05
query3	0.26	0.12	0.12
query4	1.61	0.13	0.13
query5	0.24	0.21	0.22
query6	1.16	0.84	0.80
query7	0.04	0.01	0.01
query8	0.05	0.03	0.03
query9	0.36	0.30	0.32
query10	0.53	0.52	0.54
query11	0.18	0.13	0.13
query12	0.18	0.13	0.14
query13	0.45	0.46	0.45
query14	1.00	0.99	0.97
query15	0.59	0.57	0.57
query16	0.31	0.31	0.32
query17	1.08	1.14	1.09
query18	0.20	0.19	0.19
query19	2.06	1.91	1.93
query20	0.01	0.01	0.01
query21	15.44	0.18	0.15
query22	5.08	0.05	0.05
query23	16.11	0.30	0.12
query24	2.97	0.41	0.32
query25	0.11	0.05	0.04
query26	0.73	0.21	0.14
query27	0.04	0.04	0.03
query28	3.55	0.82	0.35
query29	12.49	4.04	3.19
query30	0.27	0.16	0.15
query31	2.77	0.55	0.32
query32	3.23	0.59	0.49
query33	3.15	3.21	3.11
query34	15.61	3.94	3.26
query35	3.22	3.21	3.22
query36	0.55	0.42	0.42
query37	0.09	0.07	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.09	0.03	0.04
query42	0.05	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 96.27 s
Total hot run time: 23.63 s

hello-stephen · 2026-08-11T12:27:06Z

FE UT Coverage Report

Increment line coverage 86.21% (25/29) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-08-11T13:41:18Z

FE Regression Coverage Report

Increment line coverage 74.36% (29/39) 🎉
Increment coverage report
Complete coverage report

…ed through projects #66531 (#66741) picked from #66531

…projects (apache#66531) ### What problem does this PR solve? Related PR: apache#63690 Problem Summary: Eager aggregation pushdown may fail when different aggregate functions become the same expression after passing through a Project. report error: ```text 2026-08-06 04:26:54,285 INFO (mysql-nio-pool-14|325) [PushDownAggregation.visitLogicalAggregate():280] PushDownAggregation failed: Cannot invoke "org.apache.doris.nereids.trees.expressions.NamedExpression.toSlot()" because "namedExpression" is null at org.apache.doris.nereids.rules.rewrite.eageraggregation.EagerAggRewriter.visitLogicalProject(EagerAggRewriter.java:718) at org.apache.doris.nereids.rules.rewrite.eageraggregation.EagerAggRewriter.visitLogicalProject(EagerAggRewriter.java:90) at org.apache.doris.nereids.trees.plans.logical.LogicalProject.accept(LogicalProject.java:160) at org.apache.doris.nereids.rules.rewrite.eageraggregation.EagerAggRewriter.visitLogicalUnion(EagerAggRewriter.java:582) at org.apache.doris.nereids.rules.rewrite.eageraggregation.EagerAggRewriter.visitLogicalUnion(EagerAggRewriter.java:90) at org.apache.doris.nereids.trees.plans.logical.LogicalUnion.accept(LogicalUnion.java:155) ``` For example: ```text Aggregate: SUM(x)#4, SUM(y)apache#5 Union All Project: 0 AS x, 0 AS y Join ``` After pushing the aggregates through the Project, both functions become`SUM(0)`: ```text functions: [SUM(0), SUM(0)] aliasMap: SUM(0) -> apache#5 ``` Because `aliasMap` uses expression equality, only one entry is retained. The Project still tries to read both `#4` and `apache#5` from `BilateralState`, causing a null lookup. This PR deduplicates the child aggregate and records the ExprId mapping: ```text child aggregate: SUM(0) -> apache#8 ExprId mapping: #4 -> apache#8, apache#5 -> apache#8 ``` The Project then restores both required outputs: ```text slot#8 AS slot#4 slot#8 AS slot#5 ``` When no aggregate functions are merged, the original ExprIds are reused to avoid unnecessary aliases. The same fix also covers cases such as: ```text Project: a#1 AS x, a#1 AS y ``` where `SUM(x)` and `SUM(y)` both become `SUM(a#1)` after pushdown. ### Release note None ### Check List (For Author) - Test - [x] Regression test - `query_p0/eager_agg/bilateral_eager_agg` - Covers two aggregate functions that become the same function after Project pushdown. - [ ] Unit Test - [ ] Manual test - [ ] No need to test or manual test. - Behavior changed: - [x] Yes. - Prevents eager aggregation pushdown from failing when multiple aggregate functions become identical after Project rewriting. - `eager_aggregation_mode=1` can force eligible pushdown after a UNION. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label

feiniaofeiafei added 2 commits August 6, 2026 14:27

use identifyMap and add test
d999fb5

add test output
c52b6e7

feiniaofeiafei requested review from 924060929, englefly, morrySnow and starocean999 as code owners August 6, 2026 06:30

feiniaofeiafei added 2 commits August 6, 2026 21:22

use map in eageragg visitlogicalproject
eccaaf8

fix test
b3d4f0d

feiniaofeiafei changed the title ~~[fix](eager-agg) Preserve identical aggregate functions in eager aggregation~~[fix](eager-agg) Handle duplicate aggregate functions pushed through projectsAug 7, 2026

englefly reviewed Aug 8, 2026
View reviewed changes

reduce for loop times
c386135

use one for loop
72aa4d6

englefly approved these changes Aug 12, 2026
View reviewed changes

englefly merged commit c45d1a6 into apache:masterAug 12, 2026
30 of 31 checks passed

feiniaofeiafei added dev/4.1.x dev/4.2.x labels Aug 12, 2026

github-actionsBot added the dev/4.1.x-conflict label Aug 12, 2026

morrySnow removed the dev/4.2.x label Aug 13, 2026

feiniaofeiafei mentioned this pull request Aug 13, 2026
branch-4.1:[fix](eager-agg) Handle duplicate aggregate functions pushed through projects #66531 #66741
Merged

yiguolei pushed a commit that referenced this pull request Aug 14, 2026
branch-4.1:[fix](eager-agg) Handle duplicate aggregate functions push…
f2aeb63
…ed through projects #66531 (#66741) picked from #66531

yiguolei added dev/4.1.4-merged and removed dev/4.1.x dev/4.1.x-conflict labels Aug 14, 2026

yiguolei mentioned this pull request Sep 1, 2026
4.1.4 Release Note #67355
Open

		}
		boolean needDifferentExprId = aggFunctions.size() != context.getAliasMap().size();

Uh oh!

Conversation

feiniaofeiafei commented Aug 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Aug 6, 2026

Uh oh!

feiniaofeiafei commented Aug 6, 2026

Uh oh!

hello-stephen commented Aug 6, 2026

FE Regression Coverage Report

Uh oh!

feiniaofeiafei commented Aug 6, 2026

Uh oh!

hello-stephen commented Aug 6, 2026

Uh oh!

hello-stephen commented Aug 6, 2026

Uh oh!

hello-stephen commented Aug 6, 2026

Uh oh!

hello-stephen commented Aug 6, 2026

FE Regression Coverage Report

Uh oh!

engleflyAug 8, 2026

Choose a reason for hiding this comment

Uh oh!

feiniaofeiafei commented Aug 10, 2026

Uh oh!

hello-stephen commented Aug 10, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Aug 10, 2026

FE Regression Coverage Report

Uh oh!

feiniaofeiafei commented Aug 10, 2026

Uh oh!

feiniaofeiafei commented Aug 10, 2026

Uh oh!

feiniaofeiafei commented Aug 10, 2026

Uh oh!

feiniaofeiafei commented Aug 10, 2026

Uh oh!

feiniaofeiafei commented Aug 10, 2026

Uh oh!

hello-stephen commented Aug 10, 2026

Uh oh!

hello-stephen commented Aug 10, 2026

Uh oh!

hello-stephen commented Aug 10, 2026

Uh oh!

feiniaofeiafei commented Aug 11, 2026

Uh oh!

hello-stephen commented Aug 11, 2026

Uh oh!

hello-stephen commented Aug 11, 2026

Uh oh!

hello-stephen commented Aug 11, 2026

Uh oh!

hello-stephen commented Aug 11, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Aug 11, 2026

FE Regression Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

feiniaofeiafei commented Aug 6, 2026•
edited
Loading