[fix](nereids) runtime filter generation bug for topn lazy materialization by englefly · Pull Request #56137 · apache/doris

englefly · 2025-09-17T06:17:28Z

What problem does this PR solve?

Previous pr (topn lazy materialization, #51329 commit id a4b5008) introduces a bug in runtime filter target translation.
the runtime filter target should be basased on PhysicalLazyMaterializeOlapScan, not the inner PhysicalOlapScan.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-09-17T06:17:33Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

englefly · 2025-09-17T06:17:37Z

run buildall

github-actions · 2025-09-17T06:25:11Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-09-17T06:25:14Z

PR approved by anyone and no changes requested.

doris-robot · 2025-09-17T06:50:09Z

TPC-H: Total hot run time: 34754 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0fdf51fea52cdc84e610c330d7047a56563ebbb9, data reload: false
------ Round 1 ----------------------------------
q1	17596	5194	5031	5031
q2	1988	319	213	213
q3	10292	1312	716	716
q4	10242	1018	523	523
q5	7533	2436	2324	2324
q6	182	165	137	137
q7	935	783	637	637
q8	9371	1339	1151	1151
q9	6952	5201	5195	5195
q10	6950	2377	1968	1968
q11	513	320	281	281
q12	361	363	229	229
q13	17789	3593	3024	3024
q14	251	227	221	221
q15	570	498	494	494
q16	1024	985	935	935
q17	602	860	367	367
q18	7484	7346	7142	7142
q19	1533	935	568	568
q20	337	337	230	230
q21	3836	3173	2386	2386
q22	1066	1036	982	982
Total cold run time: 107407 ms
Total hot run time: 34754 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5207	5091	5109	5091
q2	252	338	230	230
q3	2187	2658	2280	2280
q4	1324	1806	1349	1349
q5	4189	4470	4597	4470
q6	228	187	142	142
q7	2006	1980	1842	1842
q8	2653	2470	2595	2470
q9	7422	7319	7445	7319
q10	3089	3372	2898	2898
q11	604	552	512	512
q12	675	770	622	622
q13	3520	3989	3476	3476
q14	296	304	268	268
q15	543	473	470	470
q16	1045	1136	1072	1072
q17	1125	1572	1474	1474
q18	8042	7805	7595	7595
q19	812	836	974	836
q20	1936	1954	1810	1810
q21	4766	4378	4330	4330
q22	1097	1031	1011	1011
Total cold run time: 53018 ms
Total hot run time: 51567 ms

doris-robot · 2025-09-17T07:01:49Z

TPC-DS: Total hot run time: 188311 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0fdf51fea52cdc84e610c330d7047a56563ebbb9, data reload: false
query1	1119	433	406	406
query2	6579	1676	1690	1676
query3	6751	220	226	220
query4	26116	23168	22758	22758
query5	4372	618	488	488
query6	318	234	212	212
query7	4648	505	296	296
query8	307	249	245	245
query9	9163	2587	2607	2587
query10	507	388	293	293
query11	15295	14984	14719	14719
query12	163	115	111	111
query13	1681	587	422	422
query14	10177	9291	9194	9194
query15	221	192	177	177
query16	7308	691	487	487
query17	1226	738	609	609
query18	2002	448	331	331
query19	202	209	180	180
query20	132	130	118	118
query21	216	133	118	118
query22	4317	4424	4190	4190
query23	33662	32825	32636	32636
query24	8457	2398	2415	2398
query25	581	505	431	431
query26	1233	271	161	161
query27	2746	512	364	364
query28	4365	2222	2207	2207
query29	809	606	508	508
query30	296	229	232	229
query31	876	824	752	752
query32	76	73	72	72
query33	574	372	318	318
query34	791	851	524	524
query35	814	847	756	756
query36	986	1032	919	919
query37	124	120	88	88
query38	3499	3519	3534	3519
query39	1642	1438	1442	1438
query40	217	133	126	126
query41	68	62	68	62
query42	128	115	113	113
query43	527	508	488	488
query44	1327	843	876	843
query45	185	184	172	172
query46	867	1036	639	639
query47	1780	1838	1778	1778
query48	429	431	335	335
query49	786	526	415	415
query50	713	739	427	427
query51	3884	4042	3923	3923
query52	129	131	121	121
query53	277	294	223	223
query54	693	666	619	619
query55	100	94	98	94
query56	367	377	334	334
query57	1244	1223	1161	1161
query58	294	303	302	302
query59	2680	2900	2596	2596
query60	365	364	356	356
query61	198	193	187	187
query62	807	753	691	691
query63	236	193	194	193
query64	4643	1160	843	843
query65	4071	4015	3979	3979
query66	1168	435	335	335
query67	15351	15662	15127	15127
query68	7911	922	586	586
query69	493	317	277	277
query70	1369	1297	1231	1231
query71	580	333	319	319
query72	6079	5127	5180	5127
query73	687	654	361	361
query74	8982	9156	8897	8897
query75	3545	3323	2860	2860
query76	3321	1173	756	756
query77	581	414	309	309
query78	9603	9738	8896	8896
query79	1635	839	602	602
query80	633	662	507	507
query81	501	269	224	224
query82	240	166	134	134
query83	268	273	282	273
query84	262	117	104	104
query85	864	465	420	420
query86	374	334	323	323
query87	3734	3777	3620	3620
query88	2906	2212	2200	2200
query89	385	342	298	298
query90	1993	212	214	212
query91	166	163	133	133
query92	80	66	64	64
query93	1873	985	647	647
query94	684	412	349	349
query95	454	312	309	309
query96	490	558	276	276
query97	2907	3012	2868	2868
query98	251	218	209	209
query99	1339	1418	1281	1281
Total cold run time: 273075 ms
Total hot run time: 188311 ms

doris-robot · 2025-09-17T07:07:18Z

ClickBench: Total hot run time: 29.51 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0fdf51fea52cdc84e610c330d7047a56563ebbb9, data reload: false
query1	0.06	0.05	0.05
query2	0.09	0.05	0.05
query3	0.25	0.10	0.08
query4	1.61	0.12	0.12
query5	0.28	0.26	0.24
query6	1.18	0.66	0.64
query7	0.03	0.02	0.03
query8	0.06	0.04	0.05
query9	0.61	0.53	0.52
query10	0.59	0.58	0.57
query11	0.16	0.12	0.11
query12	0.17	0.12	0.12
query13	0.62	0.62	0.62
query14	1.02	1.03	1.04
query15	0.87	0.88	0.87
query16	0.42	0.40	0.39
query17	1.02	1.07	1.03
query18	0.23	0.20	0.21
query19	1.99	1.88	1.81
query20	0.02	0.02	0.02
query21	15.43	0.92	0.58
query22	0.77	1.20	0.65
query23	14.96	1.40	0.60
query24	7.05	1.04	0.43
query25	0.49	0.16	0.07
query26	0.63	0.17	0.14
query27	0.07	0.06	0.05
query28	9.51	0.94	0.45
query29	12.57	3.90	3.28
query30	0.28	0.13	0.11
query31	2.82	0.61	0.40
query32	3.24	0.57	0.48
query33	3.16	3.10	3.15
query34	16.20	5.52	4.84
query35	4.95	4.91	4.98
query36	0.69	0.51	0.53
query37	0.11	0.08	0.07
query38	0.07	0.05	0.05
query39	0.03	0.03	0.03
query40	0.18	0.15	0.15
query41	0.10	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 104.67 s
Total hot run time: 29.51 s

hello-stephen · 2025-09-17T09:06:04Z

FE Regression Coverage Report

Increment line coverage 90.48% (19/21) 🎉
Increment coverage report
Complete coverage report

yujun777

LGTM

…ation (apache#56137) ### What problem does this PR solve? Previous pr (topn lazy materialization, apache#51329 commit id a4b5008) introduces a bug in runtime filter target translation. the runtime filter target should be basased on PhysicalLazyMaterializeOlapScan, not the inner PhysicalOlapScan.

runtime filter generation bug for topn lazy materialize
0fdf51f

morrySnow approved these changes Sep 17, 2025
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Sep 17, 2025

yujun777 approved these changes Sep 18, 2025
View reviewed changes

morrySnow merged commit 7f9b130 into apache:masterSep 19, 2025
29 of 31 checks passed

morrySnow added the dev/4.0.0-merged label Jun 9, 2026

Navigation Menu

Uh oh!

[fix](nereids) runtime filter generation bug for topn lazy materialization - #56137

[fix](nereids) runtime filter generation bug for topn lazy materialization#56137
morrySnow merged 1 commit into
apache:masterfrom
englefly:lazy-rf

englefly commented Sep 17, 2025•
edited by morrySnow
Loading

Uh oh!

hello-stephen commented Sep 17, 2025

Uh oh!

englefly commented Sep 17, 2025

Uh oh!

github-actionsBot commented Sep 17, 2025

Uh oh!

github-actionsBot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

hello-stephen commented Sep 17, 2025

Uh oh!

yujun777 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

englefly commented Sep 17, 2025• edited by morrySnow Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Sep 17, 2025

Uh oh!

englefly commented Sep 17, 2025

Uh oh!

github-actionsBot commented Sep 17, 2025

Uh oh!

github-actionsBot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

doris-robot commented Sep 17, 2025

Uh oh!

hello-stephen commented Sep 17, 2025

FE Regression Coverage Report

Uh oh!

yujun777 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

englefly commented Sep 17, 2025•
edited by morrySnow
Loading