[fix](topnFilter)Fix TopN filter probe expressions to wrap nullable slots when pushed through outer joins. by englefly · Pull Request #59074 · apache/doris

englefly · 2025-12-16T06:06:35Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-12-16T06:06:41Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

englefly · 2025-12-22T07:05:51Z

run buildall

englefly · 2025-12-22T07:11:30Z

run buildall

hello-stephen · 2025-12-22T10:16:44Z

FE Regression Coverage Report

Increment line coverage 100.00% (19/19) 🎉
Increment coverage report
Complete coverage report

englefly · 2025-12-25T13:36:15Z

run buildall

doris-robot · 2025-12-25T15:33:22Z

TPC-H: Total hot run time: 35198 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f726423894c5d0a457dd3f3859c8ce6ad987af2a, data reload: false
------ Round 1 ----------------------------------
q1	17630	4148	4067	4067
q2	2028	349	238	238
q3	10163	1316	747	747
q4	10231	871	319	319
q5	7632	2154	1945	1945
q6	249	170	134	134
q7	1000	863	703	703
q8	9370	1451	1169	1169
q9	7126	5370	5372	5370
q10	6883	2395	1981	1981
q11	516	331	312	312
q12	689	746	554	554
q13	17784	3656	3043	3043
q14	285	306	288	288
q15	598	531	514	514
q16	698	694	642	642
q17	703	807	512	512
q18	8033	7138	6999	6999
q19	1097	959	608	608
q20	407	374	247	247
q21	4281	3938	3851	3851
q22	1013	1009	955	955
Total cold run time: 108416 ms
Total hot run time: 35198 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4125	4058	4055	4055
q2	320	396	338	338
q3	2165	2651	2335	2335
q4	1355	1789	1285	1285
q5	4231	4772	4792	4772
q6	226	173	127	127
q7	2218	1961	1850	1850
q8	2709	2521	2510	2510
q9	7517	7597	7562	7562
q10	3128	3246	2849	2849
q11	643	548	519	519
q12	663	888	637	637
q13	3654	3927	3400	3400
q14	322	322	302	302
q15	577	544	521	521
q16	662	715	629	629
q17	1207	1439	1414	1414
q18	7808	7783	7652	7652
q19	908	868	910	868
q20	2049	2077	1907	1907
q21	4708	4239	4256	4239
q22	1078	1041	972	972
Total cold run time: 52273 ms
Total hot run time: 50743 ms

doris-robot · 2025-12-25T15:44:32Z

TPC-DS: Total hot run time: 179967 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f726423894c5d0a457dd3f3859c8ce6ad987af2a, data reload: false
query5	4429	604	425	425
query6	360	223	209	209
query7	4233	459	271	271
query8	313	266	261	261
query9	8785	2526	2550	2526
query10	500	382	342	342
query11	15355	15087	14793	14793
query12	184	123	121	121
query13	1276	498	400	400
query14	5527	3026	2759	2759
query14_1	2660	2634	2640	2634
query15	218	196	176	176
query16	806	472	473	472
query17	1161	752	602	602
query18	2452	448	355	355
query19	235	231	222	222
query20	119	116	120	116
query21	229	148	124	124
query22	4001	4022	3968	3968
query23	16631	16264	16074	16074
query23_1	16102	16237	16145	16145
query24	7335	1643	1234	1234
query24_1	1250	1218	1259	1218
query25	598	514	452	452
query26	1252	282	165	165
query27	2748	473	315	315
query28	4484	2137	2127	2127
query29	842	578	477	477
query30	325	244	223	223
query31	829	703	640	640
query32	81	75	72	72
query33	563	359	304	304
query34	917	901	561	561
query35	759	825	708	708
query36	861	914	887	887
query37	131	96	76	76
query38	2962	3080	2976	2976
query39	750	748	727	727
query39_1	731	711	697	697
query40	220	142	126	126
query41	67	63	65	63
query42	107	108	107	107
query43	437	425	411	411
query44	1349	742	733	733
query45	194	189	190	189
query46	865	977	620	620
query47	1703	1719	1633	1633
query48	312	327	257	257
query49	635	444	351	351
query50	655	292	231	231
query51	3811	3801	3830	3801
query52	107	108	101	101
query53	321	345	305	305
query54	287	272	251	251
query55	86	77	75	75
query56	295	293	297	293
query57	1175	1160	1108	1108
query58	275	253	257	253
query59	2537	2483	2412	2412
query60	316	319	294	294
query61	199	163	172	163
query62	720	730	686	686
query63	330	295	300	295
query64	5039	1322	1007	1007
query65	4124	3987	3983	3983
query66	1460	445	321	321
query67	15261	15142	14875	14875
query68	8324	1003	724	724
query69	505	358	313	313
query70	1083	1029	993	993
query71	356	304	280	280
query72	6134	5015	5077	5015
query73	692	610	314	314
query74	9006	8971	8858	8858
query75	3181	3146	2849	2849
query76	3828	1147	749	749
query77	535	398	301	301
query78	9418	9716	8828	8828
query79	1667	865	629	629
query80	770	645	555	555
query81	511	272	237	237
query82	203	133	102	102
query83	272	258	250	250
query84	262	117	102	102
query85	946	525	471	471
query86	380	291	274	274
query87	3182	3306	3107	3107
query88	3317	2325	2283	2283
query89	475	419	391	391
query90	2048	167	162	162
query91	185	177	145	145
query92	79	67	71	67
query93	1184	924	551	551
query94	497	326	281	281
query95	593	389	310	310
query96	601	478	218	218
query97	2264	2277	2198	2198
query98	217	205	199	199
query99	1333	1343	1277	1277
Total cold run time: 260268 ms
Total hot run time: 179967 ms

doris-robot · 2025-12-25T15:49:34Z

ClickBench: Total hot run time: 27.54 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f726423894c5d0a457dd3f3859c8ce6ad987af2a, data reload: false
query1	0.06	0.05	0.04
query2	0.11	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.12	0.11
query5	0.28	0.26	0.25
query6	1.17	0.64	0.64
query7	0.03	0.03	0.03
query8	0.06	0.05	0.04
query9	0.57	0.50	0.50
query10	0.56	0.55	0.55
query11	0.16	0.11	0.12
query12	0.15	0.12	0.13
query13	0.61	0.60	0.60
query14	0.99	0.99	0.97
query15	0.83	0.79	0.84
query16	0.40	0.39	0.42
query17	1.01	1.06	0.98
query18	0.24	0.22	0.21
query19	1.92	1.82	1.80
query20	0.01	0.01	0.01
query21	15.44	0.27	0.14
query22	4.77	0.06	0.05
query23	16.08	0.28	0.10
query24	1.45	0.63	0.44
query25	0.07	0.08	0.06
query26	0.13	0.13	0.13
query27	0.05	0.05	0.05
query28	3.93	1.22	1.02
query29	12.61	4.02	3.39
query30	0.28	0.14	0.12
query31	2.81	0.62	0.39
query32	3.23	0.54	0.45
query33	3.04	3.08	3.03
query34	16.90	5.19	4.50
query35	4.57	4.56	4.55
query36	0.67	0.50	0.48
query37	0.11	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.04	0.03
query43	0.05	0.04	0.03
Total cold run time: 97.62 s
Total hot run time: 27.54 s

hello-stephen · 2025-12-25T19:41:02Z

FE Regression Coverage Report

Increment line coverage 100.00% (20/20) 🎉
Increment coverage report
Complete coverage report

englefly · 2026-02-12T06:10:26Z

run buildall

hello-stephen · 2026-02-12T07:20:17Z

FE UT Coverage Report

Increment line coverage 80.00% (16/20) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2026-02-12T08:13:02Z

TPC-H: Total hot run time: 31808 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f726423894c5d0a457dd3f3859c8ce6ad987af2a, data reload: false
------ Round 1 ----------------------------------
q1	17664	5137	4963	4963
q2	2127	358	231	231
q3	10066	1391	766	766
q4	10193	826	317	317
q5	7760	2336	2096	2096
q6	251	198	151	151
q7	972	785	600	600
q8	9277	1570	1249	1249
q9	5139	4891	4892	4891
q10	6932	1973	1562	1562
q11	471	276	245	245
q12	402	410	228	228
q13	17801	4337	3251	3251
q14	247	247	222	222
q15	923	831	795	795
q16	706	677	646	646
q17	723	921	533	533
q18	7163	5915	5848	5848
q19	1504	1137	670	670
q20	526	506	391	391
q21	2682	2019	1908	1908
q22	350	296	245	245
Total cold run time: 103879 ms
Total hot run time: 31808 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5487	5421	5432	5421
q2	280	357	258	258
q3	2236	2756	2250	2250
q4	1474	1822	1332	1332
q5	4467	4352	4798	4352
q6	308	220	163	163
q7	2212	1964	1741	1741
q8	3023	2865	2809	2809
q9	7634	7776	7897	7776
q10	3083	3123	2748	2748
q11	563	453	431	431
q12	698	765	658	658
q13	4157	4776	3456	3456
q14	316	324	302	302
q15	915	840	806	806
q16	689	745	690	690
q17	1475	1686	1584	1584
q18	8244	8098	7866	7866
q19	1015	895	897	895
q20	2100	2231	1910	1910
q21	4965	4468	4418	4418
q22	522	456	433	433
Total cold run time: 55863 ms
Total hot run time: 52299 ms

doris-robot · 2026-02-12T08:24:30Z

TPC-DS: Total hot run time: 189266 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f726423894c5d0a457dd3f3859c8ce6ad987af2a, data reload: false
query5	4302	602	514	514
query6	345	225	212	212
query7	4230	468	273	273
query8	346	274	242	242
query9	8733	2757	2734	2734
query10	474	368	336	336
query11	17205	17021	16784	16784
query12	183	122	124	122
query13	1267	458	369	369
query14	6213	3220	3022	3022
query14_1	2828	2827	2805	2805
query15	206	197	174	174
query16	994	468	377	377
query17	1099	701	607	607
query18	2446	449	345	345
query19	209	212	190	190
query20	144	134	132	132
query21	229	148	126	126
query22	5066	5152	5164	5152
query23	17396	16865	16711	16711
query23_1	16784	16769	16903	16769
query24	7154	1616	1243	1243
query24_1	1228	1242	1263	1242
query25	544	456	424	424
query26	1293	260	152	152
query27	2781	455	303	303
query28	4516	1876	1865	1865
query29	773	538	447	447
query30	320	255	215	215
query31	874	743	646	646
query32	86	76	74	74
query33	521	334	290	290
query34	932	916	571	571
query35	636	680	607	607
query36	1091	1147	964	964
query37	137	100	85	85
query38	2958	2949	2886	2886
query39	868	841	821	821
query39_1	821	801	871	801
query40	220	135	119	119
query41	66	62	64	62
query42	110	104	108	104
query43	383	388	366	366
query44	1302	722	733	722
query45	198	191	181	181
query46	886	975	615	615
query47	2108	2170	2061	2061
query48	308	326	240	240
query49	624	443	357	357
query50	676	272	211	211
query51	4127	4192	4182	4182
query52	105	107	96	96
query53	301	339	286	286
query54	291	264	260	260
query55	90	81	85	81
query56	309	316	305	305
query57	1381	1315	1250	1250
query58	284	278	304	278
query59	2520	2677	2521	2521
query60	333	335	325	325
query61	156	149	152	149
query62	628	581	534	534
query63	310	284	277	277
query64	4834	1240	956	956
query65	4616	4515	4480	4480
query66	1446	458	351	351
query67	16366	16615	16444	16444
query68	2398	1080	722	722
query69	398	312	287	287
query70	1026	990	904	904
query71	331	328	286	286
query72	2950	2747	2283	2283
query73	521	547	318	318
query74	9612	9509	9389	9389
query75	2810	2749	2450	2450
query76	2301	1096	681	681
query77	352	377	320	320
query78	10986	11088	10502	10502
query79	1068	915	596	596
query80	676	560	509	509
query81	504	279	257	257
query82	1365	152	112	112
query83	361	252	240	240
query84	254	127	98	98
query85	821	474	419	419
query86	362	302	296	296
query87	3098	3111	3021	3021
query88	3561	2684	2663	2663
query89	430	393	355	355
query90	1855	181	174	174
query91	199	155	130	130
query92	79	74	72	72
query93	892	865	488	488
query94	446	317	308	308
query95	587	329	337	329
query96	643	511	228	228
query97	2524	2501	2392	2392
query98	229	215	209	209
query99	1004	1002	934	934
Total cold run time: 259566 ms
Total hot run time: 189266 ms

doris-robot · 2026-02-12T08:29:38Z

ClickBench: Total hot run time: 28.41 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f726423894c5d0a457dd3f3859c8ce6ad987af2a, data reload: false
query1	0.05	0.04	0.04
query2	0.09	0.04	0.04
query3	0.26	0.08	0.08
query4	1.60	0.12	0.10
query5	0.26	0.24	0.24
query6	1.16	0.67	0.68
query7	0.04	0.03	0.02
query8	0.05	0.04	0.04
query9	0.57	0.50	0.50
query10	0.56	0.55	0.56
query11	0.15	0.09	0.09
query12	0.14	0.11	0.11
query13	0.62	0.62	0.63
query14	1.08	1.06	1.06
query15	0.88	0.89	0.88
query16	0.39	0.42	0.42
query17	1.14	1.12	1.13
query18	0.22	0.21	0.20
query19	2.03	2.02	1.97
query20	0.02	0.01	0.02
query21	15.40	0.28	0.14
query22	5.39	0.05	0.05
query23	16.05	0.30	0.11
query24	2.10	0.25	0.19
query25	0.07	0.11	0.05
query26	0.17	0.15	0.14
query27	0.06	0.09	0.05
query28	4.17	1.16	0.97
query29	12.58	3.90	3.15
query30	0.28	0.13	0.12
query31	2.82	0.62	0.42
query32	3.24	0.60	0.50
query33	3.29	3.30	3.24
query34	16.42	5.39	4.82
query35	4.84	4.83	4.89
query36	0.64	0.50	0.48
query37	0.11	0.08	0.08
query38	0.08	0.05	0.05
query39	0.05	0.03	0.03
query40	0.20	0.16	0.15
query41	0.08	0.04	0.03
query42	0.04	0.03	0.02
query43	0.05	0.04	0.03
Total cold run time: 99.44 s
Total hot run time: 28.41 s

hello-stephen · 2026-02-12T11:27:23Z

FE Regression Coverage Report

Increment line coverage 100.00% (20/20) 🎉
Increment coverage report
Complete coverage report

englefly · 2026-02-13T01:41:43Z

run external

englefly · 2026-02-13T01:42:22Z

run nonConcurrent

hello-stephen · 2026-02-13T03:58:42Z

FE Regression Coverage Report

Increment line coverage 100.00% (20/20) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-02-13T04:13:32Z

FE Regression Coverage Report

Increment line coverage 100.00% (20/20) 🎉
Increment coverage report
Complete coverage report

github-actions · 2026-02-13T06:55:14Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-02-13T06:55:16Z

PR approved by anyone and no changes requested.

…lots when pushed through outer joins. (#59074) ### What problem does this PR solve? Fix TopN filter probe expressions to wrap nullable slots when pushed through outer joins. (#59074)

…p nullable slots when pushed through outer joins. #59074 (#61268) Cherry-picked from #59074 Co-authored-by: minghong <zhouminghong@selectdb.com>

fix
fc3f827

englefly force-pushed the topn-filter-expr branch from c5a97b3 to fc3f827Compare December 22, 2025 03:10

englefly added 2 commits December 22, 2025 14:48

fix
6a23122

fix
8c04bed

englefly changed the title ~~Topn filter expr~~[fix](topnFilter)Fix TopN filter probe expressions to wrap nullable slots when pushed through outer joins.Dec 22, 2025

englefly marked this pull request as ready for review December 22, 2025 07:05

fmt
2d8ee30

fix
f726423

starocean999 approved these changes Feb 13, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 13, 2026

github-actionsBot added the reviewed label Feb 13, 2026

zclllyybb approved these changes Feb 13, 2026
View reviewed changes

englefly merged commit 67019ff into apache:masterFeb 13, 2026
29 of 30 checks passed

englefly deleted the topn-filter-expr branch February 13, 2026 06:57

morrySnow added dev/4.0.x dev/4.1.x labels Mar 12, 2026

github-actionsBot mentioned this pull request Mar 12, 2026
branch-4.0: [fix](topnFilter)Fix TopN filter probe expressions to wrap nullable slots when pushed through outer joins. #59074 #61268
Merged

yiguolei added dev/4.0.5-merged dev/4.1.0-merged and removed dev/4.0.x dev/4.1.x labels Mar 13, 2026

yiguolei mentioned this pull request Mar 27, 2026
4.0.5 Release Notes #61817
Closed

turbo-one-off mentioned this pull request Apr 18, 2026
[Bug] SELECT * FROM view ORDER BY ... LIMIT returns NULL for non-key/non-orderby columns (TopN lazy materialization) #62598
Open
3 tasks

Uh oh!

Conversation

englefly commented Dec 16, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Dec 16, 2025

Uh oh!

englefly commented Dec 22, 2025

Uh oh!

englefly commented Dec 22, 2025

Uh oh!

hello-stephen commented Dec 22, 2025

FE Regression Coverage Report

Uh oh!

englefly commented Dec 25, 2025

Uh oh!

doris-robot commented Dec 25, 2025

Uh oh!

doris-robot commented Dec 25, 2025

Uh oh!

doris-robot commented Dec 25, 2025

Uh oh!

hello-stephen commented Dec 25, 2025

FE Regression Coverage Report

Uh oh!

englefly commented Feb 12, 2026

Uh oh!

hello-stephen commented Feb 12, 2026

FE UT Coverage Report

Uh oh!

doris-robot commented Feb 12, 2026

Uh oh!

doris-robot commented Feb 12, 2026

Uh oh!

doris-robot commented Feb 12, 2026

Uh oh!

hello-stephen commented Feb 12, 2026

FE Regression Coverage Report

Uh oh!

englefly commented Feb 13, 2026

Uh oh!

englefly commented Feb 13, 2026

Uh oh!

hello-stephen commented Feb 13, 2026

FE Regression Coverage Report

Uh oh!

hello-stephen commented Feb 13, 2026

FE Regression Coverage Report

Uh oh!

github-actionsBot commented Feb 13, 2026

Uh oh!

github-actionsBot commented Feb 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

englefly commented Dec 16, 2025•
edited
Loading