Skip to content

[refactor](predicate) Initialize topN predicate with correct cid - #59088

Merged
Gabriel39 merged 4 commits into
apache:masterfrom
Gabriel39:dev_1216
Dec 17, 2025
Merged

[refactor](predicate) Initialize topN predicate with correct cid#59088
Gabriel39 merged 4 commits into
apache:masterfrom
Gabriel39:dev_1216

Conversation

@Gabriel39

@Gabriel39Gabriel39 commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39

Gabriel39 commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@hello-stephen

hello-stephen commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 23.91% (11/46) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage53.41% (18831/35256)
Line Coverage39.20% (174357/444822)
Region Coverage33.75% (134882/399652)
Branch Coverage34.66% (58107/167639)

@hello-stephen

hello-stephen commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 45.65% (21/46) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage58.02% (20047/34549)
Line Coverage43.72% (194194/444207)
Region Coverage38.06% (153864/404279)
Branch Coverage38.86% (65461/168450)

@Gabriel39

Gabriel39 commented Dec 17, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34972 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2a1c277e3193ef9e2f72531d905cbe04e1688402, data reload: false
------ Round 1 ----------------------------------
q1	17689	4199	4056	4056
q2	2013	358	262	262
q3	10183	1329	749	749
q4	10207	855	318	318
q5	7543	2126	1959	1959
q6	183	172	137	137
q7	980	859	703	703
q8	9348	1487	1133	1133
q9	7173	5277	5251	5251
q10	6820	2395	1969	1969
q11	531	310	296	296
q12	672	723	561	561
q13	17794	3724	3103	3103
q14	288	293	274	274
q15	597	514	519	514
q16	691	683	622	622
q17	706	783	545	545
q18	7753	7041	7070	7041
q19	1101	959	607	607
q20	401	359	248	248
q21	4295	3893	3669	3669
q22	1035	1014	955	955
Total cold run time: 108003 ms
Total hot run time: 34972 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4081	4018	4060	4018
q2	337	395	321	321
q3	2123	2676	2253	2253
q4	1337	1732	1283	1283
q5	4231	4578	4711	4578
q6	222	171	130	130
q7	2034	1984	1833	1833
q8	2803	2580	2534	2534
q9	7547	7566	7576	7566
q10	3004	3269	2848	2848
q11	607	511	484	484
q12	698	767	762	762
q13	3686	3917	3316	3316
q14	307	318	293	293
q15	556	537	500	500
q16	661	687	661	661
q17	1225	1594	1490	1490
q18	7749	7659	7509	7509
q19	930	902	946	902
q20	1996	2062	1956	1956
q21	4678	4345	4142	4142
q22	1064	1061	980	980
Total cold run time: 51876 ms
Total hot run time: 50359 ms

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 25185 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2a1c277e3193ef9e2f72531d905cbe04e1688402, data reload: false
query5 query6 query7 query8 query9 query10 query11 query12 query13 query14 query14_1 query15 query16 query17 query18 query19 query20 query21 query22 query23 query23_1 query24 query24_1 query25 query26 query27 query28 query29 query30 query31 query32 query33 query34 query35 query36 query37 query38 query39 query39_1 query40 query41 query42 query43 query44 query45 query46 query47 query48 query49 query50 query51 query52 query53 query54 query55 query56 query57 query58 query59 query60 query61 query62 query63 query64 query65 query66 query67 query68 query69 query70 query71 query72 query73 query74 query75 query76 query77 query78 query79 query80 query81 query82 query83 query84 query85 query86 query87 query88 query89 query90 query91 query92 query93 query94 query95 query96 query97 query98 query99 Total cold run time: 39912 ms
Total hot run time: 25185 ms

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.23 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2a1c277e3193ef9e2f72531d905cbe04e1688402, data reload: false
query1	0.05	0.05	0.04
query2	0.12	0.05	0.05
query3	0.27	0.09	0.08
query4	1.61	0.12	0.11
query5	0.27	0.25	0.25
query6	1.17	0.65	0.63
query7	0.03	0.02	0.03
query8	0.06	0.04	0.04
query9	0.57	0.50	0.51
query10	0.54	0.54	0.55
query11	0.15	0.11	0.11
query12	0.14	0.12	0.11
query13	0.63	0.60	0.60
query14	0.99	0.98	0.98
query15	0.82	0.80	0.80
query16	0.40	0.41	0.39
query17	1.06	1.03	0.99
query18	0.24	0.22	0.22
query19	1.93	1.79	1.85
query20	0.02	0.02	0.01
query21	15.40	0.28	0.15
query22	4.82	0.05	0.05
query23	15.99	0.29	0.10
query24	1.13	0.29	0.43
query25	0.10	0.08	0.06
query26	0.15	0.14	0.14
query27	0.06	0.06	0.06
query28	3.78	1.23	1.02
query29	12.57	4.01	3.28
query30	0.28	0.15	0.12
query31	2.82	0.62	0.39
query32	3.23	0.56	0.46
query33	3.06	3.01	2.98
query34	17.00	5.24	4.52
query35	4.57	4.55	4.52
query36	0.65	0.50	0.48
query37	0.10	0.07	0.06
query38	0.08	0.04	0.04
query39	0.05	0.03	0.03
query40	0.18	0.14	0.14
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 97.25 s
Total hot run time: 27.23 s

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 22.92% (11/48) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage53.40% (18827/35256)
Line Coverage39.18% (174269/444828)
Region Coverage33.76% (134932/399666)
Branch Coverage34.65% (58094/167641)

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 43.75% (21/48) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage58.08% (20066/34549)
Line Coverage43.80% (194575/444213)
Region Coverage38.46% (155479/404293)
Branch Coverage39.11% (65875/168452)

@Gabriel39

Gabriel39 commented Dec 17, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 180246 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 293836a74862682e3bd3e27d1e665d3562c29e7d, data reload: false
query5	5351	601	465	465
query6	343	233	222	222
query7	4223	476	284	284
query8	331	253	249	249
query9	8790	2532	2541	2532
query10	598	388	341	341
query11	15173	15016	15745	15016
query12	204	132	119	119
query13	1587	516	397	397
query14	6505	3167	3013	3013
query14_1	2803	2819	2843	2819
query15	244	217	190	190
query16	832	560	479	479
query17	1144	754	635	635
query18	2857	487	402	402
query19	256	254	222	222
query20	133	112	117	112
query21	221	143	114	114
query22	4431	4437	4259	4259
query23	17217	16701	16224	16224
query23_1	16488	16524	16349	16349
query24	7671	1669	1246	1246
query24_1	1276	1267	1275	1267
query25	561	502	445	445
query26	1253	266	158	158
query27	2735	462	312	312
query28	4439	2124	2112	2112
query29	807	610	441	441
query30	320	243	212	212
query31	823	723	635	635
query32	77	67	62	62
query33	546	340	289	289
query34	915	888	556	556
query35	799	813	736	736
query36	868	910	835	835
query37	136	97	77	77
query38	2853	2863	2898	2863
query39	744	757	731	731
query39_1	712	685	706	685
query40	225	136	117	117
query41	63	64	60	60
query42	105	104	102	102
query43	423	432	384	384
query44	1324	760	745	745
query45	196	188	184	184
query46	900	984	612	612
query47	1661	1657	1604	1604
query48	318	328	242	242
query49	608	432	364	364
query50	668	304	223	223
query51	3978	3853	3809	3809
query52	108	105	96	96
query53	320	352	298	298
query54	294	266	282	266
query55	79	77	75	75
query56	298	304	297	297
query57	1130	1144	1071	1071
query58	271	253	247	247
query59	2382	2498	2333	2333
query60	320	315	291	291
query61	163	161	172	161
query62	726	667	647	647
query63	338	311	296	296
query64	4917	1295	999	999
query65	4012	3966	3951	3951
query66	1389	440	309	309
query67	15302	14906	14627	14627
query68	8577	1031	727	727
query69	521	354	310	310
query70	1118	987	986	986
query71	426	309	282	282
query72	6087	5008	5012	5008
query73	667	599	308	308
query74	8808	8688	8593	8593
query75	3488	3194	2798	2798
query76	4216	1157	750	750
query77	850	437	312	312
query78	9538	9754	8779	8779
query79	1475	927	605	605
query80	747	656	586	586
query81	545	268	248	248
query82	203	130	103	103
query83	259	261	242	242
query84	263	116	101	101
query85	896	534	470	470
query86	379	298	263	263
query87	3071	3046	3029	3029
query88	3198	2272	2280	2272
query89	459	420	381	381
query90	2182	158	159	158
query91	179	162	143	143
query92	80	69	62	62
query93	1299	930	557	557
query94	440	293	298	293
query95	565	379	303	303
query96	586	458	208	208
query97	2286	2278	2201	2201
query98	210	209	193	193
query99	1304	1302	1207	1207
Total cold run time: 264806 ms
Total hot run time: 180246 ms

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.63 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 293836a74862682e3bd3e27d1e665d3562c29e7d, data reload: false
query1	0.06	0.04	0.04
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.12	0.11
query5	0.27	0.26	0.24
query6	1.17	0.64	0.62
query7	0.03	0.03	0.02
query8	0.05	0.04	0.04
query9	0.59	0.51	0.50
query10	0.56	0.55	0.56
query11	0.17	0.11	0.12
query12	0.16	0.11	0.12
query13	0.62	0.61	0.60
query14	0.99	0.98	0.99
query15	0.82	0.79	0.79
query16	0.41	0.38	0.39
query17	0.97	1.02	1.07
query18	0.23	0.21	0.22
query19	1.93	1.89	1.78
query20	0.02	0.02	0.01
query21	15.46	0.29	0.15
query22	4.92	0.05	0.05
query23	15.94	0.28	0.10
query24	1.18	0.70	0.62
query25	0.11	0.06	0.07
query26	0.14	0.15	0.13
query27	0.06	0.06	0.06
query28	4.64	1.22	1.02
query29	12.58	3.98	3.30
query30	0.28	0.14	0.12
query31	2.82	0.63	0.39
query32	3.26	0.56	0.45
query33	3.02	3.03	3.02
query34	16.92	5.23	4.49
query35	4.51	4.60	4.60
query36	0.66	0.50	0.49
query37	0.11	0.07	0.06
query38	0.08	0.04	0.04
query39	0.04	0.02	0.03
query40	0.17	0.14	0.13
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.09 s
Total hot run time: 27.63 s

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 20.75% (11/53) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage53.39% (18821/35252)
Line Coverage39.17% (174212/444797)
Region Coverage33.74% (134825/399600)
Branch Coverage34.63% (58060/167635)

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 17, 2025
@github-actions

github-actionsBot commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

BiteTheDDDDt
BiteTheDDDDt approved these changes Dec 17, 2025
View reviewed changes

@BiteTheDDDDtBiteTheDDDDt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 90.57% (48/53) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage72.31% (24980/34545)
Line Coverage59.03% (262222/444182)
Region Coverage53.82% (217547/404227)
Branch Coverage55.43% (93378/168446)

@Gabriel39
Gabriel39 merged commit 262b16e into apache:masterDec 17, 2025
27 of 30 checks passed
Gabriel39 added a commit to Gabriel39/incubator-doris that referenced this pull request Jan 13, 2026
@Gabriel39
[refactor](predicate) Initialize topN predicate with correct cid (apa…
73569c7
…che#59088)
@Gabriel39Gabriel39 mentioned this pull request Jan 13, 2026
Merged
16 tasks
yiguolei pushed a commit that referenced this pull request Jan 14, 2026
@Gabriel39
[branch-4.0]Pick predicates-related PR (#59804)
0174967
### What problem does this PR solve?
pick #57397#58283#58290#58282#58832#58905#58960#59005#59088#59098#59126#59187#59581#59625#59775
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
ybtsdst pushed a commit to ybtsdst/doris that referenced this pull request Feb 27, 2026
@Gabriel39bingtao.yin
[branch-4.0]Pick predicates-related PR (apache#59804)
f702132
### What problem does this PR solve?
pick apache#57397apache#58283apache#58290apache#58282apache#58832apache#58905apache#58960apache#59005apache#59088apache#59098apache#59126apache#59187apache#59581apache#59625apache#59775
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@BiteTheDDDDtBiteTheDDDDt mentioned this pull request Jun 1, 2026
Merged
BiteTheDDDDt added a commit that referenced this pull request Jun 2, 2026
@BiteTheDDDDt
[fix](be) Fix TopN runtime filter activation (#63969)
f1ad42e
#59088 changed TopN runtime predicate target
initialization to rely on a storage column id. For targets that cannot
create a storage column predicate, such as non-pushdown TopN predicates
or unsupported storage columns, init_target returned before marking the
target as detected. That left RuntimePredicate disabled, so the scan
side ignored the TopN source even though FE had sent the source id. This
PR keeps the target detected when no storage predicate is created,
removes obsolete compatibility skips for missing runtime predicate
descs, and adds FE/BE coverage for the source marking and no-column
target paths.
zhaorongsheng pushed a commit to zhaorongsheng/doris that referenced this pull request Jun 4, 2026
@BiteTheDDDDtzhaorongsheng
[fix](be) Fix TopN runtime filter activation (apache#63969)
f1638f7
apache#59088 changed TopN runtime predicate target
initialization to rely on a storage column id. For targets that cannot
create a storage column predicate, such as non-pushdown TopN predicates
or unsupported storage columns, init_target returned before marking the
target as detected. That left RuntimePredicate disabled, so the scan
side ignored the TopN source even though FE had sent the source id. This
PR keeps the target detected when no storage predicate is created,
removes obsolete compatibility skips for missing runtime predicate
descs, and adds FE/BE coverage for the source marking and no-column
target paths.
@Gabriel39
Gabriel39 deleted the dev_1216 branch July 30, 2026 06:26
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDtBiteTheDDDDtBiteTheDDDDt approved these changes
@yiguoleiyiguoleiyiguolei approved these changes
@MryangeMryangeMryange approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/4.0.3-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Gabriel39@hello-stephen@doris-robot@BiteTheDDDDt@yiguolei@Mryange