[fix](nereids)set RuntimeFilterInfo only on BE which is merge node by englefly · Pull Request #57108 · apache/doris

englefly · 2025-10-17T09:36:13Z

What problem does this PR solve?

set RuntimeFilterInfo only on BE which is merge node.
note: RuntimeFilterInfo only contains runtime-filter merge information, not runtime filter descriptor
In the previous pull request #56978, we moved the runtime info (which records how runtime filters are merged) from the instance level to the Backend (BE) level. In a multi-BE environment, when a runtime filter needs to be merged, the BE does not perform the merge using the mergeInstance specified by the Runtime Filter descriptor. Instead, it selects a BE that has the RuntimeInfo set to perform the merge. This causes BEs that should not be responsible for merging to wait for other nodes to send local runtime filters, resulting in a timeout.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-10-17T09:36:19Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

englefly · 2025-10-20T01:33:41Z

run buildall

doris-robot · 2025-10-20T02:23:27Z

ClickBench: Total hot run time: 32.44 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ae913b2bd6755b22dc1273b2c252c673745d10a4, data reload: false
query1	0.06	0.06	0.06
query2	0.11	0.06	0.06
query3	0.27	0.09	0.10
query4	1.62	0.13	0.12
query5	0.30	0.28	0.28
query6	1.18	0.68	0.68
query7	0.04	0.03	0.04
query8	0.06	0.06	0.05
query9	0.66	0.57	0.56
query10	0.64	0.63	0.63
query11	0.19	0.14	0.13
query12	0.19	0.15	0.14
query13	0.65	0.64	0.64
query14	1.09	1.05	1.07
query15	0.93	0.93	0.93
query16	0.43	0.46	0.45
query17	1.15	1.26	1.13
query18	0.24	0.23	0.22
query19	2.10	1.97	2.00
query20	0.02	0.02	0.02
query21	15.39	1.11	0.68
query22	0.79	1.26	0.71
query23	14.78	1.56	0.73
query24	7.23	0.80	0.97
query25	0.49	0.15	0.08
query26	0.64	0.19	0.17
query27	0.07	0.06	0.06
query28	9.09	1.49	0.96
query29	12.61	4.64	3.98
query30	0.29	0.15	0.13
query31	2.84	0.67	0.43
query32	3.25	0.59	0.50
query33	3.20	3.29	3.10
query34	16.27	5.52	4.88
query35	5.02	5.04	5.04
query36	0.72	0.58	0.52
query37	0.11	0.08	0.08
query38	0.07	0.06	0.05
query39	0.04	0.04	0.03
query40	0.20	0.16	0.16
query41	0.10	0.04	0.04
query42	0.05	0.03	0.04
query43	0.05	0.04	0.04
Total cold run time: 105.23 s
Total hot run time: 32.44 s

hello-stephen · 2025-10-20T02:49:49Z

FE UT Coverage Report

Increment line coverage 0.00% (0/9) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-10-20T04:46:18Z

FE Regression Coverage Report

Increment line coverage 66.67% (6/9) 🎉
Increment coverage report
Complete coverage report

englefly · 2025-10-21T03:40:29Z

run buildall

doris-robot · 2025-10-21T04:03:07Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	80.77% (1647/2039)
Line Coverage	67.02% (29050/43346)
Region Coverage	67.30% (14369/21352)
Branch Coverage	57.66% (7637/13246)

hello-stephen · 2025-10-21T04:47:52Z

FE UT Coverage Report

Increment line coverage 0.00% (0/9) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-10-21T05:31:00Z

TPC-DS: Total hot run time: 190659 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fdd387aca088a76f66688ed8f3a44763e8338ab4, data reload: false
query1	1111	415	395	395
query2	6584	1708	1683	1683
query3	6763	223	227	223
query4	25898	23485	23189	23189
query5	4426	636	488	488
query6	328	240	246	240
query7	4655	515	303	303
query8	300	262	284	262
query9	8710	2688	2664	2664
query10	476	344	306	306
query11	15521	15092	15310	15092
query12	195	124	117	117
query13	1679	594	417	417
query14	11522	9212	9223	9212
query15	208	192	178	178
query16	7798	745	513	513
query17	1297	843	721	721
query18	2173	511	349	349
query19	245	239	195	195
query20	157	134	134	134
query21	220	146	133	133
query22	4861	4936	4525	4525
query23	34836	34401	33871	33871
query24	8392	2533	2529	2529
query25	673	538	466	466
query26	1296	290	167	167
query27	2843	555	395	395
query28	4823	2271	2220	2220
query29	817	666	523	523
query30	313	239	217	217
query31	952	848	786	786
query32	81	74	70	70
query33	591	421	342	342
query34	842	901	535	535
query35	842	881	810	810
query36	983	1052	951	951
query37	131	130	94	94
query38	3699	3665	3624	3624
query39	1484	1465	1432	1432
query40	224	131	124	124
query41	66	63	64	63
query42	124	127	114	114
query43	498	507	476	476
query44	1230	764	752	752
query45	189	188	174	174
query46	913	1007	647	647
query47	1795	1808	1760	1760
query48	414	440	325	325
query49	783	532	437	437
query50	666	691	419	419
query51	3915	4145	3890	3890
query52	113	117	100	100
query53	246	267	201	201
query54	634	621	537	537
query55	94	84	85	84
query56	416	340	324	324
query57	1171	1204	1129	1129
query58	289	282	286	282
query59	2691	2712	2497	2497
query60	358	356	334	334
query61	155	169	153	153
query62	782	737	654	654
query63	237	201	208	201
query64	4329	1206	894	894
query65	4071	4033	3961	3961
query66	1088	420	326	326
query67	15707	15331	15031	15031
query68	9782	953	602	602
query69	497	328	291	291
query70	1385	1283	1265	1265
query71	495	347	335	335
query72	5755	4948	4802	4802
query73	749	591	360	360
query74	8874	8923	8713	8713
query75	4498	3341	2772	2772
query76	5327	1164	742	742
query77	989	408	312	312
query78	9685	9695	8973	8973
query79	4489	798	572	572
query80	744	639	507	507
query81	498	280	230	230
query82	319	160	130	130
query83	316	270	252	252
query84	305	120	90	90
query85	860	470	427	427
query86	336	315	295	295
query87	3759	3796	3774	3774
query88	2887	2230	2217	2217
query89	438	329	298	298
query90	2178	218	217	217
query91	162	170	134	134
query92	86	72	70	70
query93	2845	984	637	637
query94	691	436	335	335
query95	411	323	320	320
query96	508	579	293	293
query97	2937	2973	2918	2918
query98	238	228	209	209
query99	1422	1451	1307	1307
Total cold run time: 286388 ms
Total hot run time: 190659 ms

doris-robot · 2025-10-21T05:36:13Z

ClickBench: Total hot run time: 28.53 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fdd387aca088a76f66688ed8f3a44763e8338ab4, data reload: false
query1	0.06	0.05	0.04
query2	0.10	0.04	0.06
query3	0.26	0.08	0.08
query4	1.61	0.12	0.12
query5	0.30	0.26	0.25
query6	1.20	0.67	0.66
query7	0.03	0.02	0.03
query8	0.06	0.04	0.04
query9	0.63	0.53	0.52
query10	0.58	0.59	0.59
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.62	0.62	0.62
query14	1.03	1.04	1.04
query15	0.87	0.89	0.86
query16	0.40	0.40	0.41
query17	1.06	1.12	1.04
query18	0.25	0.20	0.21
query19	1.97	1.87	1.82
query20	0.02	0.02	0.02
query21	15.45	0.19	0.12
query22	5.18	0.07	0.05
query23	15.69	0.25	0.10
query24	2.58	0.50	0.65
query25	0.08	0.08	0.06
query26	0.14	0.13	0.15
query27	0.06	0.05	0.07
query28	4.75	1.16	0.95
query29	12.56	4.04	3.27
query30	0.30	0.15	0.13
query31	2.83	0.61	0.39
query32	3.24	0.57	0.49
query33	3.10	3.12	3.07
query34	16.10	5.50	4.89
query35	4.91	4.95	4.87
query36	0.71	0.51	0.51
query37	0.11	0.07	0.07
query38	0.07	0.04	0.03
query39	0.03	0.03	0.03
query40	0.18	0.16	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 99.6 s
Total hot run time: 28.53 s

doris-robot · 2025-10-21T06:40:56Z

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.64% (17895/33995)
Line Coverage	37.86% (162309/428747)
Region Coverage	32.26% (123729/383544)
Branch Coverage	33.67% (54222/161025)

hello-stephen · 2025-10-21T07:29:53Z

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.40% (23796/33328)
Line Coverage	57.78% (247575/428499)
Region Coverage	52.83% (205228/388502)
Branch Coverage	54.56% (88345/161920)

hello-stephen · 2025-10-21T07:43:35Z

FE Regression Coverage Report

Increment line coverage 66.67% (6/9) 🎉
Increment coverage report
Complete coverage report

github-actions · 2025-10-21T07:48:37Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-10-21T07:48:40Z

PR approved by anyone and no changes requested.

…57108) set RuntimeFilterInfo only on BE which is merge node. note: RuntimeFilterInfo only contains runtime-filter merge information, not runtime filter descriptor In the previous pull request #56978, we moved the runtime info (which records how runtime filters are merged) from the instance level to the Backend (BE) level. In a multi-BE environment, when a runtime filter needs to be merged, the BE does not perform the merge using the mergeInstance specified by the Runtime Filter descriptor. Instead, it selects a BE that has the RuntimeInfo set to perform the merge. This causes BEs that should not be responsible for merging to wait for other nodes to send local runtime filters, resulting in a time

…erge node #57108 (#57199) Cherry-picked from #57108 Co-authored-by: minghong <zhouminghong@selectdb.com>

…pache#57108) set RuntimeFilterInfo only on BE which is merge node. note: RuntimeFilterInfo only contains runtime-filter merge information, not runtime filter descriptor In the previous pull request apache#56978, we moved the runtime info (which records how runtime filters are merged) from the instance level to the Backend (BE) level. In a multi-BE environment, when a runtime filter needs to be merged, the BE does not perform the merge using the mergeInstance specified by the Runtime Filter descriptor. Instead, it selects a BE that has the RuntimeInfo set to perform the merge. This causes BEs that should not be responsible for merging to wait for other nodes to send local runtime filters, resulting in a time

yiguolei added the dev/4.0.x label Oct 17, 2025

move runtime filter info to BE level
fdd387a

englefly force-pushed the rf-merge-node branch from ae913b2 to fdd387aCompare October 21, 2025 03:40

BiteTheDDDDt approved these changes Oct 21, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 21, 2025

github-actionsBot added the reviewed label Oct 21, 2025

starocean999 approved these changes Oct 21, 2025
View reviewed changes

BiteTheDDDDt merged commit 33db05e into apache:masterOct 21, 2025
26 of 28 checks passed

github-actionsBot mentioned this pull request Oct 21, 2025
branch-4.0: [fix](nereids)set RuntimeFilterInfo only on BE which is merge node #57108 #57199
Merged

yiguolei pushed a commit that referenced this pull request Oct 23, 2025
branch-4.0: [fix](nereids)set RuntimeFilterInfo only on BE which is m…
e05ceb5
…erge node #57108 (#57199) Cherry-picked from #57108 Co-authored-by: minghong <zhouminghong@selectdb.com>

yiguolei added dev/4.0.1-merged and removed dev/4.0.x labels Oct 23, 2025

yiguolei mentioned this pull request Nov 5, 2025
4.0.1 Release Notes #57717
Closed

Uh oh!

Conversation

englefly commented Oct 17, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Oct 17, 2025

Uh oh!

englefly commented Oct 20, 2025

Uh oh!

doris-robot commented Oct 20, 2025

Uh oh!

hello-stephen commented Oct 20, 2025

FE UT Coverage Report

Uh oh!

hello-stephen commented Oct 20, 2025

FE Regression Coverage Report

Uh oh!

englefly commented Oct 21, 2025

Uh oh!

doris-robot commented Oct 21, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Oct 21, 2025

FE UT Coverage Report

Uh oh!

doris-robot commented Oct 21, 2025

Uh oh!

doris-robot commented Oct 21, 2025

Uh oh!

doris-robot commented Oct 21, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Oct 21, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Oct 21, 2025

FE Regression Coverage Report

Uh oh!

github-actionsBot commented Oct 21, 2025

Uh oh!

github-actionsBot commented Oct 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

englefly commented Oct 17, 2025•
edited
Loading