Skip to content

[fix](cloud) Fix NPE in group commit when backend belongs to a different cluster - #60652

Merged
hello-stephen merged 1 commit into
apache:masterfrom
deardeng:fix-group-commit-be
Feb 24, 2026
Merged

[fix](cloud) Fix NPE in group commit when backend belongs to a different cluster#60652
hello-stephen merged 1 commit into
apache:masterfrom
deardeng:fix-group-commit-be

Conversation

@deardeng

@deardengdeardeng commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

Fix:

  • Add cluster membership check in getCachedBackend() to invalidate cached backends that no longer belong to the expected cluster, which can happen during auto-scaling.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@deardeng
[fix](cloud) Fix NPE in group commit when backend belongs to a differ…
bc88160
…ent cluster
Fix:
- Add cluster membership check in getCachedBackend() to invalidate
cached backends that no longer belong to the expected cluster,
which can happen during auto-scaling.
@hello-stephen

hello-stephen commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@deardeng

deardeng commented Feb 10, 2026

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Feb 10, 2026

Copy link
Copy Markdown
TPC-H: Total hot run time: 29986 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc88160ab814d4f0dad01e3790c68c55f0cc8bdd, data reload: false
------ Round 1 ----------------------------------
q1	17636	4442	4290	4290
q2	2091	347	226	226
q3	10131	1310	724	724
q4	10193	787	303	303
q5	7482	2155	1923	1923
q6	189	175	144	144
q7	861	755	593	593
q8	9278	1387	1142	1142
q9	4757	4674	4569	4569
q10	6835	1935	1541	1541
q11	449	264	245	245
q12	336	373	225	225
q13	17775	4037	3183	3183
q14	233	227	214	214
q15	901	817	796	796
q16	676	667	614	614
q17	699	811	583	583
q18	6534	5665	5653	5653
q19	1237	986	627	627
q20	502	498	388	388
q21	2549	1767	1782	1767
q22	327	283	236	236
Total cold run time: 101671 ms
Total hot run time: 29986 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4341	4385	4339	4339
q2	255	373	261	261
q3	2075	2642	2174	2174
q4	1333	1738	1282	1282
q5	4304	4160	4193	4160
q6	220	178	138	138
q7	1863	1818	1687	1687
q8	2471	2576	2570	2570
q9	7820	7581	7623	7581
q10	2889	3204	2781	2781
q11	539	443	438	438
q12	719	767	649	649
q13	4181	4523	3991	3991
q14	299	310	293	293
q15	862	827	806	806
q16	683	744	687	687
q17	1166	1350	1421	1350
q18	8733	8437	7796	7796
q19	895	894	853	853
q20	2029	2188	1976	1976
q21	4698	4604	4111	4111
q22	491	447	405	405
Total cold run time: 52866 ms
Total hot run time: 50328 ms

@doris-robot

doris-robot commented Feb 10, 2026

Copy link
Copy Markdown
TPC-DS: Total hot run time: 190956 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc88160ab814d4f0dad01e3790c68c55f0cc8bdd, data reload: false
query5	4319	617	512	512
query6	342	218	212	212
query7	4231	484	266	266
query8	329	242	243	242
query9	8750	2761	2758	2758
query10	483	381	330	330
query11	16510	16379	16214	16214
query12	177	119	120	119
query13	1261	461	372	372
query14	5889	3180	2939	2939
query14_1	2844	2801	2777	2777
query15	209	190	176	176
query16	972	478	458	458
query17	1085	677	591	591
query18	2430	456	342	342
query19	208	213	189	189
query20	131	126	129	126
query21	221	154	129	129
query22	5132	5145	4836	4836
query23	19129	18499	18342	18342
query23_1	18496	18413	18316	18316
query24	7126	1585	1215	1215
query24_1	1205	1223	1240	1223
query25	517	438	385	385
query26	1232	280	153	153
query27	2754	454	288	288
query28	4525	1884	1879	1879
query29	782	546	470	470
query30	316	244	216	216
query31	872	742	666	666
query32	87	79	79	79
query33	536	329	299	299
query34	919	898	572	572
query35	642	674	615	615
query36	1095	1093	997	997
query37	132	101	83	83
query38	2946	2944	2861	2861
query39	943	885	910	885
query39_1	882	870	882	870
query40	218	132	124	124
query41	68	63	61	61
query42	104	103	103	103
query43	436	442	405	405
query44	1370	733	735	733
query45	191	190	183	183
query46	872	987	617	617
query47	2112	2193	2039	2039
query48	310	327	248	248
query49	614	438	353	353
query50	677	277	217	217
query51	4054	4067	4063	4063
query52	103	103	94	94
query53	294	326	277	277
query54	307	276	258	258
query55	90	85	78	78
query56	308	319	322	319
query57	1371	1420	1282	1282
query58	268	267	270	267
query59	2067	2171	1979	1979
query60	333	336	314	314
query61	145	149	148	148
query62	603	552	488	488
query63	306	268	270	268
query64	4914	1238	954	954
query65	4541	4433	4424	4424
query66	1461	473	359	359
query67	16231	16421	16275	16275
query68	2387	1098	719	719
query69	412	326	305	305
query70	1023	998	914	914
query71	334	319	304	304
query72	3105	2945	2658	2658
query73	512	546	311	311
query74	9700	9679	9583	9583
query75	2807	2759	2411	2411
query76	2281	1049	678	678
query77	347	390	318	318
query78	11377	11430	10898	10898
query79	2077	913	611	611
query80	1469	552	502	502
query81	558	285	253	253
query82	943	153	113	113
query83	323	256	245	245
query84	256	120	94	94
query85	869	485	425	425
query86	428	317	286	286
query87	3112	3057	2979	2979
query88	3612	2678	2674	2674
query89	390	352	328	328
query90	1969	186	176	176
query91	169	162	130	130
query92	80	75	93	75
query93	1043	856	481	481
query94	690	319	303	303
query95	598	397	329	329
query96	645	511	230	230
query97	2485	2464	2424	2424
query98	224	216	208	208
query99	912	953	846	846
Total cold run time: 263440 ms
Total hot run time: 190956 ms

@doris-robot

doris-robot commented Feb 10, 2026

Copy link
Copy Markdown
ClickBench: Total hot run time: 28.29 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc88160ab814d4f0dad01e3790c68c55f0cc8bdd, data reload: false
query1	0.05	0.04	0.05
query2	0.09	0.05	0.04
query3	0.25	0.08	0.08
query4	1.60	0.12	0.10
query5	0.26	0.25	0.24
query6	1.16	0.67	0.68
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.54	0.50	0.49
query10	0.53	0.54	0.55
query11	0.15	0.10	0.10
query12	0.14	0.10	0.11
query13	0.62	0.61	0.62
query14	1.07	1.05	1.06
query15	0.88	0.88	0.86
query16	0.39	0.38	0.40
query17	1.05	1.09	1.14
query18	0.23	0.21	0.21
query19	2.04	1.96	1.96
query20	0.02	0.01	0.01
query21	15.42	0.29	0.15
query22	5.07	0.05	0.05
query23	16.07	0.30	0.11
query24	0.96	0.25	0.34
query25	0.11	0.11	0.07
query26	0.15	0.14	0.13
query27	0.07	0.06	0.11
query28	3.55	1.20	0.97
query29	12.55	4.01	3.21
query30	0.27	0.14	0.11
query31	2.82	0.67	0.41
query32	3.24	0.59	0.50
query33	3.19	3.29	3.28
query34	16.00	5.56	4.71
query35	4.75	4.78	4.82
query36	0.65	0.50	0.49
query37	0.12	0.07	0.07
query38	0.08	0.04	0.04
query39	0.05	0.03	0.03
query40	0.20	0.15	0.15
query41	0.09	0.03	0.04
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 96.64 s
Total hot run time: 28.29 s

@hello-stephen

hello-stephen commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 0.00% (0/27) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 0.00% (0/27) 🎉
Increment coverage report
Complete coverage report

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 11, 2026
@github-actions

github-actionsBot commented Feb 11, 2026

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Feb 11, 2026

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@hello-stephen
hello-stephen merged commit 48894b9 into apache:masterFeb 24, 2026
30 of 31 checks passed
mymeiyi pushed a commit to mymeiyi/doris that referenced this pull request Mar 31, 2026
@deardeng@mymeiyi
[fix](cloud) Fix NPE in group commit when backend belongs to a differ…
0da500f
…ent cluster (apache#60652)
Fix:
- Add cluster membership check in getCachedBackend() to invalidate
cached backends that no longer belong to the expected cluster, which can
happen during auto-scaling.
@mymeiyimymeiyi mentioned this pull request Mar 31, 2026
Merged
mymeiyi pushed a commit to mymeiyi/doris that referenced this pull request Mar 31, 2026
@deardeng@mymeiyi
[fix](cloud) Fix NPE in group commit when backend belongs to a differ…
03cf8d5
…ent cluster (apache#60652)
Fix:
- Add cluster membership check in getCachedBackend() to invalidate
cached backends that no longer belong to the expected cluster, which can
happen during auto-scaling.
@mymeiyimymeiyi mentioned this pull request Mar 31, 2026
Merged
yiguolei pushed a commit that referenced this pull request Mar 31, 2026
@mymeiyi@deardeng
branch-4.1: pick some group commit prs(#60652)(#61555) (#61953)
c49e88b
pick
1. Fix NPE in group commit when backend belongs to a different cluster
(#60652)
2. group commit select backend should consider isLoadAvailable
(#61555)
---------
Co-authored-by: deardeng <dengxin@selectdb.com>
yiguolei pushed a commit that referenced this pull request Apr 15, 2026
@mymeiyi@deardeng
branch-4.0: pick some group commit prs(#60652)(#61555) (#61952)
13a0007
pick
1. Fix NPE in group commit when backend belongs to a different cluster
(#60652)
2. group commit select backend should consider isLoadAvailable
(#61555)
---------
Co-authored-by: deardeng <dengxin@selectdb.com>
@morningmanmorningman mentioned this pull request Jun 3, 2026
Closed
AngryCucumber pushed a commit to AngryCucumber/doris that referenced this pull request Aug 27, 2026
@deardengmassdb-dev
[fix](cloud) Fix NPE in group commit when backend belongs to a differ…
3d3b870
…ent cluster (apache#60652)
Fix:
- Add cluster membership check in getCachedBackend() to invalidate
cached backends that no longer belong to the expected cluster, which can
happen during auto-scaling.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mymeiyimymeiyimymeiyi approved these changes
@hello-stephenhello-stephenhello-stephen approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/4.0.6-mergeddev/4.1.0-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@deardeng@hello-stephen@doris-robot@mymeiyi@yiguolei