Skip to content

[fix](fe) modify tablet cooldownConfLock to reduce memory - #59356

Merged
dataroaring merged 1 commit into
apache:masterfrom
mymeiyi:fix-fe-3
Dec 26, 2025
Merged

[fix](fe) modify tablet cooldownConfLock to reduce memory#59356
dataroaring merged 1 commit into
apache:masterfrom
mymeiyi:fix-fe-3

Conversation

@mymeiyi

@mymeiyimymeiyi commented Dec 25, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

the usage of cooldownConfLock is simple, does not need ReentrantReadWriteLock because it cost too much memory, especially when a cluster has many tablets.

before:
1

after:
2

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

CopilotAI review requested due to automatic review settings December 25, 2025 07:09
@Thearas

Thearas commented Dec 25, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

CopilotAI reviewed Dec 25, 2025
View reviewed changes

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes memory usage in the Tablet class by replacing MonitoredReentrantReadWriteLock with a simple Object for the cooldownConfLock synchronization mechanism.

Key Changes:

  • Replaced MonitoredReentrantReadWriteLock with Object for cooldownConfLock to reduce memory footprint
  • Converted write lock operations in setCooldownConf() to use synchronized block
  • Converted read lock operations with try-finally in getCooldownConf() to use synchronized block

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadfe/fe-core/src/main/java/org/apache/doris/catalog/Tablet.java Outdated
@mymeiyi

mymeiyi commented Dec 25, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Dec 25, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34913 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 98ff3a05db0d64f3b986d54d0601a3754b8fff59, data reload: false
------ Round 1 ----------------------------------
q1	17654	4171	4085	4085
q2	2001	349	230	230
q3	10189	1328	740	740
q4	10218	879	320	320
q5	7540	2163	1892	1892
q6	199	173	140	140
q7	1017	856	706	706
q8	9359	1358	1200	1200
q9	7107	5365	5264	5264
q10	6828	2401	1982	1982
q11	538	341	299	299
q12	668	701	581	581
q13	17782	3647	3015	3015
q14	293	294	286	286
q15	594	518	510	510
q16	686	684	663	663
q17	684	804	535	535
q18	7775	7067	7025	7025
q19	1108	957	601	601
q20	403	363	247	247
q21	4239	3904	3664	3664
q22	1028	953	928	928
Total cold run time: 107910 ms
Total hot run time: 34913 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4094	4028	4047	4028
q2	319	396	330	330
q3	2119	2670	2308	2308
q4	1344	1737	1311	1311
q5	4271	4672	4664	4664
q6	256	190	145	145
q7	2090	1937	1810	1810
q8	2760	2468	2538	2468
q9	7535	7538	7483	7483
q10	3089	3280	2840	2840
q11	590	531	542	531
q12	679	882	678	678
q13	3460	3949	3422	3422
q14	296	306	277	277
q15	549	513	511	511
q16	698	688	630	630
q17	1147	1406	1439	1406
q18	7798	7716	7818	7716
q19	927	892	924	892
q20	2041	2142	1993	1993
q21	5008	4349	4221	4221
q22	1105	1031	975	975
Total cold run time: 52175 ms
Total hot run time: 50639 ms

@doris-robot

doris-robot commented Dec 25, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 180017 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 98ff3a05db0d64f3b986d54d0601a3754b8fff59, data reload: false
query5	4380	608	450	450
query6	334	229	218	218
query7	4214	460	274	274
query8	328	261	251	251
query9	8765	2544	2551	2544
query10	520	378	329	329
query11	15931	14916	14791	14791
query12	184	117	115	115
query13	1271	531	396	396
query14	5535	3039	2786	2786
query14_1	2691	2672	2636	2636
query15	210	200	181	181
query16	846	472	447	447
query17	1145	700	622	622
query18	2433	453	356	356
query19	238	229	216	216
query20	131	122	114	114
query21	223	144	125	125
query22	3987	4079	3857	3857
query23	16715	16213	15980	15980
query23_1	16213	16183	16015	16015
query24	7371	1668	1243	1243
query24_1	1279	1233	1277	1233
query25	598	528	452	452
query26	1256	291	169	169
query27	2738	477	304	304
query28	4484	2149	2142	2142
query29	821	578	473	473
query30	316	235	214	214
query31	821	704	645	645
query32	79	78	72	72
query33	561	354	311	311
query34	912	887	543	543
query35	786	831	720	720
query36	884	918	804	804
query37	129	102	75	75
query38	3073	3103	3078	3078
query39	765	723	723	723
query39_1	701	697	731	697
query40	227	138	120	120
query41	69	65	64	64
query42	110	105	106	105
query43	447	444	420	420
query44	1326	763	748	748
query45	186	188	184	184
query46	870	987	618	618
query47	1632	1690	1632	1632
query48	305	329	245	245
query49	629	431	355	355
query50	669	316	217	217
query51	3830	3863	3799	3799
query52	111	109	101	101
query53	326	349	290	290
query54	284	258	248	248
query55	76	76	74	74
query56	308	297	304	297
query57	1165	1200	1091	1091
query58	275	260	271	260
query59	2444	2498	2337	2337
query60	311	304	301	301
query61	167	154	157	154
query62	755	726	668	668
query63	333	295	299	295
query64	4987	1292	1013	1013
query65	4072	3975	3944	3944
query66	1469	434	307	307
query67	15095	15217	15091	15091
query68	8401	986	711	711
query69	500	351	312	312
query70	1053	1038	1004	1004
query71	356	304	281	281
query72	6203	4999	5081	4999
query73	713	648	316	316
query74	9164	8876	8840	8840
query75	3246	3189	2799	2799
query76	3862	1158	748	748
query77	533	389	292	292
query78	9372	9628	8859	8859
query79	1595	849	614	614
query80	698	643	555	555
query81	496	269	230	230
query82	312	137	107	107
query83	274	261	250	250
query84	261	122	102	102
query85	973	512	465	465
query86	336	315	284	284
query87	3211	3195	3213	3195
query88	4144	2296	2279	2279
query89	480	427	413	413
query90	2100	166	162	162
query91	176	169	152	152
query92	79	69	69	69
query93	1334	916	556	556
query94	484	293	252	252
query95	579	326	302	302
query96	595	490	210	210
query97	2285	2291	2206	2206
query98	225	200	194	194
query99	1345	1341	1315	1315
Total cold run time: 261731 ms
Total hot run time: 180017 ms

@doris-robot

doris-robot commented Dec 25, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.04 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 98ff3a05db0d64f3b986d54d0601a3754b8fff59, data reload: false
query1	0.05	0.05	0.05
query2	0.10	0.05	0.04
query3	0.26	0.08	0.08
query4	1.60	0.11	0.10
query5	0.28	0.24	0.25
query6	1.17	0.63	0.62
query7	0.04	0.02	0.03
query8	0.06	0.04	0.04
query9	0.56	0.50	0.49
query10	0.59	0.55	0.54
query11	0.16	0.10	0.10
query12	0.15	0.11	0.12
query13	0.61	0.59	0.60
query14	0.99	0.99	0.98
query15	0.82	0.79	0.81
query16	0.39	0.41	0.40
query17	1.07	1.06	1.01
query18	0.23	0.21	0.22
query19	1.94	1.78	1.78
query20	0.02	0.02	0.01
query21	15.46	0.29	0.13
query22	4.95	0.05	0.05
query23	16.16	0.28	0.10
query24	1.52	0.28	0.56
query25	0.09	0.06	0.06
query26	0.14	0.13	0.14
query27	0.07	0.05	0.06
query28	4.13	1.22	1.02
query29	12.61	4.08	3.19
query30	0.29	0.16	0.11
query31	2.83	0.62	0.40
query32	3.23	0.55	0.45
query33	2.98	3.05	3.00
query34	17.03	5.16	4.50
query35	4.53	4.55	4.55
query36	0.64	0.50	0.49
query37	0.11	0.06	0.07
query38	0.08	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.14	0.13
query41	0.09	0.03	0.02
query42	0.04	0.02	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.33 s
Total hot run time: 27.04 s

@hello-stephen

hello-stephen commented Dec 25, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented Dec 25, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (6/6) 🎉
Increment coverage report
Complete coverage report

dataroaring
dataroaring approved these changes Dec 26, 2025
View reviewed changes

@dataroaringdataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 26, 2025
@github-actions

github-actionsBot commented Dec 26, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Dec 26, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

deardeng
deardeng approved these changes Dec 26, 2025
View reviewed changes

@deardengdeardeng left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring
dataroaring merged commit 66a7d11 into apache:masterDec 26, 2025
26 of 28 checks passed
github-actionsBot pushed a commit that referenced this pull request Dec 30, 2025
@mymeiyiYour Name
[fix](fe) modify tablet cooldownConfLock to reduce memory (#59356)
6843cce
### What problem does this PR solve?
the usage of `cooldownConfLock` is simple, does not need
ReentrantReadWriteLock because it cost too much memory, especially when
a cluster has many tablets.
before:
<img width="1280" height="575" alt="1"
src="https://github.com/user-attachments/assets/df626e1e-84ef-4078-8da7-d857ac960678"
/>
after:
<img width="1280" height="510" alt="2"
src="https://github.com/user-attachments/assets/3fee7407-0743-4d07-9462-a9f0e101765f"
/>
@github-actionsgithub-actionsBot mentioned this pull request Dec 30, 2025
Merged
yiguolei pushed a commit that referenced this pull request Dec 31, 2025
@github-actions@mymeiyi
branch-4.0: [fix](fe) modify tablet cooldownConfLock to reduce memory#…
2078d35
…59356 (#59499)
Cherry-picked from #59356
Co-authored-by: meiyi <meiyi@selectdb.com>
@yiguoleiyiguolei mentioned this pull request Jan 23, 2026
Closed
ybtsdst pushed a commit to ybtsdst/doris that referenced this pull request Feb 27, 2026
@github-actions@mymeiyibingtao.yin
branch-4.0: [fix](fe) modify tablet cooldownConfLock to reduce memorya…
3e063cf
…pache#59356 (apache#59499)
Cherry-picked from apache#59356
Co-authored-by: meiyi <meiyi@selectdb.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@deardengdeardengdeardeng approved these changes
@dataroaringdataroaringdataroaring approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/4.0.3-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@mymeiyi@Thearas@doris-robot@hello-stephen@deardeng@dataroaring@gavinchou@yiguolei