Uh oh!
There was an error while loading. Please reload this page.
[improve](cloud) cloud reduce get_tablet_stats rpc to meta_service - #60543
Conversation
hello-stephen
commented
Feb 5, 2026
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
There was a problem hiding this comment.
Pull request overview
This PR optimizes cloud tablet statistics collection by reducing redundant RPC calls to the meta-service. It introduces an intelligent two-tier stats fetching strategy: active tablets (those recently modified or accessed) are fetched immediately, while stable tablets use an adaptive interval ladder (1m → 5m → 10m → 30m → 2h → 6h → 12h → 3d → infinite). Additionally, it implements master-to-follower FE stats synchronization to ensure consistency across the cluster.
Changes:
- Implements active tablet tracking with immediate stats updates for recently modified tablets
- Adds interval-based lazy fetching for stable tablets using an adaptive backoff strategy
- Introduces master FE → follower/observer FE stats synchronization via new Thrift RPC
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| gensrc/thrift/FrontendService.thrift | Adds tabletIds field to TReportCommitTxnResultRequest and new syncCloudTabletStats RPC for FE-to-FE stats synchronization |
| fe/fe-core/src/main/java/org/apache/doris/transaction/GlobalTransactionMgrIface.java | Updates afterCommitTxnResp signature to accept tablet IDs parameter |
| fe/fe-core/src/main/java/org/apache/doris/transaction/GlobalTransactionMgr.java | Implements empty afterCommitTxnResp method with new signature in base class |
| fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java | Handles tablet stats reporting from BE and implements syncCloudTabletStats RPC endpoint |
| fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java | Adds cloud_force_sync_tablet_stats session variable for manual stats refresh |
| fe/fe-core/src/main/java/org/apache/doris/common/proc/TabletsProcDir.java | Integrates with active tablets tracking when cloud_force_sync_tablet_stats is enabled |
| fe/fe-core/src/main/java/org/apache/doris/common/ClientPool.java | Adds frontendStatsPool for FE-to-FE stats synchronization with heartbeat timeout |
| fe/fe-core/src/main/java/org/apache/doris/cloud/transaction/CloudGlobalTransactionMgr.java | Passes tablet IDs to stats manager after transaction commits |
| fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudReplica.java | Adds lastGetTabletStatsTime and statsIntervalIndex fields for adaptive interval tracking |
| fe/fe-core/src/main/java/org/apache/doris/catalog/CloudTabletStatMgr.java | Core implementation of active tablet tracking, interval ladder, and master-follower synchronization |
| fe/fe-core/src/main/java/org/apache/doris/alter/CloudSchemaChangeJobV2.java | Marks schema change tablets as active for immediate stats updates |
| fe/fe-common/src/main/java/org/apache/doris/common/Config.java | Adds cloud_sync_tablet_stats_task_threads_num configuration |
| be/src/cloud/cloud_meta_mgr.cpp | Passes tablet IDs to FE when reporting commit results and after compaction jobs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
mymeiyi
commented
Feb 5, 2026
run buildall |
doris-robot
commented
Feb 5, 2026
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
doris-robot
commented
Feb 5, 2026
TPC-H: Total hot run time: 31144 ms |
doris-robot
commented
Feb 5, 2026
ClickBench: Total hot run time: 28.85 s |
doris-robot
commented
Feb 5, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 5, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 5, 2026
FE Regression Coverage ReportIncrement line coverage |
c1abd4d to
a825e3aComparemymeiyi
commented
Feb 6, 2026
run buildall |
doris-robot
commented
Feb 6, 2026
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 6, 2026
FE UT Coverage ReportIncrement line coverage |
doris-robot
commented
Feb 6, 2026
TPC-H: Total hot run time: 30504 ms |
doris-robot
commented
Feb 6, 2026
ClickBench: Total hot run time: 28.37 s |
hello-stephen
commented
Feb 6, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 6, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 6, 2026
FE Regression Coverage ReportIncrement line coverage |
a825e3a to
ce9797dComparemymeiyi
commented
Feb 9, 2026
run buildall |
hello-stephen
commented
Feb 9, 2026
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
doris-robot
commented
Feb 9, 2026
TPC-H: Total hot run time: 30338 ms |
doris-robot
commented
Feb 9, 2026
ClickBench: Total hot run time: 28.07 s |
hello-stephen
commented
Feb 9, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Feb 9, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
doris-robot
commented
Mar 18, 2026
TPC-DS: Total hot run time: 167825 ms |
mymeiyi
commented
Mar 18, 2026
run external |
hello-stephen
commented
Mar 18, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Mar 18, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
Uh oh!
There was an error while loading. Please reload this page.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)