Uh oh!
There was an error while loading. Please reload this page.
[improvement](filecache) limit file cache LRU replay queues - #64381
Conversation
hello-stephen
commented
Jun 10, 2026
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
freemandealer
commented
Jun 10, 2026
run buildall |
### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: File cache LRU log replay needs tighter default replay latency, bounded in-memory queues, and observability for pending block LRU updates and LRU log replay. This change lowers the replay interval default to 1 ms, adds hard caps for the pending block update queue and per-type LRU log queues, preserves existing LRU log backlog when tail recording is disabled, and exposes queue length plus monotonic produce/consume/idle counters. QPS bvars are not added because Prometheus can derive rates from the counters. Tests that depended on background replay timing now use a deterministic single-replay helper with a high background interval. ### Release note Add bounded file cache LRU recorder queues and replay/update queue count metrics. ### Check List (For Author) - Test: Manual test - `PATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH build-support/clang-format.sh` - `PATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH build-support/check-format.sh` - Focused BE UT rerun was started but stopped during compilation at reviewer request; no completed BE UT result after the latest metric cleanup. - Behavior changed: Yes. File cache LRU replay defaults to 1 ms, LRU log and block update queues are capped at 500000 by default, and disabling tail record enqueueing no longer clears existing replay backlog. - Does this need documentation: No
ff4fde7 to
de23f15Comparefreemandealer
commented
Jun 10, 2026
run buildall |
freemandealer
commented
Jun 10, 2026
/review |
There was a problem hiding this comment.
Automated review result: request changes.
I found one blocking correctness issue in the LRU recorder overflow/disable path. The patch bounds memory, but when a record is dropped the shadow LRU queue can diverge from the real cache and later be dumped/restored as authoritative metadata. See inline comment.
Critical checkpoint conclusions:
- Goal/test: the bounded queue/metrics goal is mostly implemented, but the lossy record path does not preserve dump/restore correctness. Existing tests cover caps and deterministic replay, but not overflow/drop followed by dump/restore or dynamic disable/reenable of tail recording.
- Scope: the code change is focused on file-cache LRU queue bounds and observability.
- Concurrency: NeedUpdateLRUBlocks size reservation and drain accounting look internally consistent; no lock-order issue found in the reviewed paths.
- Lifecycle/static initialization: no new static initialization or ownership lifecycle issue found.
- Configuration: new configs are mutable and read dynamically; however cap exhaustion or tail-record disabling needs a resync/invalid-state plan before later dumps.
- Compatibility/storage: no file format change, but stale shadow dumps can create incompatible in-memory cache state after restart.
- Parallel code paths: background replay and the new one-shot helper use the same replay implementation.
- Observability: queue length/produce/consume/idle metrics were added and are directionally useful.
- Transaction/data-write concerns: not applicable for this file-cache-only change.
- Performance: queue caps reduce memory risk; the 1 ms idle replay interval should be watched, but I did not find a separate blocking performance issue.
- User focus: no additional user-provided review focus was supplied.
I did not run local tests. Current GitHub checks show macOS BE UT failed before build with ERROR: The JAVA version is 25, it must be JDK-17; TeamCity BE UT/compile checks were still pending when reviewed.
Uh oh!
There was an error while loading. Please reload this page.
hello-stephen
commented
Jun 10, 2026
TPC-H: Total hot run time: 28852 ms |
hello-stephen
commented
Jun 10, 2026
TPC-DS: Total hot run time: 168581 ms |
hello-stephen
commented
Jun 10, 2026
TPC-H: Total hot run time: 29063 ms |
The invalid shadow queue dump will be cured automatically. So this won't be a serious problem.
hello-stephen
commented
Jun 10, 2026
TPC-DS: Total hot run time: 168500 ms |
hello-stephen
commented
Jun 10, 2026
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Jun 10, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Jun 10, 2026
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
Uh oh!
There was an error while loading. Please reload this page.
Problem Summary: File cache LRU log replay needs tighter default replay latency, bounded in-memory queues, and observability for pending block LRU updates and LRU log replay. This change lowers the replay interval default to 1 ms, adds hard caps for the pending block update queue and per-type LRU log queues, preserves existing LRU log backlog when tail recording is disabled, and exposes queue length plus monotonic produce/consume/idle counters. QPS bvars are not added because Prometheus can derive rates from the counters. Tests that depended on background replay timing now use a deterministic single-replay helper with a high background interval.
### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: File cache LRU log replay needs tighter default replay latency, bounded in-memory queues, and observability for pending block LRU updates and LRU log replay. This change lowers the replay interval default to 1 ms, adds hard caps for the pending block update queue and per-type LRU log queues, preserves existing LRU log backlog when tail recording is disabled, and exposes queue length plus monotonic produce/consume/idle counters. QPS bvars are not added because Prometheus can derive rates from the counters. Tests that depended on background replay timing now use a deterministic single-replay helper with a high background interval. ### Release note Add bounded file cache LRU recorder queues and replay/update queue count metrics. ### Check List (For Author) - Test: Manual test - `PATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH build-support/clang-format.sh` - `PATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH build-support/check-format.sh` - Focused BE UT rerun was started but stopped during compilation at reviewer request; no completed BE UT result after the latest metric cleanup. - Behavior changed: Yes. File cache LRU replay defaults to 1 ms, LRU log and block update queues are capped at 500000 by default, and disabling tail record enqueueing no longer clears existing replay backlog. - Does this need documentation: No (cherry picked from commit de23f15) Picked-from: apache#64381
Problem Summary: File cache LRU log replay needs tighter default replay latency, bounded in-memory queues, and observability for pending block LRU updates and LRU log replay. This change lowers the replay interval default to 1 ms, adds hard caps for the pending block update queue and per-type LRU log queues, preserves existing LRU log backlog when tail recording is disabled, and exposes queue length plus monotonic produce/consume/idle counters. QPS bvars are not added because Prometheus can derive rates from the counters. Tests that depended on background replay timing now use a deterministic single-replay helper with a high background interval.
### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: File cache LRU log replay needs tighter default replay latency, bounded in-memory queues, and observability for pending block LRU updates and LRU log replay. This change lowers the replay interval default to 1 ms, adds hard caps for the pending block update queue and per-type LRU log queues, preserves existing LRU log backlog when tail recording is disabled, and exposes queue length plus monotonic produce/consume/idle counters. QPS bvars are not added because Prometheus can derive rates from the counters. Tests that depended on background replay timing now use a deterministic single-replay helper with a high background interval. ### Release note Add bounded file cache LRU recorder queues and replay/update queue count metrics. ### Check List (For Author) - Test: Manual test - `PATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH build-support/clang-format.sh` - `PATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH build-support/check-format.sh` - Focused BE UT rerun was started but stopped during compilation at reviewer request; no completed BE UT result after the latest metric cleanup. - Behavior changed: Yes. File cache LRU replay defaults to 1 ms, LRU log and block update queues are capped at 500000 by default, and disabling tail record enqueueing no longer clears existing replay backlog. - Does this need documentation: No (cherry picked from commit de23f15) Picked-from: apache#64381
| DORIS_CHECK_GE(cur_size, delta); | ||
| if (_size.compare_exchange_weak(cur_size, cur_size - delta, std::memory_order_relaxed)) { | ||
| return; | ||
| } |
| DORIS_CHECK_GT(cur_size, 0); | ||
| if (queue_size.compare_exchange_weak(cur_size, cur_size - 1, std::memory_order_relaxed)) { | ||
| return; | ||
| } |
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: File cache LRU log replay needs tighter default replay latency, bounded in-memory queues, and observability for pending block LRU updates and LRU log replay. This change lowers the replay interval default to 1 ms, adds hard caps for the pending block update queue and per-type LRU log queues, preserves existing LRU log backlog when tail recording is disabled, and exposes queue length plus monotonic produce/consume/idle counters. QPS bvars are not added because Prometheus can derive rates from the counters. Tests that depended on background replay timing now use a deterministic single-replay helper with a high background interval.
Release note
Add bounded file cache LRU recorder queues and replay/update queue count metrics.
Check List (For Author)
PATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH build-support/clang-format.shPATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH build-support/check-format.sh