Uh oh!
There was an error while loading. Please reload this page.
[fix](cloud) avoid broadcast remote read in topn query - #58044
Merged
dataroaring merged 4 commits intoNov 19, 2025
Conversation
Thearas
commented
Nov 14, 2025
Contributor
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
freemandealer
commented
Nov 14, 2025
MemberAuthor
run buildall |
hello-stephen
commented
Nov 14, 2025
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
dataroaring
commented
Nov 14, 2025
Contributor
Pls add a test to avoid duplicated bug in the future. |
hello-stephen
commented
Nov 14, 2025
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
gavinchou
previously approved these changes
Nov 15, 2025
Contributor
PR approved by at least one committer and no changes requested. |
Contributor
PR approved by anyone and no changes requested. |
doris-robot
commented
Nov 15, 2025
TPC-H: Total hot run time: 34264 ms |
doris-robot
commented
Nov 15, 2025
TPC-DS: Total hot run time: 188559 ms |
doris-robot
commented
Nov 15, 2025
ClickBench: Total hot run time: 27.65 s |
Topn lazy read only read the cache in cloud mode to reduce tablet cache read. fixapache#50803 failed to pass force_use_only_cached to tablet_mgr Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
freemandealerforce-pushed
the
fix-get-tablet-download-cache
branch
from
November 18, 2025 07:50
25b4af0 to
2bcdb0dComparefreemandealer
commented
Nov 18, 2025
MemberAuthor
run buildall |
Contributor
PR approved by at least one committer and no changes requested. |
doris-robot
commented
Nov 18, 2025
TPC-H: Total hot run time: 34772 ms |
doris-robot
commented
Nov 18, 2025
ClickBench: Total hot run time: 27.8 s |
hello-stephen
commented
Nov 18, 2025
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Nov 18, 2025
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
gavinchou
approved these changes
Nov 19, 2025
Contributor
PR approved by at least one committer and no changes requested. |
Uh oh!
There was an error while loading. Please reload this page.
github-actionsBot
pushed a commit
that referenced
this pull request
Nov 19, 2025
Topn lazy read only read the cache in cloud mode to reduce tablet cache read. fix#50803 failed to pass force_use_only_cached to tablet_mgr Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
github-actionsBot
pushed a commit
that referenced
this pull request
Nov 19, 2025
Topn lazy read only read the cache in cloud mode to reduce tablet cache read. fix#50803 failed to pass force_use_only_cached to tablet_mgr Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
yiguolei pushed a commit
that referenced
this pull request
Nov 20, 2025
morrySnow pushed a commit
that referenced
this pull request
Nov 25, 2025
morrySnow pushed a commit
that referenced
this pull request
Nov 25, 2025
Closed
nagisa-kunhah pushed a commit
to nagisa-kunhah/doris
that referenced
this pull request
Dec 14, 2025
Topn lazy read only read the cache in cloud mode to reduce tablet cache read. fixapache#50803 failed to pass force_use_only_cached to tablet_mgr Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
w41ter pushed a commit
to w41ter/incubator-doris
that referenced
this pull request
Dec 26, 2025
…pache#5786) pick: apache#58044 Topn lazy read only read the cache in cloud mode to reduce tablet cache read. fixapache#50803 failed to pass force_use_only_cached to tablet_mgr Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
16 tasks
zhaorongsheng pushed a commit
to zhaorongsheng/doris
that referenced
this pull request
Jun 4, 2026
…pache#62886) apache#58044 changed the error message when tablet is not found in cache but don't change the condition in `CloudInternalServiceImpl::warm_up_rowset`. This will cause passive warm up don't retry because the status code is not `TABLE_NOT_FOUND`. This PR fix it.
uchenily added a commit
to uchenily/doris
that referenced
this pull request
Aug 19, 2026
…s in topn deferred materialization In cloud mode, the old topn deferred materialization path (DeferMaterializeTopNResult, enabled when topn_lazy_materialization_threshold=0) broadcasts a multiget_data RPC to all BEs in the compute group. Each BE uses force_use_only_cached=true to only read the local tablet cache, avoiding meta-service RPCs on BEs that do not hold the tablet (optimization introduced by apache#50803, parameter-passing bug fixed by apache#58044). This optimization assumes the tablet cache is enabled and that the BE which performed the scan phase still has the tablet cached by the time the deferred-materialization RPC arrives. When this assumption breaks, every BE returns 0 rows and the coordinator reports: Miss matched return row loc count 0, expected 0, input N Two scenarios break the assumption: 1. tablet_cache_capacity=0: CloudTabletMgr falls back to DummyLRUCache (every insert and lookup is a no-op), so force_use_only_cached always misses -- even on the BE that performed the scan. This is a silent misconfiguration with no visible warning. 2. Cache eviction under a small tablet_cache_capacity: in a multi-BE cluster the scan phase loads the tablet into the scanning BE's cache, but between the scan and the deferred- materialization RPC other concurrent queries may load enough tablets to evict it. The scanning BE then misses just like the non-scanning BEs, and since it is the only BE that holds the rowset data, the coordinator cannot recover the rows. (In a normal multi-BE cluster, BEs that did not perform the scan are *expected* to miss the cache and skip -- the coordinator merges results from the scanning BE. The bug only manifests when the scanning BE itself misses, which happens with cache disabled or evicted.) This patch makes the old path resilient without giving up the apache#50803 optimization: rowid_fetcher.cpp: when force_use_only_cached=true misses, fall back to a normal get_tablet() load (force_use_only_cached=false). The cache-only fast path is still attempted first, so the broadcast-remote-read avoidance is preserved when the cache hits. The fallback only pays a meta-service RPC on cache miss, which is the same behavior as the new multiget_data_v2 path (apache#55870). Reproduction ------------ Environment: single-node cloud-mode cluster CREATE TABLE t (k int, v int) DUPLICATE KEY(k) DISTRIBUTED BY HASH(k) BUCKETS 1 PROPERTIES("replication_num"="1"); INSERT INTO t VALUES (1, 1), (2, 2); Set tablet_cache_capacity=0 in be_custom.conf and restart BE. Before fix: mysql> SET enable_sql_cache=false; mysql> SET topn_lazy_materialization_threshold=0; mysql> SELECT k,v FROM t ORDER BY k LIMIT 10; ERROR 1105 (HY000): errCode = 2, detailMessage = (127.0.0.1)[INTERNAL_ERROR] Miss matched return row loc count 0, expected 0, input 2 After fix: mysql> SET enable_sql_cache=false; mysql> SET topn_lazy_materialization_threshold=0; mysql> SELECT k,v FROM t ORDER BY k LIMIT 10; +------+------+ | k | v | +------+------+ | 1 | 1 | | 2 | 2 | +------+------+
16 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Topn lazy read only read the cache in cloud mode to reduce tablet cache read. fix#50803 failed to pass force_use_only_cached to tablet_mgr
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)