Skip to content

branch-4.0: [fix](filecache) avoid BE crash when finalize misses local cache writer #62389 - #63178

Merged
morningman merged 3 commits into
branch-4.0from
auto-pick-62389-branch-4.0
Jun 10, 2026
Merged

branch-4.0: [fix](filecache) avoid BE crash when finalize misses local cache writer #62389#63178
morningman merged 3 commits into
branch-4.0from
auto-pick-62389-branch-4.0

Conversation

@github-actions

@github-actionsgithub-actionsBot commented May 12, 2026

Copy link
Copy Markdown
Contributor

Cherry-picked from #62389

@freemandealerYour Name
[fix](filecache) avoid BE crash when finalize misses local cache writ…
9be60fd
…er (#62389)
## Problem
`FSFileCacheStorage::finalize()` relied on `DCHECK` to assume the writer
entry always existed in `_key_to_writer`. In release builds a missing
writer entry could still fall through, dereference `end()`, and crash
BE.
## Root Cause
When `(hash, offset)` is missing from `_key_to_writer`, the current
finalize path moves and erases through an invalid iterator. The target
branch already uses the newer `finalize(key, size)` signature, so this
PR ports the hardening change to that path.
## Solution
- return `InternalError` immediately when finalize cannot find the
writer entry
- include hash, offset, cache type, and expiration time in the error
message for diagnosis
- keep the normal finalize path unchanged when the writer exists
- rely on the existing `FileBlock::set_downloaded()` cleanup path to
abort the pending cache write and reset block state
## Tests
-
`BlockFileCacheTest.fs_file_cache_storage_finalize_missing_writer_returns_error`
- `DORIS_TOOLCHAIN=clang DISABLE_BE_JAVA_EXTENSIONS=ON
ENABLE_INJECTION_POINT=ON ENABLE_CACHE_LOCK_DEBUG=0 ENABLE_PCH=0 sh
run-be-ut.sh --clean --run
--filter=BlockFileCacheTest.fs_file_cache_storage_finalize_missing_writer_returns_error
-j 39`
- Source patch: `selectdb/selectdb-core#8321`
@hello-stephen

hello-stephen commented May 12, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hello-stephen

hello-stephen commented May 12, 2026

Copy link
Copy Markdown
Contributor

run buildall

1 similar comment
@freemandealer

freemandealer commented May 28, 2026

Copy link
Copy Markdown
Member

run buildall

@freemandealer

freemandealer commented May 29, 2026

Copy link
Copy Markdown
Member

run buildall

@freemandealer

freemandealer commented May 29, 2026

Copy link
Copy Markdown
Member

run buildall

@hello-stephen

hello-stephen commented May 29, 2026

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 100.00% (7/7) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage53.23% (19410/36466)
Line Coverage36.36% (181272/498505)
Region Coverage32.98% (140848/427134)
Branch Coverage33.85% (60953/180088)

@hello-stephen

hello-stephen commented May 29, 2026

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (7/7) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.49% (25521/35699)
Line Coverage54.32% (270326/497622)
Region Coverage51.78% (223430/431460)
Branch Coverage53.29% (96298/180721)

@freemandealer

freemandealer commented May 31, 2026

Copy link
Copy Markdown
Member

run cloud_p0

@freemandealer

freemandealer commented May 31, 2026

Copy link
Copy Markdown
Member

run p0

@freemandealer

freemandealer commented May 31, 2026

Copy link
Copy Markdown
Member

run nonConcurrent

@hello-stephen

hello-stephen commented May 31, 2026

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (7/7) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.49% (25521/35699)
Line Coverage54.32% (270326/497622)
Region Coverage51.78% (223430/431460)
Branch Coverage53.29% (96298/180721)

@hello-stephen

hello-stephen commented May 31, 2026

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (7/7) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.50% (25525/35699)
Line Coverage54.34% (270413/497622)
Region Coverage51.83% (223625/431460)
Branch Coverage53.31% (96344/180721)

@hello-stephen

hello-stephen commented May 31, 2026

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (7/7) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.51% (25528/35699)
Line Coverage54.35% (270478/497622)
Region Coverage51.87% (223807/431460)
Branch Coverage53.34% (96400/180721)

@freemandealer

freemandealer commented Jun 1, 2026

Copy link
Copy Markdown
Member

merge after 4.0.6 release

@freemandealer

freemandealer commented Jun 1, 2026

Copy link
Copy Markdown
Member

run cloud_p0

@morningman
morningman merged commit 5b73b7d into branch-4.0Jun 10, 2026
28 of 32 checks passed
@github-actions
github-actionsBot deleted the auto-pick-62389-branch-4.0 branch June 10, 2026 09:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningmanmorningmanmorningman approved these changes
@yiguoleiyiguoleiAwaiting requested review from yiguolei

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hello-stephen@freemandealer@morningman