Skip to content

branch-4.0: [fix](be) Poll packed file async close without blocking #62938 - #63531

Merged
morningman merged 1 commit into
apache:branch-4.0from
wyxxxcat:pick_62938_4_0
May 28, 2026
Merged

branch-4.0: [fix](be) Poll packed file async close without blocking #62938#63531
morningman merged 1 commit into
apache:branch-4.0from
wyxxxcat:pick_62938_4_0

Conversation

@wyxxxcat

@wyxxxcatwyxxxcat commented May 22, 2026

Copy link
Copy Markdown
Collaborator

pick: #62938

@wyxxxcat
[fix](be) Poll packed file async close without blocking (apache#62938)
cadce9f
fix: apache#57770
This change adds a non-blocking try_finish_close() hook to FileWriter
and implements it for S3FileWriter. PackedFileManager now polls the
async close result instead of calling close(true) repeatedly while
packed files
are uploading.
The change avoids re-submitting async close work and lets packed file
upload state transition only after the previous async close has actually
completed. It also propagates async close failures back into the packed
file state so failed uploads are recorded correctly.
A ut test was added to verify that PackedFileManager polls async close
without blocking or issuing an extra close call, and that async close
failures are handled as packed file upload failures.
@wyxxxcat

wyxxxcat commented May 22, 2026

Copy link
Copy Markdown
CollaboratorAuthor

run buildall

@hello-stephen

hello-stephen commented May 22, 2026

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 80.95% (51/63) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.37% (25458/35671)
Line Coverage54.15% (269318/497318)
Region Coverage51.60% (222391/430969)
Branch Coverage53.04% (95756/180541)

@wyxxxcat

wyxxxcat commented May 26, 2026

Copy link
Copy Markdown
CollaboratorAuthor

/review

github-actions[bot]
github-actionsBot previously requested changes May 26, 2026
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found blocking issues in this PR. Critical checkpoints: build correctness is at risk because a newly introduced std::this_thread usage is missing its required standard header; regression coverage is not reliable because the new cloud packed-file test disables packed files and therefore does not exercise the PackedFileManager async-close path being changed. Existing review context had no prior inline threads, and there were no additional user focus points to address.

Comment threadbe/src/olap/rowset/segment_creator.cpp
Comment threadregression-test/suites/cloud_p0/packed_file/test_packed_file_async_close_error.groovy
@morningman
morningman merged commit ead1da6 into apache:branch-4.0May 28, 2026
31 of 33 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningmanmorningmanmorningman approved these changes
@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@yiguoleiyiguoleiAwaiting requested review from yiguolei

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@wyxxxcat@hello-stephen@morningman