Uh oh!
There was an error while loading. Please reload this page.
[fix](cloud) Skip empty rowsets before accessor lookup in batch delete_rowset_data - #60919
Conversation
…e_rowset_data
Empty rowsets produced by base compaction of empty rowsets have no
resource_id set (rowset_meta_size=181). When batch delete_rowset_data
encounters these rowsets, accessor_map_.find("") fails and sets ret=-1,
which causes the caller lambda to skip txn_remove for the entire batch.
This prevents recycle KV keys from being cleaned up, creating a
perpetual loop where the same rowsets are scanned every recycle round.
Move the num_segments <= 0 check before the accessor_map_ lookup so
these empty rowsets are safely skipped without poisoning the batch
return value.hello-stephen
commented
Feb 28, 2026
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
There was a problem hiding this comment.
Pull request overview
Fixes batch delete_rowset_data behavior in the cloud recycler by skipping empty rowsets (e.g., base compaction outputs with num_segments=0 and unset resource_id) before doing an accessor_map_ lookup, preventing a spurious batch failure that can block recycle KV cleanup and cause repeated rescans.
Changes:
- Move the
num_segments <= 0early-continue check ahead ofaccessor_map_.find(rs.resource_id())in batch rowset deletion. - Add explanatory comments describing the failure mode and why the ordering matters.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
liaoxin01
commented
Feb 28, 2026
run buildall |
doris-robot
commented
Feb 28, 2026
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
PR approved by anyone and no changes requested. |
PR approved by at least one committer and no changes requested. |
Uh oh!
There was an error while loading. Please reload this page.
…e_rowset_data (#60919) ## Proposed changes In batch `delete_rowset_data`, empty rowsets (e.g. base compaction output of empty rowsets) have `num_segments=0` and no `resource_id` set. The `accessor_map_.find("")` fails and sets `ret=-1`, which causes the caller to skip `txn_remove` for the entire batch. This prevents recycle KV keys from being cleaned up, creating a perpetual loop where the same rowsets are scanned every recycle round. The fix moves the `num_segments <= 0` check before the `accessor_map_` lookup so these empty rowsets are safely skipped without poisoning the batch return value. ## Problem summary - Empty rowsets from base compaction have `resource_id=""`, `rowset_meta_size=181`, `num_segments=0` - `accessor_map_.find("")` fails, sets `ret = -1`, `txn_remove` skipped for entire batch - Normal rowsets in the same batch: object storage data already deleted, but recycle KV not cleaned up - Next recycle round re-scans the same rowsets, forming a dead loop
Proposed changes
In batch
delete_rowset_data, empty rowsets (e.g. base compaction output of empty rowsets) havenum_segments=0and noresource_idset. Theaccessor_map_.find("")fails and setsret=-1, which causes the caller to skiptxn_removefor the entire batch. This prevents recycle KV keys from being cleaned up, creating a perpetual loop where the same rowsets are scanned every recycle round.The fix moves the
num_segments <= 0check before theaccessor_map_lookup so these empty rowsets are safely skipped without poisoning the batch return value.Problem summary
resource_id="",rowset_meta_size=181,num_segments=0accessor_map_.find("")fails, setsret = -1,txn_removeskipped for entire batch