[opt](rowset) Remote fetch rowsets to avoid -230 error when capturing rowsets (#52995) by Hastyshell · Pull Request #52440 · apache/doris

Hastyshell · 2025-06-27T09:08:12Z

What problem does this PR solve?

Problem Summary:

Make stale rowsets accessible across BEs to avoid E-230 (versions already merged) in Read-Write Splitting senario.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

… rowsets (apache#3967)

Thearas · 2025-06-27T09:08:25Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Hastyshell · 2025-06-27T09:34:49Z

run buildall

hello-stephen · 2025-06-27T09:41:02Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.22% (1106/1329)
Line Coverage	66.74% (18799/28166)
Region Coverage	66.69% (9284/13922)
Branch Coverage	56.45% (5041/8930)

Hastyshell · 2025-06-27T12:01:44Z

run buildall

Hastyshell · 2025-06-27T12:12:49Z

run buildall

Hastyshell · 2025-06-30T02:49:17Z

run buildall

Hastyshell · 2025-06-30T06:26:21Z

run buildall

hello-stephen · 2025-06-30T06:43:49Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	82.63% (1118/1353)
Line Coverage	66.51% (19485/29296)
Region Coverage	65.61% (9325/14212)
Branch Coverage	55.58% (5073/9128)

Hastyshell · 2025-07-01T00:44:01Z

run buildall

doris-robot · 2025-07-01T01:07:07Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	82.63% (1118/1353)
Line Coverage	66.48% (19475/29296)
Region Coverage	65.59% (9322/14212)
Branch Coverage	55.49% (5065/9128)

Hastyshell · 2025-07-01T02:26:19Z

run buildall

Hastyshell · 2025-07-01T02:32:32Z

run buildall

hello-stephen · 2025-07-01T02:32:48Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	82.63% (1118/1353)
Line Coverage	66.47% (19472/29296)
Region Coverage	65.52% (9311/14212)
Branch Coverage	55.48% (5064/9128)

Hastyshell · 2025-07-01T02:42:05Z

run buildall

doris-robot · 2025-07-01T02:54:57Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	82.63% (1118/1353)
Line Coverage	66.46% (19470/29296)
Region Coverage	65.58% (9320/14212)
Branch Coverage	55.50% (5066/9128)

github-actions · 2025-07-01T03:09:10Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-07-01T03:09:12Z

PR approved by anyone and no changes requested.

doris-robot · 2025-07-02T09:37:36Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	82.67% (1121/1356)
Line Coverage	66.45% (19491/29332)
Region Coverage	65.88% (9609/14585)
Branch Coverage	55.43% (5062/9132)

hello-stephen · 2025-07-02T11:03:00Z

BE UT Coverage Report

Increment line coverage 16.75% (99/591) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	41.21% (10946/26562)
Line Coverage	31.99% (93752/293102)
Region Coverage	31.10% (48340/155448)
Branch Coverage	27.54% (24755/89874)

github-actions · 2025-07-03T03:21:24Z

PR approved by at least one committer and no changes requested.

dataroaring

LGTM

…52582) ### What problem does this PR solve? Currently, `DeleteBitmap`'s wrong move assignment operator can cause correctness problem on mow table if SC's alter process produce delete bitmaps when calculating delete bitmaps for incremental rowsets after the modification in `CloudSchemaChangeJob::_process_delete_bitmap` of #52440 because `origin_dbm` and `delete_bitmap` refer to the same `DeleteBitmap` object. ![image](https://github.com/user-attachments/assets/254d975a-594a-4b47-b028-64e47a05cb72)

### What problem does this PR solve? Fix 1. cloud_decommission increased it robustness 2. test_cloud_full_compaction_multi_segments, test_cloud_mow_retry_txn_interleave, test_cloud_concurrent_calc_dbm_task fix its not set fe debug point 3. fix CloudTablet.capture_rs_readers.return.e-230 debug point not work, due to #52440,

Fix 1. cloud_decommission increased it robustness 2. test_cloud_full_compaction_multi_segments, test_cloud_mow_retry_txn_interleave, test_cloud_concurrent_calc_dbm_task fix its not set fe debug point 3. fix CloudTablet.capture_rs_readers.return.e-230 debug point not work, due to apache#52440,

…kends except for warmup jobs (#54131) ### What problem does this PR solve? Problem Summary: Fix logic conflict of #52514 and #52440 ### Release note None ### Check List (For Author) - Test  - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason  - Behavior changed: - [ ] No. - [ ] Yes.  - Does this need documentation? - [ ] No. - [ ] Yes.  ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label

It is intruduced by apache#40716 while it is dismissed by apache#52440.

It is intruduced by #40716 while it is dismissed by #52440.

It is intruduced by apache#40716 while it is dismissed by apache#52440.

Fix 1. cloud_decommission increased it robustness 2. test_cloud_full_compaction_multi_segments, test_cloud_mow_retry_txn_interleave, test_cloud_concurrent_calc_dbm_task fix its not set fe debug point 3. fix CloudTablet.capture_rs_readers.return.e-230 debug point not work, due to apache#52440, pick from apache#54227

… rowsets (#52995) Related PR: #52440 In read-write splitting scenarios, some BE (Backend) nodes may have already merged certain rowset versions, while another BE still attempts to capture or access those rowsets. When this happens, the BE reports error E-230 (versions already merged), causing data access or synchronization to fail. This PR introduces a remote rowset fetching mechanism, allowing a BE that lacks the required rowset to fetch it from other BE nodes, instead of failing with E-230. - Added a remote fetch mechanism in the rowset management layer: When a BE detects that a rowset is missing locally but has already been merged, it will try to fetch the rowset from other BE nodes. - Updated version and state checking logic to correctly identify the “merged but missing” condition. - Adjusted the rowset access path to trigger remote fetch rather than throwing an immediate error. - Added tests (unit/integration) to cover the new logic where applicable. - Ensured backward compatibility: If the BE already has the rowset locally or read-write splitting is not enabled, the behavior remains unchanged. ### Release note Introduce a remote rowset fetching mechanism to prevent E-230 (“versions already merged”) errors in read-write splitting scenarios. This improves BE fault tolerance when some nodes have merged versions that others have not yet synchronized.

… rowsets (apache#52995) Related PR: apache#52440 In read-write splitting scenarios, some BE (Backend) nodes may have already merged certain rowset versions, while another BE still attempts to capture or access those rowsets. When this happens, the BE reports error E-230 (versions already merged), causing data access or synchronization to fail. This PR introduces a remote rowset fetching mechanism, allowing a BE that lacks the required rowset to fetch it from other BE nodes, instead of failing with E-230. - Added a remote fetch mechanism in the rowset management layer: When a BE detects that a rowset is missing locally but has already been merged, it will try to fetch the rowset from other BE nodes. - Updated version and state checking logic to correctly identify the “merged but missing” condition. - Adjusted the rowset access path to trigger remote fetch rather than throwing an immediate error. - Added tests (unit/integration) to cover the new logic where applicable. - Ensured backward compatibility: If the BE already has the rowset locally or read-write splitting is not enabled, the behavior remains unchanged. Introduce a remote rowset fetching mechanism to prevent E-230 (“versions already merged”) errors in read-write splitting scenarios. This improves BE fault tolerance when some nodes have merged versions that others have not yet synchronized.

… rowsets (apache#52995) Related PR: apache#52440 In read-write splitting scenarios, some BE (Backend) nodes may have already merged certain rowset versions, while another BE still attempts to capture or access those rowsets. When this happens, the BE reports error E-230 (versions already merged), causing data access or synchronization to fail. This PR introduces a remote rowset fetching mechanism, allowing a BE that lacks the required rowset to fetch it from other BE nodes, instead of failing with E-230. - Added a remote fetch mechanism in the rowset management layer: When a BE detects that a rowset is missing locally but has already been merged, it will try to fetch the rowset from other BE nodes. - Updated version and state checking logic to correctly identify the “merged but missing” condition. - Adjusted the rowset access path to trigger remote fetch rather than throwing an immediate error. - Added tests (unit/integration) to cover the new logic where applicable. - Ensured backward compatibility: If the BE already has the rowset locally or read-write splitting is not enabled, the behavior remains unchanged. ### Release note Introduce a remote rowset fetching mechanism to prevent E-230 (“versions already merged”) errors in read-write splitting scenarios. This improves BE fault tolerance when some nodes have merged versions that others have not yet synchronized.

[opt](rowset) Remote fetch rowsets to avoid -230 error when capturing…
ff0ab70
… rowsets (apache#3967)

Hastyshell requested a review from dataroaring as a code owner June 27, 2025 09:08

merge-3.0
5dedf0a

Update cloud_tablet_mgr.h
c11f009

Hastyshell added 2 commits June 27, 2025 20:06

Update cloud_meta_mgr.cpp
f7e64f3

Update storage_engine.cpp
5959dea

Hastyshell changed the title ~~[opt](rowset) Remote fetch rowsets to avoid -230 error when capturing rowsets (#3967)~~[opt](rowset) Remote fetch rowsets to avoid -230 error when capturing rowsetsJun 27, 2025

Merge branch 'branch-3.0' into fix-E-230-3.0
1128ea5

fix compile error
2f5bb77

fix license
ded249d

gavinchou previously approved these changes Jul 1, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 1, 2025

github-actionsBot added the reviewed label Jul 1, 2025

gavinchou approved these changes Jul 3, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 3, 2025

dataroaring approved these changes Jul 7, 2025
View reviewed changes

dataroaring merged commit 0e477b6 into apache:branch-3.0Jul 7, 2025

bobhan1 mentioned this pull request Jul 7, 2025
[Fix](mow) Fix DeleteBitmap's assignment operator and constructor #52582
Merged
16 tasks

Hastyshell changed the title ~~[opt](rowset) Remote fetch rowsets to avoid -230 error when capturing rowsets~~[opt](rowset) Remote fetch rowsets to avoid -230 error when capturing rowsets (#52995)Jul 9, 2025

Hastyshell mentioned this pull request Jul 31, 2025
[fix](replica) Get tablet replica infos should return all primary backends except for warmup jobs #54131
Merged
16 tasks

deardeng mentioned this pull request Aug 1, 2025
[Fix](case) Fix some docker case #54227
Merged
16 tasks

dataroaring pushed a commit to dataroaring/incubator-doris that referenced this pull request Aug 13, 2025
[fix](errmsg) recover errmsg for -230
37ea9ea
It is intruduced by apache#40716 while it is dismissed by apache#52440.

dataroaring mentioned this pull request Aug 13, 2025
[fix](errmsg) recover errmsg for -230 #54697
Merged
16 tasks

gavinchou pushed a commit that referenced this pull request Aug 14, 2025
[fix](errmsg) recover errmsg for -230 (#54697)
4d84afd
It is intruduced by #40716 while it is dismissed by #52440.

dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Aug 14, 2025
[fix](errmsg) recover errmsg for -230 (apache#54697)
ed49bdf
It is intruduced by apache#40716 while it is dismissed by apache#52440.

dataroaring mentioned this pull request Aug 14, 2025
branch-3.1: [fix](errmsg) recover errmsg for -230 #54697 #54731
Merged

gavinchou mentioned this pull request Aug 15, 2025
Release Note 3.0.7 #54868
Closed

Hastyshell mentioned this pull request Oct 20, 2025
[opt](rowset) Remote fetch rowsets to avoid -230 error when capturing rowsets #52995
Merged
16 tasks

Hastyshell mentioned this pull request Oct 23, 2025
[opt](rowset) Remote fetch rowsets to avoid -230 error when capturing rowsets (#52995) #57271
Merged
16 tasks

Uh oh!

Conversation

Hastyshell commented Jun 27, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Jun 27, 2025

Uh oh!

Hastyshell commented Jun 27, 2025

Uh oh!

hello-stephen commented Jun 27, 2025

Cloud UT Coverage Report

Uh oh!

Hastyshell commented Jun 27, 2025

Uh oh!

Hastyshell commented Jun 27, 2025

Uh oh!

Hastyshell commented Jun 30, 2025

Uh oh!

Hastyshell commented Jun 30, 2025

Uh oh!

hello-stephen commented Jun 30, 2025

Cloud UT Coverage Report

Uh oh!

Hastyshell commented Jul 1, 2025

Uh oh!

doris-robot commented Jul 1, 2025

Cloud UT Coverage Report

Uh oh!

Hastyshell commented Jul 1, 2025

Uh oh!

Hastyshell commented Jul 1, 2025

Uh oh!

hello-stephen commented Jul 1, 2025

Cloud UT Coverage Report

Uh oh!

Hastyshell commented Jul 1, 2025

Uh oh!

doris-robot commented Jul 1, 2025

Cloud UT Coverage Report

Uh oh!

github-actionsBot commented Jul 1, 2025

Uh oh!

github-actionsBot commented Jul 1, 2025

Uh oh!

doris-robot commented Jul 2, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Jul 2, 2025

BE UT Coverage Report

Uh oh!

github-actionsBot commented Jul 3, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Hastyshell commented Jun 27, 2025•
edited
Loading