[fix](cloud) Add SC_COMPACTION_CONFLICT error code to retry cross-V1 compaction failures by Yukang-Lian · Pull Request #62272 · apache/doris

Yukang-Lian · 2026-04-09T07:37:12Z

Summary

Add dedicated SC_COMPACTION_CONFLICT(101) TStatusCode for the cross-V1 compaction detection in cloud schema change
Include this error code in FE's schema change retry whitelist (AlterJobV2.getRetryTimes())
Previously, this transient condition returned INTERNAL_ERROR which FE does not retry, causing the schema change job to be permanently cancelled

Background

The cloud SC compaction optimization allows new tablets to do compaction during schema change queue wait. A safety check detects when a compaction rowset crosses the alter_version (V1) boundary. This is a transient condition - on retry, V1 will be higher and the crossing rowset falls within [2, V1'].

The bug was that FE treated INTERNAL_ERROR as a permanent failure (no retry), so this transient condition caused the entire schema change to be cancelled.

Changes

File	Change
`gensrc/thrift/Status.thrift`	Add `SC_COMPACTION_CONFLICT = 101`
`be/src/common/status.h`	Register `TStatusError(SC_COMPACTION_CONFLICT, false)`
`be/src/cloud/cloud_schema_change_job.cpp`	Cross-V1 check: `INTERNAL_ERROR` → `SC_COMPACTION_CONFLICT`
`fe/.../alter/AlterJobV2.java`	Add `SC_COMPACTION_CONFLICT` to retry whitelist
`fe/.../alter/AlterJobV2RetryTest.java`	Unit tests for `getRetryTimes()` (7 cases)
`regression-test/.../test_sc_compaction_cross_v1_retry.groovy`	E2E test: cross-V1 failure → retry → success

Test plan

FE unit test: AlterJobV2RetryTest - verifies retry whitelist for all error codes, null handling, and config toggle
Regression test: test_sc_compaction_cross_v1_retry - uses debug points to trigger cross-V1, verifies SC stays RUNNING under override, then succeeds after override removal. Asserts data integrity and schema correctness.

hello-stephen · 2026-04-09T07:37:17Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Hastyshell

LGTM

github-actions · 2026-04-10T07:25:12Z

PR approved by anyone and no changes requested.

Yukang-Lian · 2026-04-13T08:07:13Z

run buildall

hello-stephen · 2026-04-13T09:35:48Z

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-04-13T12:06:47Z

BE Regression && UT Coverage Report

Increment line coverage 11.11% (1/9) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.70% (27409/37189)
Line Coverage	57.35% (295985/516141)
Region Coverage	54.55% (246509/451883)
Branch Coverage	56.21% (106815/190044)

…compaction failures The cloud SC compaction optimization introduced a cross-V1 safety check that returns INTERNAL_ERROR when a compaction rowset crosses the alter_version boundary. However, FE does not retry INTERNAL_ERROR, causing the schema change job to be permanently cancelled. Add a dedicated SC_COMPACTION_CONFLICT(101) error code for this transient condition, and include it in FE's schema change retry whitelist. When the cross-V1 check triggers, abort the SC job in meta-service first so the next retry registers a fresh job with a higher alter_version.

…ion init Log per_row, sample_bytes, sample_rows immediately after all merge inputs finish loading their first block, before the actual merge starts. This helps diagnose memory issues by showing the actual per-row memory size at init time.

… compaction init" This reverts commit 6cfa629.

Yukang-Lian · 2026-04-15T03:37:21Z

run buildall

Yukang-Lian · 2026-04-15T07:16:02Z

run feut

Yukang-Lian · 2026-04-15T07:16:10Z

run nonConcurrent

Yukang-Lian · 2026-04-15T07:16:17Z

run p0

hello-stephen · 2026-04-15T07:20:00Z

FE Regression Coverage Report

Increment line coverage 0.00% (0/32) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-04-15T07:44:19Z

BE UT Coverage Report

Increment line coverage 60.00% (6/10) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.09% (20171/37996)
Line Coverage	36.67% (189984/518102)
Region Coverage	32.94% (147595/448122)
Branch Coverage	34.05% (64568/189628)

hello-stephen · 2026-04-15T07:48:25Z

BE Regression && UT Coverage Report

Increment line coverage 60.00% (6/10) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.95% (26773/37210)
Line Coverage	55.00% (284104/516518)
Region Coverage	52.06% (235444/452247)
Branch Coverage	53.46% (101681/190206)

hello-stephen · 2026-04-15T09:01:56Z

BE Regression && UT Coverage Report

Increment line coverage 60.00% (6/10) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.98% (26783/37210)
Line Coverage	55.00% (284110/516518)
Region Coverage	52.05% (235392/452247)
Branch Coverage	53.45% (101671/190206)

hello-stephen · 2026-04-15T09:05:16Z

BE Regression && UT Coverage Report

Increment line coverage 60.00% (6/10) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.98% (26783/37210)
Line Coverage	55.00% (284110/516518)
Region Coverage	52.05% (235392/452247)
Branch Coverage	53.45% (101671/190206)

hello-stephen · 2026-04-15T09:08:24Z

FE Regression Coverage Report

Increment line coverage 0.00% (0/32) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-04-15T09:17:08Z

FE Regression Coverage Report

Increment line coverage 0.00% (0/32) 🎉
Increment coverage report
Complete coverage report

Yukang-Lian · 2026-04-15T09:43:25Z

run p0

dismiss

…compaction failures (#62272) ## Summary - Add dedicated `SC_COMPACTION_CONFLICT(101)` TStatusCode for the cross-V1 compaction detection in cloud schema change - Include this error code in FE's schema change retry whitelist (`AlterJobV2.getRetryTimes()`) - Previously, this transient condition returned `INTERNAL_ERROR` which FE does not retry, causing the schema change job to be permanently cancelled ## Background The cloud SC compaction optimization allows new tablets to do compaction during schema change queue wait. A safety check detects when a compaction rowset crosses the alter_version (V1) boundary. This is a transient condition - on retry, V1 will be higher and the crossing rowset falls within `[2, V1']`. The bug was that FE treated `INTERNAL_ERROR` as a permanent failure (no retry), so this transient condition caused the entire schema change to be cancelled. ## Changes | File | Change | |------|--------| | `gensrc/thrift/Status.thrift` | Add `SC_COMPACTION_CONFLICT = 101` | | `be/src/common/status.h` | Register `TStatusError(SC_COMPACTION_CONFLICT, false)` | | `be/src/cloud/cloud_schema_change_job.cpp` | Cross-V1 check: `INTERNAL_ERROR` → `SC_COMPACTION_CONFLICT` | | `fe/.../alter/AlterJobV2.java` | Add `SC_COMPACTION_CONFLICT` to retry whitelist | | `fe/.../alter/AlterJobV2RetryTest.java` | Unit tests for `getRetryTimes()` (7 cases) | | `regression-test/.../test_sc_compaction_cross_v1_retry.groovy` | E2E test: cross-V1 failure → retry → success | ## Test plan - [x] FE unit test: `AlterJobV2RetryTest` - verifies retry whitelist for all error codes, null handling, and config toggle - [x] Regression test: `test_sc_compaction_cross_v1_retry` - uses debug points to trigger cross-V1, verifies SC stays RUNNING under override, then succeeds after override removal. Asserts data integrity and schema correctness.

…ry cross-V1 compaction failures #62272 (#62643) Cherry-picked from #62272 Co-authored-by: Jimmy <lianyukang@selectdb.com>

…compaction failures (apache#62272) ## Summary - Add dedicated `SC_COMPACTION_CONFLICT(101)` TStatusCode for the cross-V1 compaction detection in cloud schema change - Include this error code in FE's schema change retry whitelist (`AlterJobV2.getRetryTimes()`) - Previously, this transient condition returned `INTERNAL_ERROR` which FE does not retry, causing the schema change job to be permanently cancelled ## Background The cloud SC compaction optimization allows new tablets to do compaction during schema change queue wait. A safety check detects when a compaction rowset crosses the alter_version (V1) boundary. This is a transient condition - on retry, V1 will be higher and the crossing rowset falls within `[2, V1']`. The bug was that FE treated `INTERNAL_ERROR` as a permanent failure (no retry), so this transient condition caused the entire schema change to be cancelled. ## Changes | File | Change | |------|--------| | `gensrc/thrift/Status.thrift` | Add `SC_COMPACTION_CONFLICT = 101` | | `be/src/common/status.h` | Register `TStatusError(SC_COMPACTION_CONFLICT, false)` | | `be/src/cloud/cloud_schema_change_job.cpp` | Cross-V1 check: `INTERNAL_ERROR` → `SC_COMPACTION_CONFLICT` | | `fe/.../alter/AlterJobV2.java` | Add `SC_COMPACTION_CONFLICT` to retry whitelist | | `fe/.../alter/AlterJobV2RetryTest.java` | Unit tests for `getRetryTimes()` (7 cases) | | `regression-test/.../test_sc_compaction_cross_v1_retry.groovy` | E2E test: cross-V1 failure → retry → success | ## Test plan - [x] FE unit test: `AlterJobV2RetryTest` - verifies retry whitelist for all error codes, null handling, and config toggle - [x] Regression test: `test_sc_compaction_cross_v1_retry` - uses debug points to trigger cross-V1, verifies SC stays RUNNING under override, then succeeds after override removal. Asserts data integrity and schema correctness.

Cherry-pick the following PRs to `branch-4.0`: - #61696 [Feature](compaction) add CompactionTaskTracker with system table and HTTP API - #61621 [fix](metrics) Fix prepared statement QPS metrics not counted when audit log disabled - #62272 [fix](cloud) Add SC_COMPACTION_CONFLICT error code to retry cross-V1 compaction failures #61696 had conflicts due to the BE directory layout difference between master (`be/src/storage/...`) and branch-4.0 (`be/src/olap/...`). New files were relocated to the branch-4.0 paths; include paths and FE schema/thrift entries were merged accordingly.

…compaction failures (apache#62272) ## Summary - Add dedicated `SC_COMPACTION_CONFLICT(101)` TStatusCode for the cross-V1 compaction detection in cloud schema change - Include this error code in FE's schema change retry whitelist (`AlterJobV2.getRetryTimes()`) - Previously, this transient condition returned `INTERNAL_ERROR` which FE does not retry, causing the schema change job to be permanently cancelled ## Background The cloud SC compaction optimization allows new tablets to do compaction during schema change queue wait. A safety check detects when a compaction rowset crosses the alter_version (V1) boundary. This is a transient condition - on retry, V1 will be higher and the crossing rowset falls within `[2, V1']`. The bug was that FE treated `INTERNAL_ERROR` as a permanent failure (no retry), so this transient condition caused the entire schema change to be cancelled. ## Changes | File | Change | |------|--------| | `gensrc/thrift/Status.thrift` | Add `SC_COMPACTION_CONFLICT = 101` | | `be/src/common/status.h` | Register `TStatusError(SC_COMPACTION_CONFLICT, false)` | | `be/src/cloud/cloud_schema_change_job.cpp` | Cross-V1 check: `INTERNAL_ERROR` → `SC_COMPACTION_CONFLICT` | | `fe/.../alter/AlterJobV2.java` | Add `SC_COMPACTION_CONFLICT` to retry whitelist | | `fe/.../alter/AlterJobV2RetryTest.java` | Unit tests for `getRetryTimes()` (7 cases) | | `regression-test/.../test_sc_compaction_cross_v1_retry.groovy` | E2E test: cross-V1 failure → retry → success | ## Test plan - [x] FE unit test: `AlterJobV2RetryTest` - verifies retry whitelist for all error codes, null handling, and config toggle - [x] Regression test: `test_sc_compaction_cross_v1_retry` - uses debug points to trigger cross-V1, verifies SC stays RUNNING under override, then succeeds after override removal. Asserts data integrity and schema correctness.

Related PR: #62272 Problem Summary: Cloud schema change retries could keep registering a stale alter version after a cross-V1 compaction conflict. The retry path synced the base tablet only up to the request alter version, so a local base tablet whose max version already exceeded that request could skip refreshing newer visible rowsets. When a new schema-change tablet had a compaction rowset crossing the stale V1, every retry reused that V1 and eventually exhausted the retry limit. This change refreshes the base tablet without capping sync by the request alter version before computing schema change V1, allowing retries to register a fresh boundary. It also adds a docker regression case that blocks schema change, creates a cross-V1 retry, injects stale local max-version state for query-version sync, and verifies the retry finishes after the base tablet is refreshed. ### Release note Fix cloud schema change retry stale V1 failure.

Yukang-Lian force-pushed the fix/cloud-sc-compaction-conflict-retry branch from 8183948 to dad5109Compare April 9, 2026 07:58

Yukang-Lian requested review from dataroaring and gavinchou as code owners April 9, 2026 07:58

Yukang-Lian force-pushed the fix/cloud-sc-compaction-conflict-retry branch from dad5109 to bc9b76eCompare April 9, 2026 08:58

Hastyshell reviewed Apr 10, 2026
View reviewed changes

Comment threadbe/src/cloud/cloud_schema_change_job.cpp Outdated

Yukang-Lian force-pushed the fix/cloud-sc-compaction-conflict-retry branch from 43ac15e to 779a597Compare April 10, 2026 07:08

Hastyshell approved these changes Apr 10, 2026
View reviewed changes

github-actionsBot added the reviewed label Apr 10, 2026

Yukang-Lian force-pushed the fix/cloud-sc-compaction-conflict-retry branch from 779a597 to 91835d1Compare April 14, 2026 08:11

Yukang-Lian requested a review from w41ter as a code owner April 14, 2026 08:11

Yukang-Lian added 2 commits April 15, 2026 11:18

Revert "[improve](compaction) Add block sample logging after vertical…
668f034
… compaction init" This reverts commit 6cfa629.

luwei16 added dev/4.0.x dev/4.1.x labels Apr 20, 2026

gavinchou approved these changes Apr 20, 2026
View reviewed changes

luwei16 merged commit c847318 into apache:masterApr 20, 2026
35 of 36 checks passed

github-actionsBot mentioned this pull request Apr 20, 2026
branch-4.0: [fix](cloud) Add SC_COMPACTION_CONFLICT error code to retry cross-V1 compaction failures #62272 #62641
Closed

github-actionsBot mentioned this pull request Apr 20, 2026
branch-4.1: [fix](cloud) Add SC_COMPACTION_CONFLICT error code to retry cross-V1 compaction failures #62272 #62643
Merged

mymeiyi mentioned this pull request Apr 21, 2026
[fix](wal) Fix WAL replay loading 0 rows due to columns being marked as missing #62658
Merged

yiguolei pushed a commit that referenced this pull request Apr 22, 2026
branch-4.1: [fix](cloud) Add SC_COMPACTION_CONFLICT error code to ret…
136f04e
…ry cross-V1 compaction failures #62272 (#62643) Cherry-picked from #62272 Co-authored-by: Jimmy <lianyukang@selectdb.com>

yiguolei added dev/4.1.1-merged and removed dev/4.1.x labels Apr 22, 2026

Yukang-Lian mentioned this pull request Apr 22, 2026
branch-4.0: pick #61696 #61621 #62272 #62706
Merged

yiguolei added dev/4.0.6-merged and removed dev/4.0.x labels Apr 27, 2026

yiguolei mentioned this pull request May 20, 2026
4.1.1 Release Notes #63426
Closed

morningman mentioned this pull request Jun 3, 2026
4.0.6 Release Notes #64065
Closed

Yukang-Lian mentioned this pull request Jun 9, 2026
[fix](cloud) Refresh base tablet before schema change V1 #64312
Merged

gavinchou removed the dev/3.1.x label Jun 24, 2026

Uh oh!

Conversation

Yukang-Lian commented Apr 9, 2026

Summary

Background

Changes

Test plan

Uh oh!

hello-stephen commented Apr 9, 2026

Uh oh!

Uh oh!

Hastyshell left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Apr 10, 2026

Uh oh!

Yukang-Lian commented Apr 13, 2026

Uh oh!

hello-stephen commented Apr 13, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Apr 13, 2026

BE Regression && UT Coverage Report

Uh oh!

Yukang-Lian commented Apr 15, 2026

Uh oh!

Yukang-Lian commented Apr 15, 2026

Uh oh!

Yukang-Lian commented Apr 15, 2026

Uh oh!

Yukang-Lian commented Apr 15, 2026

Uh oh!

hello-stephen commented Apr 15, 2026

FE Regression Coverage Report

Uh oh!

hello-stephen commented Apr 15, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Apr 15, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Apr 15, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Apr 15, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Apr 15, 2026

FE Regression Coverage Report

Uh oh!

hello-stephen commented Apr 15, 2026

FE Regression Coverage Report

Uh oh!

Yukang-Lian commented Apr 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants