Skip to content

<feature>[zbs]: ZSV-12666 support whole-VM backup - #4587

Closed
ZStack-Robot wants to merge 1 commit into
feature-zsv-5.1.0-encryptionfrom
sync/zstackio/zbs-volume-encryption-backup@@4
Closed

<feature>[zbs]: ZSV-12666 support whole-VM backup#4587
ZStack-Robot wants to merge 1 commit into
feature-zsv-5.1.0-encryptionfrom
sync/zstackio/zbs-volume-encryption-backup@@4

Conversation

@ZStack-Robot

Copy link
Copy Markdown
Collaborator

Summary

Support ZBS whole-VM backup restore on the existing VolumeBackup/ImageStore flow. Plain backup disks can be restored as encrypted ZBS disks through temporary cache conversion; the backup encryption property is not changed.

Changes

  • Add ordered external-primary-storage image download hooks and restore context.
  • Carry ZBS logical/backing size information through encrypted restore.
  • Keep the existing native clone and resize flow for encrypted cache snapshots.

Testing

  • cbok Groovy case: 1 test, 0 failures/errors/skips
  • Full/auto whole-VM backup, revert, three-disk restore matrix, and rollback injection coverage
  • cbok compile/deploy to 172.24.247.15
  • MN/UI status Running
  • CI pipeline

Resolves: ZSV-12666

sync from gitlab !10557

@coderabbitai

coderabbitaiBot commented Jul 16, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: a5ef886d-ff38-43a3-a91e-c3c62900f92b

📥 Commits

Reviewing files that changed from the base of the PR and between 9904223 and f27ae34.

📒 Files selected for processing (13)
  • compute/src/main/java/org/zstack/compute/vm/VmInstantiateOtherDiskFlow.java
  • header/src/main/java/org/zstack/header/core/encrypt/CryptoServiceConstant.java
  • header/src/main/java/org/zstack/header/storage/addon/primary/DownloadExternalPrimaryStorageEncryptedImageMsg.java
  • header/src/main/java/org/zstack/header/storage/addon/primary/PrepareExternalPrimaryStorageEncryptedImageDownloadMsg.java
  • header/src/main/java/org/zstack/header/storage/primary/DownloadDataVolumeToPrimaryStorageMsg.java
  • header/src/main/java/org/zstack/header/storage/primary/InstantiateRootVolumeFromTemplateOnPrimaryStorageMsg.java
  • header/src/main/java/org/zstack/header/volume/CreateDataVolumeFromVolumeTemplateMsg.java
  • header/src/main/java/org/zstack/header/volume/InstantiateRootVolumeMsg.java
  • storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java
  • storage/src/main/java/org/zstack/storage/volume/InstantiateVolumeForNewCreatedVmExtension.java
  • storage/src/main/java/org/zstack/storage/volume/InstantiateVolumeForVmChangeImageExtension.java
  • storage/src/main/java/org/zstack/storage/volume/VolumeBase.java
  • storage/src/main/java/org/zstack/storage/volume/VolumeManagerImpl.java

Walkthrough

新增加密卷备份恢复标记及其消息契约,并将该标记从虚拟机卷实例化流程传递至外部主存储下载流程。加密场景增加准备、加密卷创建和专用下载分支,普通场景保留原有下载路径。

Changes

外部主存储加密镜像下载

Layer / File(s)Summary
加密下载消息契约
header/src/main/java/org/zstack/header/core/encrypt/CryptoServiceConstant.java, header/src/main/java/org/zstack/header/storage/addon/primary/*EncryptedImageDownloadMsg.java, header/src/main/java/org/zstack/header/storage/primary/DownloadDataVolumeToPrimaryStorageMsg.java, header/src/main/java/org/zstack/header/volume/*Msg.java
新增加密服务标识、加密准备与下载消息,并为卷实例化和数据卷下载消息增加 encryptedVolumeBackupRestore 访问器。
加密意图传递
compute/src/main/java/org/zstack/compute/vm/VmInstantiateOtherDiskFlow.java, storage/src/main/java/org/zstack/storage/volume/InstantiateVolumeFor*Extension.java, storage/src/main/java/org/zstack/storage/volume/VolumeBase.java, storage/src/main/java/org/zstack/storage/volume/VolumeManagerImpl.java, storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java
根据卷模板和加密状态设置标记,并将其传递至根卷、数据卷及镜像缓存下载调用。
加密下载执行流程
storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java
加密场景发送准备消息、配置加密卷参数并执行专用下载消息;非加密场景继续使用普通远程上传下载消息。

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
participant ExternalPrimaryStorage
participant CryptoService
participant PrimaryStorage
ExternalPrimaryStorage->>CryptoService: 发送加密下载准备消息
CryptoService-->>ExternalPrimaryStorage: 返回准备结果
ExternalPrimaryStorage->>PrimaryStorage: 创建加密卷
ExternalPrimaryStorage->>PrimaryStorage: 执行加密镜像下载
Loading

Possibly related PRs

Poem

小兔挥耳,标记已传,
加密卷儿稳稳建。
先问服务,再来下载,
普通路径照旧跑。
月光下,镜像安全到家。

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ Passed标题与变更核心一致,概括了支持 whole-VM backup 的主要改动。
Description check✅ Passed描述与实际变更相关,说明了在现有 VolumeBackup/ImageStore 流程中支持整机备份恢复。
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch sync/zstackio/zbs-volume-encryption-backup@@4

Comment @coderabbitai help to get the list of available commands.

@zstack-robot-2

Copy link
Copy Markdown
Collaborator

Comment from yaohua.wu:

Review: MR !10557 — ZSV-12666

Background (preserved across rounds)

  • Jira: ZSV-12666 — 【虚拟机加密】zbs-基于加密备份创建vm失败
  • Bug summary: ZBS 上从整机备份创建 VM 时,加密卷恢复链路无法正确处理备份密钥与临时缓存转换;明文备份还需要支持恢复为加密目标卷,且不能改变备份原始 bits/加密元数据。
  • Intent & scope: 在 ExternalPrimaryStorage 下载流程中加入可排序的扩展上下文与加密下载分支,由 premium 承接 ZBS/ImageStore 的密钥准备及下载实现。
  • Round 1 initial findings: 0 × P0, 0 × P1(本仓);联合链路存在 1 × P0,位于 zstack-store FUSE 并发导出。
  • Suggested fix direction: 本仓接口与 premium 调用契约保持不变;合入整组 MR 前先修复 zstack-store 的 FUSE 全局状态并发安全。

Findings

SeverityFile:LineIssueFix
本仓 3 个文件未发现可发布的独立 finding。扩展上下文、字段类型及 premium 实现的调用契约一致。

关联 MR

MR关系
premium !14769实现 ZBS 加密备份密钥准备、ImageStore FUSE 导出编排及多盘恢复测试。
zstack-utility !7526KVM agent 通过 sshfs 读取导出链并转换到 LUKS CBD 目标。
zstack-store !1158新增 FUSE path 导出端点;当前存在并发导出可终止 agent 的联合阻塞问题。

Coverage

  • Local authoritative diff: 3/3 files;MCP 文件数一致;generated-file filter: 0 skipped。
  • Upstream freshness: target 比 source 新 2 个提交,无同文件重叠;MR 状态为 can_be_merged。其中包含 volume encryption 相关改动,需在整组 MR 合并结果上跑回归。
  • Cross-repo contract: ExternalPrimaryStorageImageDownloadContext 的 volume/remoteTarget/spec 字段与 premium 消费一致。
  • Reviewer dispatch degraded: 并行子审查通道不可用,已由主审逐文件读取并完成跨仓调用链核验。
  • Verification gap: 本机无 mvn,未独立执行 Java/Groovy 测试。

Verdict: REVISION_REQUIRED

本仓实现未发现独立阻塞项;联合交付被 zstack-store !1158 的 FUSE 并发安全问题阻塞。


🤖 Robot Reviewer

@MatheMatrix
MatheMatrixforce-pushed the sync/zstackio/zbs-volume-encryption-backup@@4 branch from 54d7d7b to 06d28e4CompareJuly 16, 2026 11:02

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java (1)

1305-1345: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

镜像缓存命中条件要区分加密状态 现在只按 primaryStorageUuid + imageUuid 复用缓存,而 downloadImageTo(..., targetEncrypted, ...) 会生成明文/加密两种不同结果;ImageCacheVO 也没有加密状态字段,后续加密请求可能直接命中明文缓存。建议把加密状态纳入缓存标识,或命中时重建缓存。

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java`
around lines 1305 - 1345, Update doDownloadImageCache so cache reuse
distinguishes targetEncrypted state instead of matching only primaryStorageUuid
and imageUuid. Persist and query an encryption-state identifier for
ImageCacheVO, or invalidate and rebuild when the existing cache’s state differs,
ensuring encrypted requests never reuse plaintext caches and vice versa.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In
`@storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java`:
- Around line 1305-1345: Update doDownloadImageCache so cache reuse
distinguishes targetEncrypted state instead of matching only primaryStorageUuid
and imageUuid. Persist and query an encryption-state identifier for
ImageCacheVO, or invalidate and rebuild when the existing cache’s state differs,
ensuring encrypted requests never reuse plaintext caches and vice versa.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 5187dd2b-955c-4f15-9202-75d2d4a90323

📥 Commits

Reviewing files that changed from the base of the PR and between 81727e9 and 9f5d4cd.

📒 Files selected for processing (7)
  • header/src/main/java/org/zstack/header/core/encrypt/CryptoServiceConstant.java
  • header/src/main/java/org/zstack/header/storage/addon/primary/DownloadExternalPrimaryStorageEncryptedImageMsg.java
  • header/src/main/java/org/zstack/header/storage/addon/primary/PrepareExternalPrimaryStorageEncryptedImageDownloadMsg.java
  • header/src/main/java/org/zstack/header/storage/addon/primary/PrepareExternalPrimaryStorageEncryptedImageDownloadReply.java
  • header/src/main/java/org/zstack/header/storage/primary/DownloadDataVolumeToPrimaryStorageMsg.java
  • storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java
  • storage/src/main/java/org/zstack/storage/volume/VolumeManagerImpl.java

@MatheMatrix
MatheMatrixforce-pushed the sync/zstackio/zbs-volume-encryption-backup@@4 branch from c3e96f7 to 9904223CompareJuly 21, 2026 04:38

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java (1)

1270-1346: 🗄️ Data Integrity & Integration | 🔴 Critical | 🏗️ Heavy lift

ImageCacheVO 缓存命中逻辑未区分加密恢复意图,可能导致明文/加密缓存被跨场景误用。

doDownloadImageCache 的缓存复用判断(1312-1320行)仅按 (primaryStorageUuid, imageUuid) 查找已存在的 ImageCacheVO,命中后直接 completion.success(...) 返回,完全没有考虑本次调用的 encryptedVolumeBackupRestore 参数。

downloadImageToencryptedVolumeBackupRestore=true 时会显式设置 spec.setEncrypted(true) 并绑定加密密钥(1399-1404行),走全新的 prepare-encrypted-image-download / download-image-for-encryption 分支产生加密格式的缓存;encryptedVolumeBackupRestore=false 时则走普通下载产生明文缓存。这两种缓存对应的磁盘格式是不同的。

由于缓存查找 key 不包含加密意图,同一个 imageUuid 在同一主存储上:

  • 若先以普通方式恢复过(生成明文缓存),之后有 VM 以"加密卷备份恢复"方式恢复同一镜像时,会直接复用明文缓存并跳过加密准备/下载分支,最终 createRootVolume 会用明文缓存 clone 到 spec.setEncrypted(true) 的目标卷——卷被数据库标记为加密,但实际数据仍是明文(正是本文件其他位置代码注释特意提醒要规避的场景)。
  • 反过来,若先加密恢复过,之后普通恢复同一镜像会误用加密缓存 clone 到未加密的目标卷,可能导致克隆失败或产生不可读的卷。

这与 PR 描述的核心场景(同一份 plain 备份按需恢复为加密 ZBS 盘)直接相关,并非边缘 case。建议让缓存查找/持久化 key 也携带加密语义(例如为加密恢复单独打标或使用独立的 install path/记录),避免跨加密语境复用缓存。

🔒 可能的修复方向
 private void doDownloadImageCache(ImageInventory image, boolean encryptedVolumeBackupRestore,
ReturnValueCompletion<ImageCacheInventory> completion) {
CreateVolumeSpec spec = new CreateVolumeSpec();
spec.setUuid(image.getUuid());
spec.setName(buildImageName(image.getUuid()));
ImageCacheVO cache = Q.New(ImageCacheVO.class)
.eq(ImageCacheVO_.primaryStorageUuid, self.getUuid())
.eq(ImageCacheVO_.imageUuid, image.getUuid())
+ .eq(ImageCacheVO_.encrypted, encryptedVolumeBackupRestore) // 需要为ImageCacheVO增加加密维度字段
.find();
if (cache != null) {
completion.success(ImageCacheInventory.valueOf(cache));
return;
}
...
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java`
around lines 1270 - 1346, Update doDownloadImageCache so cache lookup and
persistence distinguish encryptedVolumeBackupRestore from ordinary restores,
preventing an ImageCacheVO created for one mode from being reused in the other.
Add or reuse an ImageCacheVO field/key representing the encryption intent,
include it in the existing primary-storage/image lookup, and set it when
creating the cache while preserving the encrypted download behavior in
downloadImageTo.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@storage/src/main/java/org/zstack/storage/volume/VolumeManagerImpl.java`:
- Line 456: 更新 VolumeManagerImpl 中 requiresEncryptInPlace() 的判断逻辑,将
msg.isEncryptedVolumeBackupRestore() 纳入跳过原地加密的条件。保留现有
download-image-for-encryption 分支及普通加密卷的处理行为,确保 encryptedVolumeBackupRestore 不再触发
VolumeInPlaceEncryptor。
---
Outside diff comments:
In
`@storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java`:
- Around line 1270-1346: Update doDownloadImageCache so cache lookup and
persistence distinguish encryptedVolumeBackupRestore from ordinary restores,
preventing an ImageCacheVO created for one mode from being reused in the other.
Add or reuse an ImageCacheVO field/key representing the encryption intent,
include it in the existing primary-storage/image lookup, and set it when
creating the cache while preserving the encrypted download behavior in
downloadImageTo.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e909f6b9-8623-4166-b897-19bbcbfaef48

📥 Commits

Reviewing files that changed from the base of the PR and between 9f5d4cd and 9904223.

📒 Files selected for processing (13)
  • compute/src/main/java/org/zstack/compute/vm/VmInstantiateOtherDiskFlow.java
  • header/src/main/java/org/zstack/header/core/encrypt/CryptoServiceConstant.java
  • header/src/main/java/org/zstack/header/storage/addon/primary/DownloadExternalPrimaryStorageEncryptedImageMsg.java
  • header/src/main/java/org/zstack/header/storage/addon/primary/PrepareExternalPrimaryStorageEncryptedImageDownloadMsg.java
  • header/src/main/java/org/zstack/header/storage/primary/DownloadDataVolumeToPrimaryStorageMsg.java
  • header/src/main/java/org/zstack/header/storage/primary/InstantiateRootVolumeFromTemplateOnPrimaryStorageMsg.java
  • header/src/main/java/org/zstack/header/volume/CreateDataVolumeFromVolumeTemplateMsg.java
  • header/src/main/java/org/zstack/header/volume/InstantiateRootVolumeMsg.java
  • storage/src/main/java/org/zstack/storage/addon/primary/ExternalPrimaryStorage.java
  • storage/src/main/java/org/zstack/storage/volume/InstantiateVolumeForNewCreatedVmExtension.java
  • storage/src/main/java/org/zstack/storage/volume/InstantiateVolumeForVmChangeImageExtension.java
  • storage/src/main/java/org/zstack/storage/volume/VolumeBase.java
  • storage/src/main/java/org/zstack/storage/volume/VolumeManagerImpl.java
🚧 Files skipped from review as they are similar to previous changes (2)
  • header/src/main/java/org/zstack/header/core/encrypt/CryptoServiceConstant.java
  • header/src/main/java/org/zstack/header/storage/addon/primary/DownloadExternalPrimaryStorageEncryptedImageMsg.java

Route encrypted volume/VM backup restore through the keyed download path
in ExternalPrimaryStorage instead of the keyless ImageStore convert.
- Add backup download hooks and route encrypted backup download through
the crypto service.
- Create the encrypted image volume with the sealed DEK reference.
- Compute encryptedVolumeBackupRestore in
InstantiateVolumeForVmChangeImageExtension by mirroring the parent
InstantiateVolumeForNewCreatedVmExtension pattern
(rootDisk.isVolumeBackupTemplate() && encrypted), fixing revert which
otherwise failed with "encrypt.key-secret is required for cipher".
Resolves: ZSV-12666
Change-Id: I131c5ee9b700560baa130d298aaf523d1702808b
@MatheMatrix
MatheMatrixforce-pushed the sync/zstackio/zbs-volume-encryption-backup@@4 branch from 9904223 to f27ae34CompareJuly 21, 2026 05:06
@zstack-robot-2
zstack-robot-2 deleted the sync/zstackio/zbs-volume-encryption-backup@@4 branch July 22, 2026 05:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@ZStack-Robot@zstack-robot-2