Block use of internal and external snapshots on KVM by JoaoJandre · Pull Request #11039 · apache/cloudstack

JoaoJandre · 2025-06-16T18:37:01Z

Description

On KVM, there are two types of snapshots: internal and external. Most snapshot/backup solutions use external snapshots on ACS; save for disk-and-memory VM snapshots, which use internal snapshots (this is a limitation with KVM, as far as I know).

However, since internal snapshots are stored inside the VM's volume (hence the name), if an internal snapshot is taken after an external snapshot and the external snapshot is restored, the internal snapshot is lost.

Thus, this PR blocks the use of disk-and-memory VM snapshots alongside volume snapshots, NAS backups, and disk-only VM snapshots (at least the ones created using the default volume snapshot implementation).

I encourage maintainers of 3rd party storage providers to test if their implementation is compatible with disk-and-memory VM snapshots, if it is not it their simultaneous usage should be blocked.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

I created a VM and created a few disk-and-memory VM snapshots on it; then I tried to create NAS backups, volume snapshots and disk-only VM snapshots, all of them gave me an error, which is expected.

I validated that the opposite was also true for the aforementioned cases, e.g., create volume snapshot and try to create disk-and-memory VM snapshot.

I also validated that it was possible to create multiple NAS backups, disk-only VM snapshots and volume snapshots with no issues.

JoaoJandre · 2025-06-16T18:39:18Z

@slavkap @rp- I think it would be interesting to validate if the implementations done for Storpool and Linstor are compatible with disk-and-memory VM snapshots.

codecov · 2025-06-16T18:44:53Z

Codecov Report

❌ Patch coverage is 0% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.56%. Comparing base (6dc259c) to head (3095fb1).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../storage/vmsnapshot/DefaultVMSnapshotStrategy.java	0.00%	15 Missing ⚠️
...rg/apache/cloudstack/backup/NASBackupProvider.java	0.00%	5 Missing and 1 partial ⚠️
...a/org/apache/cloudstack/backup/BackupProvider.java	0.00%	3 Missing ⚠️
...tack/storage/snapshot/DefaultSnapshotStrategy.java	0.00%	3 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #11039 +/- ##
=========================================
Coverage 17.55% 17.56% - Complexity 15535 15537 +2 
=========================================
Files 5911 5912 +1 Lines 529359 529383 +24 Branches 64655 64660 +5 =========================================
+ Hits 92949 92980 +31 + Misses 425952 425942 -10 - Partials 10458 10461 +3

Flag	Coverage Δ
uitests	`3.58% <ø> (ø)`
unittests	`18.63% <0.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

JoaoJandre · 2025-06-16T18:48:48Z

@blueorangutan package

blueorangutan · 2025-06-16T18:50:04Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-16T19:10:16Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13798

blueorangutan · 2025-06-17T07:45:56Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13809

DaanHoogland · 2025-06-17T08:43:04Z

@JoaoJandre :

08:35:48 [ERROR] /jenkins/workspace/acs-centos8-pkg-builder/dist/rpmbuild/BUILD/cloudstack-4.20.2.0-SNAPSHOT/engine/storage/snapshot/src/test/java/org/apache/cloudstack/storage/vmsnapshot/VMSnapshotStrategyKVMTest.java:32:8: Unused import - org.apache.cloudstack.backup.dao.BackupDao. [UnusedImports]

JoaoJandre · 2025-06-17T16:38:53Z

@blueorangutan package

blueorangutan · 2025-06-17T16:40:04Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-17T17:02:31Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13818

DaanHoogland · 2025-06-18T07:04:45Z

@blueorangutan package

blueorangutan · 2025-06-18T07:08:04Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-18T08:16:00Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13826

rp- · 2025-07-03T08:09:10Z

Linstor does currently not support memory snapshots (we check and throw an error if selected).
So I guess we are currently not affected by any of this?

slavkap

code LGTM
I haven't tested it with NFS, but the StorPool smoke tests are executed successfully

DaanHoogland · 2025-07-08T15:00:32Z

@blueorangutan test

blueorangutan · 2025-07-08T15:02:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-07-09T06:43:28Z

[SF] Trillian test result (tid-13723)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 53428 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11039-t13723-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

weizhouapache · 2025-08-28T13:23:15Z

@JoaoJandre
this seems to be included in PR #10632
do you still want it in 4.20.2 ?

JoaoJandre · 2025-08-28T13:31:46Z

@JoaoJandre this seems to be included in PR #10632 do you still want it in 4.20.2 ?

@weizhouapache PR #10632 blocks the usage of the feature introduced in #10632 and other incompatible features. This PR purposefully ignores #10632 and adds restrictions to avoid other interactions between internal and external snapshots; such as volume snapshot and disk-and-memory VM snapshot.

They are complementary. When merging this PR forward, care should be taken so that the validations of both PRs do not erase one another (I can make the merge forward if needed).

weizhouapache · 2025-08-28T13:36:51Z

@JoaoJandre this seems to be included in PR #10632 do you still want it in 4.20.2 ?
@weizhouapache PR #10632 blocks the usage of the feature introduced in #10632 and other incompatible features. This PR purposefully ignores #10632 and adds restrictions to avoid other interactions between internal and external snapshots; such as volume snapshot and disk-and-memory VM snapshot.
They are complementary. When merging this PR forward, care should be taken so that the validations of both PRs do not erase one another (I can make the merge forward if needed).

ok @JoaoJandre
I think the best option might be re-target this PR to 4.22 which includes #10632 , to avoid re-work.

blueorangutan · 2025-09-16T08:38:19Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15030

github-actions · 2025-09-25T08:09:28Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

JoaoJandre · 2025-11-17T12:50:16Z

@blueorangutan package

blueorangutan · 2025-11-17T12:52:03Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-11-17T14:35:19Z

Packaging result [SF]: ✔️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 15784

JoaoJandre · 2025-11-18T11:31:07Z

@blueorangutan package

blueorangutan · 2025-11-18T11:32:03Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-11-18T12:30:11Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✖️ debian ✔️ suse15. SL-JID 15789

JoaoJandre · 2025-11-19T11:58:29Z

@blueorangutan package

blueorangutan · 2025-11-19T12:00:04Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-11-19T13:16:08Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15793

weizhouapache · 2025-11-19T13:44:03Z

@blueorangutan test

blueorangutan · 2025-11-19T13:46:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-11-20T03:47:46Z

[SF] Trillian test result (tid-14847)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 47985 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11039-t14847-kvm-ol8.zip
Smoke tests completed. 150 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland · 2025-11-21T11:02:23Z

@JoaoJandre , are we good to go with this? cc @weizhouapache

JoaoJandre · 2025-11-21T13:58:44Z

I believe we have all the prerequisites to merge, @DaanHoogland. Shall we?

boring-cyborgBot added component:backup component:storage labels Jun 16, 2025

JoaoJandre added the Severity:Major label Jun 16, 2025

JoaoJandre added this to the 4.20.2 milestone Jun 16, 2025

DaanHoogland requested review from harikrishna-patnala, rp- and slavkap June 17, 2025 06:17

slavkap reviewed Jul 8, 2025
View reviewed changes

Comment thread...apshot/src/main/java/org/apache/cloudstack/storage/vmsnapshot/DefaultVMSnapshotStrategy.java Outdated

slavkap reviewed Jul 8, 2025
View reviewed changes

Comment thread...apshot/src/main/java/org/apache/cloudstack/storage/vmsnapshot/DefaultVMSnapshotStrategy.java Outdated

slavkap approved these changes Jul 8, 2025
View reviewed changes

github-actionsBot added the status:has-conflicts label Sep 25, 2025

harikrishna-patnala modified the milestones: 4.22.0, 4.22.1 Nov 12, 2025

Merge remote-tracking branch 'origin/main' into avoid-data-loss
3095fb1

github-actionsBot removed the status:has-conflicts label Nov 17, 2025

DaanHoogland changed the base branch from main to 4.22November 24, 2025 10:37

boring-cyborgBot added the distro:ubuntu label Nov 24, 2025

DaanHoogland changed the base branch from 4.22 to mainNovember 24, 2025 10:38

DaanHoogland modified the milestones: 4.22.1, 4.23 Nov 24, 2025

DaanHoogland merged commit 8171d95 into apache:mainNov 24, 2025
28 checks passed

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Dec 11, 2025
Block use of internal and external snapshots on KVM (apache#11039)
70e81a0

weizhouapache mentioned this pull request Dec 18, 2025
Failed to revert a memory snapshot #11428
Closed

owsferraro pushed a commit to owsferraro/cloudstack that referenced this pull request Mar 25, 2026
Block use of internal and external snapshots on KVM (apache#11039)
f9a2bd9

Uh oh!

Conversation

JoaoJandre commented Jun 16, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

JoaoJandre commented Jun 16, 2025

Uh oh!

codecovBot commented Jun 16, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

JoaoJandre commented Jun 16, 2025

Uh oh!

blueorangutan commented Jun 16, 2025

Uh oh!

blueorangutan commented Jun 16, 2025

Uh oh!

blueorangutan commented Jun 17, 2025

Uh oh!

DaanHoogland commented Jun 17, 2025

Uh oh!

JoaoJandre commented Jun 17, 2025

Uh oh!

blueorangutan commented Jun 17, 2025

Uh oh!

blueorangutan commented Jun 17, 2025

Uh oh!

DaanHoogland commented Jun 18, 2025

Uh oh!

blueorangutan commented Jun 18, 2025

Uh oh!

blueorangutan commented Jun 18, 2025

Uh oh!

rp- commented Jul 3, 2025

Uh oh!

Uh oh!

Uh oh!

slavkap left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Jul 8, 2025

Uh oh!

blueorangutan commented Jul 8, 2025

Uh oh!

blueorangutan commented Jul 9, 2025

Uh oh!

weizhouapache commented Aug 28, 2025

Uh oh!

JoaoJandre commented Aug 28, 2025

Uh oh!

weizhouapache commented Aug 28, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blueorangutan commented Sep 16, 2025

Uh oh!

github-actionsBot commented Sep 25, 2025

Uh oh!

JoaoJandre commented Nov 17, 2025

Uh oh!

blueorangutan commented Nov 17, 2025

Uh oh!

blueorangutan commented Nov 17, 2025

Uh oh!

JoaoJandre commented Nov 18, 2025

Uh oh!

blueorangutan commented Nov 18, 2025

Uh oh!

blueorangutan commented Nov 18, 2025

Uh oh!

JoaoJandre commented Nov 19, 2025

Uh oh!

blueorangutan commented Nov 19, 2025

Uh oh!

blueorangutan commented Nov 19, 2025

codecovBot commented Jun 16, 2025•
edited
Loading

weizhouapache commented Aug 28, 2025•
edited
Loading