[UI] Allow quiescevm and asyncbackup flags while taking volume snapshot from UI when these are supported for the volume

[sureshanaparti]

Description

This PR allows quiescevm and asyncbackup flags while taking volume snapshot from UI when these are supported for the volume, addresses #10106.

Also, updated snapshot forms to show quiescevm for VMware & KVM hypervisors.

Changes from ui/src/views/storage/TakeSnapshot.vue were removed in #7873, re-added in this PR. Now, ui/src/views/compute/CreateSnapshotWizard.vue and ui/src/views/storage/TakeSnapshot.vue are in sync with respect to quiescevm and asyncbackup flags.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI
• test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Checked the create snapshot UI from instance and volume.

How did you try to break this feature and the system with this change?

[sureshanaparti]

@blueorangutan ui

[blueorangutan]

@sureshanaparti a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

[codecov]

Codecov Report

Attention: Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 15.17%. Comparing base (f055268) to head (f2f2c7b).
Report is 12 commits behind head on 4.19.

Files with missing lines	Patch %	Lines
.../storage/vmsnapshot/StorageVMSnapshotStrategy.java	85.71%	0 Missing and 1 partial ⚠️
...om/cloud/storage/snapshot/SnapshotManagerImpl.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## 4.19 #10265 +/- ##
=========================================
Coverage 15.16% 15.17% - Complexity 11332 11333 +1 
=========================================
Files 5415 5415 Lines 474912 474938 +26 Branches 57926 57930 +4 =========================================
+ Hits 72043 72057 +14 - Misses 394814 394823 +9 - Partials 8055 8058 +3 

Flag	Coverage Δ
uitests	4.28% <ø> (-0.01%)	⬇️
unittests	15.89% <75.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[blueorangutan]

UI build: ✔️
Live QA URL: https://qa.cloudstack.cloud/simulator/pr/10265 (QA-JID-531)

[DaanHoogland]

clgtm

[DaanHoogland]

@Rubueno , any chance you can test this?

[Rubueno]

@Rubueno , any chance you can test this?

Thanks for your efforts @DaanHoogland ! I'm abroad right now, but I'll try to find time next week.

[DaanHoogland]

@Rubueno , any chance you can test this?

Thanks for your efforts @DaanHoogland ! I'm abroad right now, but I'll try to find time next week.

have a good trip @Rubueno , I hope to start on a release by the end of next week, no pressure ;)

[Rubueno]

@Rubueno , any chance you can test this?

Thanks for your efforts @DaanHoogland ! I'm abroad right now, but I'll try to find time next week.

have a good trip @Rubueno , I hope to start on a release by the end of next week, no pressure ;)

Thanks! I've just tested the change but it does not seem the quiesce option is present in the "Take Instance volume Snapshot" window. I've tested with local, RBD and NFS on KVM.

[DaanHoogland]

Thanks! I've just tested the change but it does not seem the quiesce option is present in the "Take Instance volume Snapshot" window. I've tested with local, RBD and NFS on KVM.

You are right, I only see it on the take instance snapshot dialog. Can you comment @sureshanaparti ?

[sureshanaparti]

Thanks! I've just tested the change but it does not seem the quiesce option is present in the "Take Instance volume Snapshot" window. I've tested with local, RBD and NFS on KVM.

You are right, I only see it on the take instance snapshot dialog. Can you comment @sureshanaparti ?

quiesce option will not available in both VM -> "Take Instance volume Snapshot" & Volume -> "Take Snapshot" dialogs, when the volume response returns quiescevm as false.

[Rubueno]

Thanks! I've just tested the change but it does not seem the quiesce option is present in the "Take Instance volume Snapshot" window. I've tested with local, RBD and NFS on KVM.

You are right, I only see it on the take instance snapshot dialog. Can you comment @sureshanaparti ?

quiesce option will not available in both VM -> "Take Instance volume Snapshot" & Volume -> "Take Snapshot" dialogs, when the volume response returns quiescevm as false.

But in this case it is present in Volume -> "Take Snapshot" and VM -> "Take Instance Snapshot" dialogs

[sureshanaparti]

Thanks! I've just tested the change but it does not seem the quiesce option is present in the "Take Instance volume Snapshot" window. I've tested with local, RBD and NFS on KVM.

You are right, I only see it on the take instance snapshot dialog. Can you comment @sureshanaparti ?

quiesce option will not available in both VM -> "Take Instance volume Snapshot" & Volume -> "Take Snapshot" dialogs, when the volume response returns quiescevm as false.

But in this case it is present in Volume -> "Take Snapshot" and VM -> "Take Instance Snapshot" dialogs

Both these dialogs are in sync with the quiescevm param in the volume response, shows it when quiescevm param is true.

Even when it is set, it fails here when unsupported: https://github.com/apache/cloudstack/blob/4.19/engine/storage/snapshot/src/main/java/org/apache/cloudstack/storage/snapshot/DefaultSnapshotStrategy.java#L496-L498

[sureshanaparti]

@Rubueno are you fine with these changes in this PR (both dialogs are in sync with respect to the volume response).

ui/src/views/compute/CreateSnapshotWizard.vue (VM -> "Take Instance volume Snapshot" dialog) =>

 <a-form-item name="quiescevm" ref="quiescevm" v-if="isQuiesceVm">
<template #label>
<tooltip-label :title="$t('label.quiescevm')" :tooltip="apiParams.quiescevm.description"/>
</template>
<a-switch v-model:checked="form.quiescevm"/>
</a-form-item>
<a-form-item name="asyncbackup" ref="asyncbackup" v-if="!supportsStorageSnapshot">
<template #label>
<tooltip-label :title="$t('label.asyncbackup')" :tooltip="apiParams.asyncbackup.description"/>
</template>
<a-switch v-model:checked="form.asyncbackup"/>
</a-form-item>

ui/src/views/storage/TakeSnapshot.vue (Volume -> "Take Snapshot" dialog) =>

 <a-form-item :label="$t('label.asyncbackup')" name="asyncbackup" ref="asyncbackup" v-if="!supportsStorageSnapshot">
<a-switch v-model:checked="form.asyncbackup" />
</a-form-item>
<a-form-item :label="$t('label.quiescevm')" name="quiescevm" ref="quiescevm" v-if="quiescevm">
<a-switch v-model:checked="form.quiescevm" />
</a-form-item>

[Rubueno]

@sureshanaparti In our env I cannot proceed with testing this, and thus does not resolve issue #10106, as both dialog boxes give different output at the moment

[sureshanaparti]

@sureshanaparti In our env I cannot proceed with testing this, and thus does not resolve issue #10106, as both dialog boxes give different output at the moment

ok @Rubueno, does your env has the updated code / pkgs from this PR?

[sureshanaparti]

@blueorangutan package

[blueorangutan]

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[Rubueno]

Yes I had built new packages based on 4.19

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12308

[Rubueno]

Hmm an interesting development: When taking a snapshot from VM -> "Take Instance Snapshot" with "Quiesce Instance" enabled, it will make a snapshot and quiesce process works. When taking a snapshot from Volume -> "Take Snapshot" with "Quiesce Instance" enabled, it will not quiesce the instance and even logs "quiesceVm":false

VM -> "Take Instance Snapshot" logfile: https://pastebin.com/hYMHDxKF
Volume -> "Take Snapshot" logfile: https://pastebin.com/EF33nP6z

[DaanHoogland]

@sureshanaparti can you comment on @Rubueno 's results?

[sureshanaparti]

Hmm an interesting development: When taking a snapshot from VM -> "Take Instance Snapshot" with "Quiesce Instance" enabled, it will make a snapshot and quiesce process works. When taking a snapshot from Volume -> "Take Snapshot" with "Quiesce Instance" enabled, it will not quiesce the instance and even logs "quiesceVm":false

VM -> "Take Instance Snapshot" logfile: https://pastebin.com/hYMHDxKF Volume -> "Take Snapshot" logfile: https://pastebin.com/EF33nP6z

@Rubueno Fixed to set quiescevm in the snapshot payload (from the quiescevm parameter) while taking a snapshot from volume, please check.

[sureshanaparti]

I've done some review/testing and discussed with Suresh, following are the findings.

There are 3 forms where we are using "quiescevm"

1. For a stopped VM - "Take Instance volume Snapshot"
2. For a volume - "Take Snapshot"
3. For a running VM - "Take Instance Snapshot"

For 1 and 2, as @sureshanaparti said it is controlled by "quiescevm" param in the volume response. I see that it has made it consistent with this PR. Though there is another bug like @Rubueno mentioned, the value of "quiescevm" is not honored in the API.

For 3, "Take Instance Snapshot" "quiescevm" is there by default. May be we can check for which hypervisors it is supported and keep only for those.

@harikrishna-patnala thanks for testing, I've verified "quiescevm" for all the 3 forms, it is honored as set in the UI. Changes done for "quiescevm" in "Take Instance Snapshot" form to show for VMware, KVM hypervisors only - it is used for VMware, the support of it depends on the underlying storage for KVM, ignored/unused for XenServer.

[sureshanaparti]

@blueorangutan package

[blueorangutan]

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13122

[Rubueno]

I've done some review/testing and discussed with Suresh, following are the findings.
There are 3 forms where we are using "quiescevm"

1. For a stopped VM - "Take Instance volume Snapshot"
2. For a volume - "Take Snapshot"
3. For a running VM - "Take Instance Snapshot"

For 1 and 2, as @sureshanaparti said it is controlled by "quiescevm" param in the volume response. I see that it has made it consistent with this PR. Though there is another bug like @Rubueno mentioned, the value of "quiescevm" is not honored in the API.
For 3, "Take Instance Snapshot" "quiescevm" is there by default. May be we can check for which hypervisors it is supported and keep only for those.

@harikrishna-patnala thanks for testing, I've verified "quiescevm" for all the 3 forms, it is honored as set in the UI. Changes done for "quiescevm" in "Take Instance Snapshot" form to show for VMware, KVM hypervisors only - it is used for VMware, the support of it depends on the underlying storage for KVM, ignored/unused for XenServer.

Thank you both for your efforts. I'm traveling right now and unable to test myself.

[sureshanaparti]

@blueorangutan package

[blueorangutan]

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13131

[sureshanaparti]

Noticed below issues in the Simulator CI tests, seems to be python (version) issue. cc @DaanHoogland

2025-04-22 12:23:28,676 - DEBUG - ::::::::::::STARTED : TC: Failure: :::::::::::
2025-04-22 12:23:28,677 - CRITICAL - EXCEPTION: Failure:: ['Traceback (most recent call last):\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 59, in testPartExecutor\n yield\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 591, in run\n self._callTestMethod(testMethod)\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 549, in _callTestMethod\n method()\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/failure.py", line 35, in runTest\n raise self.exc_val.with_traceback(self.tb)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/loader.py", line 335, in loadTestsFromName\n module = self.importer.importFromPath(\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 162, in importFromPath\n return self.importFromDir(dir_path, fqname)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 198, in importFromDir\n mod = load_module(part_fqname, fh, filename, desc)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 128, in load_module\n spec.loader.exec_module(mod)\n', ' File "<frozen importlib._bootstrap_external>", line 883, in exec_module\n', ' File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed\n', ' File "/home/runner/work/cloudstack/cloudstack/test/integration/smoke/test_certauthority_root.py", line 27, in <module>\n from OpenSSL.crypto import FILETYPE_PEM, verify, X509\n', "ImportError: cannot import name 'verify' from 'OpenSSL.crypto' (unknown location)\n"]
2025-04-22 12:23:28,678 - DEBUG - TestCaseName: Failure:; Time Taken: 0 Seconds; StartTime: Tue Apr 22 12:23:28 2025; EndTime: Tue Apr 22 12:23:28 2025; Result: EXCEPTION

Exception occurred during processing of request from ('127.0.0.1', 55687)
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/socketserver.py", line 683, in process_request_thread
self.finish_request(request, client_address)
File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/socketserver.py", line 747, in __init__
self.handle()
File "/home/runner/.local/lib/python3.10/site-packages/ipmisim/ipmisim.py", line 443, in handle
return IpmiServerContext().handle(data, address, socket)
File "/home/runner/.local/lib/python3.10/site-packages/ipmisim/ipmisim.py", line 125, in handle
self._got_request(data, address, self.session)
File "/home/runner/.local/lib/python3.10/site-packages/ipmisim/ipmisim.py", line 222, in _got_request
session._ipmi20(data)
File "/home/runner/.local/lib/python3.10/site-packages/ipmisim/fakesession.py", line 127, in _ipmi20
decrypter = AES.new(self.aeskey, AES.MODE_CBC, iv)
File "/home/runner/.local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
return AESCipher(key, *args, **kwargs)
File "/home/runner/.local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
File "/home/runner/.local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

[sureshanaparti]

@blueorangutan test

[blueorangutan]

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[weizhouapache]

Noticed below issues in the Simulator CI tests, seems to be python (version) issue. cc @DaanHoogland

2025-04-22 12:23:28,676 - DEBUG - ::::::::::::STARTED : TC: Failure: :::::::::::
2025-04-22 12:23:28,677 - CRITICAL - EXCEPTION: Failure:: ['Traceback (most recent call last):\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 59, in testPartExecutor\n yield\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 591, in run\n self._callTestMethod(testMethod)\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 549, in _callTestMethod\n method()\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/failure.py", line 35, in runTest\n raise self.exc_val.with_traceback(self.tb)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/loader.py", line 335, in loadTestsFromName\n module = self.importer.importFromPath(\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 162, in importFromPath\n return self.importFromDir(dir_path, fqname)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 198, in importFromDir\n mod = load_module(part_fqname, fh, filename, desc)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 128, in load_module\n spec.loader.exec_module(mod)\n', ' File "<frozen importlib._bootstrap_external>", line 883, in exec_module\n', ' File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed\n', ' File "/home/runner/work/cloudstack/cloudstack/test/integration/smoke/test_certauthority_root.py", line 27, in <module>\n from OpenSSL.crypto import FILETYPE_PEM, verify, X509\n', "ImportError: cannot import name 'verify' from 'OpenSSL.crypto' (unknown location)\n"]
2025-04-22 12:23:28,678 - DEBUG - TestCaseName: Failure:; Time Taken: 0 Seconds; StartTime: Tue Apr 22 12:23:28 2025; EndTime: Tue Apr 22 12:23:28 2025; Result: EXCEPTION

Exception occurred during processing of request from ('127.0.0.1', 55687)
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/socketserver.py", line 683, in process_request_thread
self.finish_request(request, client_address)
File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/socketserver.py", line 747, in __init__
self.handle()
File "/home/runner/.local/lib/python3.10/site-packages/ipmisim/ipmisim.py", line 443, in handle
return IpmiServerContext().handle(data, address, socket)
File "/home/runner/.local/lib/python3.10/site-packages/ipmisim/ipmisim.py", line 125, in handle
self._got_request(data, address, self.session)
File "/home/runner/.local/lib/python3.10/site-packages/ipmisim/ipmisim.py", line 222, in _got_request
session._ipmi20(data)
File "/home/runner/.local/lib/python3.10/site-packages/ipmisim/fakesession.py", line 127, in _ipmi20
decrypter = AES.new(self.aeskey, AES.MODE_CBC, iv)
File "/home/runner/.local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
return AESCipher(key, *args, **kwargs)
File "/home/runner/.local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
File "/home/runner/.local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

@sureshanaparti
refer to #10762

[blueorangutan]

[SF] Trillian test result (tid-13086)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 56548 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10265-t13086-kvm-ol8.zip
Smoke tests completed. 133 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[DaanHoogland]

@sureshanaparti , is it realistic to have this finished/fixed before 4.19.3?

[sureshanaparti]

@sureshanaparti , is it realistic to have this finished/fixed before 4.19.3?

@DaanHoogland no further changes required on this PR.

[RosiKyu]

LGTM. The 'quiescevm' option is now displayed consistently in modals. If the volume response has 'quiescevm' enabled, only then you can see that option on both "Take Instance volume Snapshot" ( "Compute" -> "Instances") and "Take Snapshot" ("Volumes") UI.