vmware: improve solidfire storage plugin integration and fix cases

[yadvr]

This fixes the following cases in which Solidfire storage integration caused issues when using Solidfire datadisks with VMware:

1. Take Volume Snapshot of Solidfire data disk
2. Delete an active Instance with Solidfire data disk attached
3. Attach used existing Solidfire data disk to a running/stopped VM
4. Stop and Start an instance with Solidfire data disks attached
5. Expand disk by resizing Solidfire data disk by providing size
6. Expand disk by changing disk offering for the Solidfire data disk

Function	Operation	Description	Fix Status
Create	Take Volume Snapshot	Take data disk volume snapshot on active instance	✅
Delete	Delete an active Instance	Delete an active Instance with attached 1:1 data disks	✅
Update	Attach used existing data disk	Attach a previous used data volume to a instance	✅
Update	Stop and Start a Instance	Stop and Start a Instance with attached 1:1 data disks	✅
Update	Resize Data volume	Expand data disk on demand	✅
Update	Change disk offering for the data volume	Expand data disk changing disk offering	✅
Update	Migrate data volume	Migrate a data volume between primaries storages Solidfire based	Unsupported feature

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

How Has This Been Tested?

Tested in a ACS 4.17 adv zone env with VMware and Solidfire setup. Tested all the following cases:

[weizhouapache]

great work @rohityadavcloud

[yadvr]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[DaanHoogland]

clgtm, but i cannot test this. Can you extract the workarounds (and containg blocks) in separate methods and where feasible add unit tests?

[shwstppr]

Code change looks good in terms of logic. Left some comments. May need manual testing with different storage providers

[codecov]

Codecov Report

Merging #7761 (76e5345) into 4.18 (b37834f) will decrease coverage by 0.01%.
Report is 1 commits behind head on 4.18.
The diff coverage is 0.00%.

@@ Coverage Diff @@## 4.18 #7761 +/- ##
============================================
- Coverage 13.06% 13.05% -0.01% - Complexity 9084 9086 +2 
============================================
Files 2720 2720 Lines 257370 257394 +24 Branches 40124 40130 +6 ============================================
- Hits 33614 33612 -2 - Misses 219534 219558 +24 - Partials 4222 4224 +2 

Files Changed	Coverage Δ
...stack/engine/orchestration/VolumeOrchestrator.java	1.89% <0.00%> (ø)
...visor/vmware/manager/VmwareStorageManagerImpl.java	6.20% <0.00%> (-0.03%)	⬇️
...oud/hypervisor/vmware/resource/VmwareResource.java	4.48% <0.00%> (-0.01%)	⬇️
...cloud/storage/resource/VmwareStorageProcessor.java	0.32% <0.00%> (-0.01%)	⬇️
...astore/driver/SolidFirePrimaryDataStoreDriver.java	0.00% <0.00%> (ø)
...ud/hypervisor/vmware/mo/HostDatastoreSystemMO.java	0.00% <0.00%> (ø)
...com/cloud/hypervisor/vmware/util/VmwareHelper.java	8.65% <ø> (ø)

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6532

[yadvr]

@blueorangutan test matrix

[blueorangutan]

@rohityadavcloud a [SF] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-7135)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43125 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7135-xenserver-71.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_cancel_host_maintenace_with_no_migration_jobs	Error	309.66	test_host_maintenance.py

[blueorangutan]

[SF] Trillian test result (tid-7137)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45954 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7137-kvm-centos7.zip
Smoke tests completed. 106 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_migrate_VM_and_root_volume	Error	76.80	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	Error	53.63	test_vm_life_cycle.py
test_hostha_enable_ha_when_host_in_maintenance	Error	303.84	test_hostha_kvm.py

[blueorangutan]

@rohityadavcloud a [SF] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian Build Failed (tid-7172)

[blueorangutan]

[SF] Trillian test result (tid-7171)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37558 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7171-xenserver-71.zip
Smoke tests completed. 108 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[blueorangutan]

[SF] Trillian test result (tid-7173)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 48780 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7173-kvm-centos7.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_migrate_VM_and_root_volume	Error	76.57	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	Error	55.40	test_vm_life_cycle.py

[weizhouapache]

@rohityadavcloud
is this ready for merge ?

[yadvr]

Yes, @weizhouapache but @NuxRo needs to sign off with his testing. I'll change the PR to draft.

[weizhouapache]

Yes, @weizhouapache but @NuxRo needs to sign off with his testing. I'll change the PR to draft.

Ok thanks @rohityadavcloud

[weizhouapache]

cc @NuxRo
plaese kindly approve it if it is tested good

[weizhouapache]

code lgtm

[yadvr]

@NuxRo completed QA on this, against a community-supplied 4.17 Solidfire env (before he went on holidays). Now with @andrijapanicsb to finalise review. Thanks.

@blueorangutan package

[blueorangutan]

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6877

[weizhouapache]

@blueorangutan test rocky8 vmware-67u3

[blueorangutan]

@weizhouapache a [SF] Trillian-Jenkins test job (rocky8 mgmt + vmware-67u3) has been kicked to run smoke tests

[weizhouapache]

@blueorangutan test ubuntu22 vmware-70u3

[blueorangutan]

@weizhouapache [SF] unsupported parameters provided. Supported mgmt server os are: centos7, centos6, suse15, alma8, ubuntu18, ubuntu22, ubuntu20, rocky8, alma9. Supported hypervisors are: kvm-centos6, kvm-centos7, kvm-rocky8, kvm-alma8, kvm-alma9, kvm-ubuntu18, kvm-ubuntu20, kvm-ubuntu22, kvm-suse15, vmware-55u3, vmware-60u2, vmware-65u2, vmware-67u3, vmware-70u1, vmware-70u2, vmware-70u3, vmware-80, vmware-80u1, xenserver-65sp1, xenserver-71, xenserver-74, xcpng74, xcpng76, xcpng80, xcpng81, xcpng82

[weizhouapache]

@blueorangutan test rocky8 vmware-70u3

[blueorangutan]

@weizhouapache a [SF] Trillian-Jenkins test job (rocky8 mgmt + vmware-70u3) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-7531)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 49474 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7531-vmware-67u3.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_upgrade_kubernetes_cluster	Failure	560.83	test_kubernetes_clusters.py

[blueorangutan]

[SF] Trillian test result (tid-7532)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 47727 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7761-t7532-vmware-70u3.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_upgrade_kubernetes_cluster	Failure	585.85	test_kubernetes_clusters.py