Skip to content

Linstor: add support for ISO block devices and direct download - #9792

Merged
DaanHoogland merged 2 commits into
apache:4.19from
LINBIT:linstor-4.19-iso-direct-download
Nov 28, 2024
Merged

Linstor: add support for ISO block devices and direct download#9792
DaanHoogland merged 2 commits into
apache:4.19from
LINBIT:linstor-4.19-iso-direct-download

Conversation

@rp-

@rp-rp- commented Oct 14, 2024

Copy link
Copy Markdown
Contributor

Description

This PR add KVM support for BLOCK driver ISO/CDROM devices and add support for Linstor to it.
It is usually only used with the "direct download" option on ISO templates.

Fixes: #9772

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Linstor hyper-converged cluster

How did you try to break this feature and the system with this change?

@codecov

codecovBot commented Oct 14, 2024•
edited
Loading

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 3.89610% with 74 lines in your changes missing coverage. Please review.

Project coverage is 15.10%. Comparing base (a6cef7a) to head (9511e78).
Report is 32 commits behind head on 4.19.

Files with missing linesPatch %Lines
.../hypervisor/kvm/storage/LinstorStorageAdaptor.java0.00%31 Missing ⚠️
...ervisor/kvm/resource/LibvirtComputingResource.java0.00%20 Missing ⚠️
...om/cloud/hypervisor/kvm/resource/LibvirtVMDef.java16.66%10 Missing ⚠️
...ypervisor/kvm/resource/LibvirtDomainXMLParser.java12.50%7 Missing ⚠️
...ud/hypervisor/kvm/storage/KVMStorageProcessor.java0.00%3 Missing ⚠️
.../hypervisor/kvm/storage/LibvirtStorageAdaptor.java0.00%3 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## 4.19 #9792 +/- ##
============================================
- Coverage 15.11% 15.10% -0.01% + Complexity 11217 11214 -3 
============================================
Files 5404 5404 Lines 473426 473468 +42 Branches 60681 61036 +355 ============================================
- Hits 71539 71533 -6 - Misses 393891 393940 +49 + Partials 7996 7995 -1
FlagCoverage Δ
uitests4.30% <ø> (ø)
unittests15.82% <3.89%> (-0.01%)⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sureshanapartisureshanaparti added this to the 4.19.2.0 milestone Oct 15, 2024
sureshanaparti
sureshanaparti approved these changes Oct 15, 2024
View reviewed changes

@sureshanapartisureshanaparti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some refactor suggestions, overall code lgtm.

@rp-
rp-force-pushed the linstor-4.19-iso-direct-download branch from d16841a to 44e2b4dCompareOctober 16, 2024 14:13
DaanHoogland
DaanHoogland approved these changes Oct 28, 2024
View reviewed changes

@DaanHooglandDaanHoogland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

Comment thread.../hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtDomainXMLParser.java Outdated
Comment thread...age/volume/linstor/src/main/java/com/cloud/hypervisor/kvm/storage/LinstorStorageAdaptor.java Outdated
Comment thread...age/volume/linstor/src/main/java/com/cloud/hypervisor/kvm/storage/LinstorStorageAdaptor.java Outdated
@rp-
rp-force-pushed the linstor-4.19-iso-direct-download branch from 44e2b4d to dc02bafCompareOctober 29, 2024 12:21
@rp-

rp- commented Oct 29, 2024

Copy link
Copy Markdown
ContributorAuthor

Did Daan's suggestions.

@rp-
rp-force-pushed the linstor-4.19-iso-direct-download branch from dc02baf to 4e970b8CompareOctober 31, 2024 07:30
@github-actions

github-actionsBot commented Oct 31, 2024

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@rp-
rp-force-pushed the linstor-4.19-iso-direct-download branch from 4e970b8 to d8b8ad5CompareOctober 31, 2024 07:32
@rp-

rp- commented Oct 31, 2024

Copy link
Copy Markdown
ContributorAuthor

Just added a CHANGELOG.md entry no functional changes

@rp-

rp- commented Oct 31, 2024

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan package

@blueorangutan

blueorangutan commented Oct 31, 2024

Copy link
Copy Markdown

@rp- a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

blueorangutan commented Oct 31, 2024

Copy link
Copy Markdown

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 11474

@DaanHoogland

DaanHoogland commented Oct 31, 2024

Copy link
Copy Markdown
Contributor

@blueorangutan LLpackage

@blueorangutan

blueorangutan commented Nov 1, 2024

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

blueorangutan commented Nov 2, 2024

Copy link
Copy Markdown

[SF] Trillian test result (tid-11731)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 71490 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9792-t11731-kvm-ol8.zip
Smoke tests completed. 86 look OK, 47 have errors, 0 did not run
Only failed and skipped tests results shown below:

TestResultTime (s)Test File
ContextSuite context=TestAddVmToSubDomain>:teardownError26.24test_accounts.py
test_DeleteDomainError181.77test_accounts.py
test_forceDeleteDomainFailure349.09test_accounts.py
test_forceDeleteDomainError349.10test_accounts.py
test_01_events_resourceError410.75test_events_resource.py
test_01_events_resourceError410.77test_events_resource.py
ContextSuite context=TestDeployVirtioSCSIVM>:setupError0.00test_deploy_virtio_scsi_vm.py
test_deploy_more_vms_than_limit_allowsError105.25test_deploy_vms_in_parallel.py
test_04_list_domains_level_filterFailure0.04test_list_domains.py
test_05_list_domains_no_filterFailure0.02test_list_domains.py
ContextSuite context=TestDomainsServiceOfferings>:teardownError294.25test_domain_service_offerings.py
ContextSuite context=TestHostControlState>:teardownError32.54test_host_control_state.py
ContextSuite context=TestImportAndUnmanageVolumes>:teardownError22.88test_import_unmanage_volumes.py
test_01_invalid_upgrade_kubernetes_clusterError354.45test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_clusterError63.93test_kubernetes_clusters.py
test_06_delete_kubernetes_clusterError2.33test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_clusterError2.24test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_clusterError1.21test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardownError42.23test_kubernetes_clusters.py
ContextSuite context=TestListIdsParams>:setupError0.00test_list_ids_parameter.py
ContextSuite context=TestListVolumes>:setupError0.00test_list_volumes.py
test_list_vms_metrics_adminError31.72test_metrics_api.py
test_list_vms_metrics_historyError27.37test_metrics_api.py
test_list_volumes_metrics_historyError30.39test_metrics_api.py
test_01_verify_ipv6_networkError204.06test_network_ipv6.py
test_01_verify_ipv6_networkError204.08test_network_ipv6.py
test_03_network_operations_on_created_vm_of_otheruserError206.74test_network_permissions.py
ContextSuite context=TestNetworkPermissions>:teardownError6.63test_network_permissions.py
ContextSuite context=TestPublicIP>:teardownError33.39test_network.py
test_reboot_routerError278.38test_network.py
test_reboot_routerError278.39test_network.py
ContextSuite context=TestNonStrictAffinityGroups>:teardownError198.17test_nonstrict_affinity_group.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:teardownError42.86test_password_server.py
test_01_isolated_persistent_networkError59.42test_persistent_network.py
ContextSuite context=TestL2PersistentNetworks>:teardownError49.93test_persistent_network.py
test_01_create_delete_portforwarding_fornonvpcError161.30test_portforwardingrules.py
test_01_add_primary_storage_disabled_hostError0.47test_primary_storage.py
test_01_primary_storage_nfsError0.20test_primary_storage.py
ContextSuite context=TestStorageTags>:setupError0.33test_primary_storage.py
test_01_primary_storage_scope_changeError0.17test_primary_storage_scope.py
test_09_project_suspendError81.95test_projects.py
test_deploy_vm_with_registered_userdataError116.10test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allowError144.26test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_appendError150.04test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_paramsError103.77test_register_userdata.py
test_01_reset_vm_on_rebootError5.22test_reset_vm_on_reboot.py
ContextSuite context=TestRAMCPUResourceAccounting>:teardownError167.14test_resource_accounting.py
test_01_restore_vmError0.05test_restore_vm.py
test_02_restore_vm_allocated_rootError0.05test_restore_vm.py
ContextSuite context=TestRouterDHCPHosts>:teardownError103.39test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:teardownError40.99test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:teardownError23.16test_router_dns.py
ContextSuite context=TestRouterDnsService>:teardownError37.63test_router_dnsservice.py
test_01_single_VPC_iptables_policiesError150.37test_routers_iptables_default_policy.py
test_01_isolate_network_FW_PF_default_routes_egress_trueError133.52test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_falseFailure107.91test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_falseError107.91test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_trueError235.57test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_falseError255.34test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_falseError255.34test_routers_network_ops.py
test_03_RVR_Network_check_router_stateError256.59test_routers_network_ops.py
test_07_stop_routerError5.24test_routers.py
test_09_reboot_routerError27.81test_routers.py
ContextSuite context=TestCpuCapServiceOfferings>:setupError0.00test_service_offerings.py
test_04_change_offering_smallFailure7.24test_service_offerings.py
test_05_disk_offering_strictness_trueError1.38test_service_offerings.py
test_06_disk_offering_strictness_falseError1.37test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setupError0.00test_snapshots.py
ContextSuite context=TestSnapshotStandaloneBackup>:setupError0.00test_snapshots.py
test_05_stop_ssvmError51.60test_ssvm.py
test_06_stop_cpvmError21.81test_ssvm.py
test_09_reboot_ssvm_forcedError17.68test_ssvm.py
test_10_reboot_cpvm_forcedError22.79test_ssvm.py
ContextSuite context=TestTemplates>:setupError115.53test_templates.py
ContextSuite context=TestLBRuleUsage>:setupError20.51test_usage.py
ContextSuite context=TestNatRuleUsage>:setupError123.10test_usage.py
ContextSuite context=TestPublicIPUsage>:setupError150.57test_usage.py
ContextSuite context=TestVmUsage>:setupError171.12test_usage.py
ContextSuite context=TestVolumeUsage>:setupError181.12test_usage.py
ContextSuite context=TestVpnUsage>:setupError191.38test_usage.py
test_01_scale_up_verifyFailure35.05test_vm_autoscaling.py
test_02_update_vmprofile_and_vmgroupFailure245.67test_vm_autoscaling.py
test_03_scale_down_verifyFailure304.60test_vm_autoscaling.py
test_04_stop_remove_vm_in_vmgroupFailure0.03test_vm_autoscaling.py
test_06_autoscaling_vmgroup_on_project_networkFailure45.48test_vm_autoscaling.py
test_06_autoscaling_vmgroup_on_project_networkError45.49test_vm_autoscaling.py
test_07_autoscaling_vmgroup_on_vpc_networkError1.24test_vm_autoscaling.py
ContextSuite context=TestVmAutoScaling>:teardownError12.33test_vm_autoscaling.py
test_01_deploy_vm_on_specific_hostError0.09test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_clusterError13.64test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_podError30.13test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_clusterError0.15test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_podError9.56test_vm_deployment_planner.py
ContextSuite context=TestDeployVM>:setupError0.00test_vm_life_cycle.py
test_01_migrate_VM_and_root_volumeError1.52test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disksError1.43test_vm_life_cycle.py
ContextSuite context=TestVMLifeCycle>:setupError3.83test_vm_life_cycle.py
ContextSuite context=TestVMSchedule>:setupError0.00test_vm_schedule.py
test_04_deploy_vnf_applianceError312.26test_vnf_templates.py
ContextSuite context=TestCreateVolume>:setupError0.00test_volumes.py
test_01_root_volume_encryptionError1.34test_volumes.py
test_02_data_volume_encryptionError1.33test_volumes.py
test_03_root_and_data_volume_encryptionError1.34test_volumes.py
ContextSuite context=TestVolumes>:setupError44.34test_volumes.py
test_01_verify_ipv6_vpcError8.56test_vpc_ipv6.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACLError15.46test_vpc_redundant.py
test_02_redundant_VPC_default_routesError10.93test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routersError13.97test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nicsError14.01test_vpc_redundant.py
test_05_rvpc_multi_tiersError14.21test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setupError0.00test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setupError0.00test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setupError0.00test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setupError0.00test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setupError0.00test_vpc_vpn.py

slavkap
slavkap requested changes Nov 4, 2024
View reviewed changes
Comment thread...ypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java Outdated
@rp-
rp-force-pushed the linstor-4.19-iso-direct-download branch from d8b8ad5 to d17325dCompareNovember 4, 2024 14:33
rp- added 2 commits November 4, 2024 15:34
@rp-
kvm-libvirtCR: fix == should be .equals errors
7f22d66
@rp-
linstor/kvm: add support for ISO block devices and direct download
9511e78
If Linstor storage pool is used, use the BLOCK qemu driver for the
Linstor block device.
# Conflicts:
#	plugins/storage/volume/linstor/CHANGELOG.md
@rp-
rp-force-pushed the linstor-4.19-iso-direct-download branch from d17325d to 9511e78CompareNovember 4, 2024 14:35
@rp-

rp- commented Nov 4, 2024

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan package

@blueorangutan

blueorangutan commented Nov 4, 2024

Copy link
Copy Markdown

@rp- a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

blueorangutan commented Nov 4, 2024

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11498

@rp-

rp- commented Nov 4, 2024

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan test

2 similar comments
@DaanHoogland

DaanHoogland commented Nov 5, 2024

Copy link
Copy Markdown
Contributor

@blueorangutan test

@DaanHoogland

DaanHoogland commented Nov 5, 2024

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

blueorangutan commented Nov 5, 2024

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

blueorangutan commented Nov 5, 2024

Copy link
Copy Markdown

[SF] Trillian test result (tid-11744)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 49159 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9792-t11744-kvm-ol8.zip
Smoke tests completed. 133 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

TestResultTime (s)Test File

@rp-rp- self-assigned this Nov 12, 2024
@rp-rp- linked an issue Nov 12, 2024 that may be closed by this pull request
bug: vm is not created for a direct download setup #9772
Closed
@rp-
rp- requested a review from slavkapNovember 12, 2024 07:58
@slavkap

slavkap commented Nov 14, 2024

Copy link
Copy Markdown
Contributor

Sorry @rp-, I had some urgent internal tasks. I'm building an environment now and will execute some tests and will check the code again

slavkap
slavkap approved these changes Nov 14, 2024
View reviewed changes

@slavkapslavkap left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code LGTM, tested with StorPool and NFS

@DaanHoogland
DaanHoogland merged commit d54b105 into apache:4.19Nov 28, 2024
@DaanHooglandDaanHoogland mentioned this pull request Nov 28, 2024
Closed
DaanHoogland added a commit that referenced this pull request Dec 3, 2024
@DaanHoogland
Merge release branch 4.20 to main
9b6f9b5
* 4.20:
UI: Tooltip on the host information card to display the CPU speed in MHz and the memory value in MB (to 3 decimal places) (#9971)
UI: Allow accounts of the `User` type to add other accounts or users to projects through UI (#9927)
enable to create VPC portfowarding rules with source cidr (#7081)
Add new column `last_id` to the table volumes (#9759)
Allow VMWare import via another host (#9787)
Linstor: add support for ISO block devices and direct download (#9792)
get expunged VM data for job result (#9949)
fix section divider display on auth page (#9966)
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Dec 12, 2024
@rp-@dhslove
Linstor: add support for ISO block devices and direct download (apach…
cc54e44
…e#9792)
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Dec 12, 2024
@DaanHoogland@dhslove
Merge release branch 4.20 to main
93d5557
* 4.20:
UI: Tooltip on the host information card to display the CPU speed in MHz and the memory value in MB (to 3 decimal places) (apache#9971)
UI: Allow accounts of the `User` type to add other accounts or users to projects through UI (apache#9927)
enable to create VPC portfowarding rules with source cidr (apache#7081)
Add new column `last_id` to the table volumes (apache#9759)
Allow VMWare import via another host (apache#9787)
Linstor: add support for ISO block devices and direct download (apache#9792)
get expunged VM data for job result (apache#9949)
fix section divider display on auth page (apache#9966)
@rp-
rp- deleted the linstor-4.19-iso-direct-download branch February 13, 2025 08:54
rp- added a commit to LINBIT/cloudstack that referenced this pull request Mar 11, 2025
@rp-
Linstor: add support for ISO block devices and direct download (apach…
aeb84b0
…e#9792)
rp- added a commit to LINBIT/cloudstack that referenced this pull request Feb 24, 2026
@rp-
Linstor: add support for ISO block devices and direct download (apach…
78ae2b8
…e#9792)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DaanHooglandDaanHooglandDaanHoogland approved these changes
@sureshanapartisureshanapartisureshanaparti approved these changes
@borisstoyanovborisstoyanovborisstoyanov approved these changes
@slavkapslavkapslavkap approved these changes
@shwstpprshwstpprAwaiting requested review from shwstppr

Assignees

@rp-rp-

Labels

component:kvmcomponent:primary-storagecomponent:storage

Projects

None yet

Milestone

4.19.2

Development

Successfully merging this pull request may close these issues.

bug: vm is not created for a direct download setup

6 participants

@rp-@blueorangutan@DaanHoogland@slavkap@sureshanaparti@borisstoyanov