Skip to content

[VMware] vSphere advanced capabilities and Full OVF properties support - #4307

Merged
borisstoyanov merged 166 commits into
apache:masterfrom
shapeblue:ovfprops-and-vsphere-adv-together
Oct 26, 2020
Merged

[VMware] vSphere advanced capabilities and Full OVF properties support#4307
borisstoyanov merged 166 commits into
apache:masterfrom
shapeblue:ovfprops-and-vsphere-adv-together

Conversation

@nvazquez

Copy link
Copy Markdown
Contributor

Description

Combined PR of #4248 and #4250

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1911

@yadvr

yadvr commented Sep 7, 2020

Copy link
Copy Markdown
Member

@blueorangutan test matrix

@blueorangutan

Copy link
Copy Markdown

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-2697)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 68565 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4307-t2697-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 81 look OK, 4 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_01_add_delete_kubernetes_supported_versionError1807.37test_kubernetes_supported_versions.py
test_02_vpc_privategw_static_routesFailure282.51test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanupFailure266.32test_privategw_acl.py
test_04_rvpc_privategw_static_routesFailure540.15test_privategw_acl.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACLFailure439.35test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routersFailure587.17test_vpc_redundant.py
test_05_rvpc_multi_tiersFailure517.08test_vpc_redundant.py
test_hostha_enable_ha_when_host_disabledError2.62test_hostha_kvm.py
test_hostha_enable_ha_when_host_in_maintenanceError302.78test_hostha_kvm.py

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-2698)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 75260 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4307-t2698-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Smoke tests completed. 81 look OK, 4 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_03_create_iso_with_checksum_md5Error72.52test_iso.py
test_03_create_iso_with_checksum_md5Error73.56test_iso.py
test_04_create_iso_with_no_checksumError73.64test_iso.py
test_01_add_delete_kubernetes_supported_versionError1813.90test_kubernetes_supported_versions.py
test_02_list_snapshots_with_removed_data_storeError179.25test_snapshots.py
test_07_reboot_ssvmFailure57.99test_ssvm.py

@harikrishna-patnala

Copy link
Copy Markdown
Member

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@harikrishna-patnala a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1924

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-2699)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 101618 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4307-t2699-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_async_job.py
Intermittent failure detected: /marvin/tests/smoke/test_certauthority_root.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermittent failure detected: /marvin/tests/smoke/test_domain_vpc_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_network_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Smoke tests completed. 79 look OK, 6 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
ContextSuite context=TestAddVmToSubDomain>:setupError1849.30test_accounts.py
ContextSuite context=TestTemplateHierarchy>:setupError1529.07test_accounts.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80Error0.10test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfacesFailure910.84test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfacesFailure1027.43test_internal_lb.py
test_03_create_vpc_domain_vpc_offeringError917.86test_domain_vpc_offerings.py
test_01_add_delete_kubernetes_supported_versionError1807.56test_kubernetes_supported_versions.py
test_network_aclError912.02test_network_acl.py
test_01_scale_vmFailure93.40test_scale_vm.py

@yadvr

yadvr commented Sep 9, 2020

Copy link
Copy Markdown
Member

@blueorangutan test centos7 vmware-67u3

@blueorangutan

Copy link
Copy Markdown

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

@harikrishna-patnala

Copy link
Copy Markdown
Member

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@harikrishna-patnala a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1952

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-2710)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 62629 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4307-t2710-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_pvlan.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_deployment_planner.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 77 look OK, 6 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_00_deploy_vm_root_resizeFailure587.23test_deploy_vm_root_resize.py
test_01_add_delete_kubernetes_supported_versionError1808.22test_kubernetes_supported_versions.py
test_create_pvlan_networkError0.03test_pvlan.py
test_02_list_snapshots_with_removed_data_storeError182.79test_snapshots.py
test_02_deploy_vm_on_specific_clusterError4.47test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_podError7.06test_vm_deployment_planner.py
ContextSuite context=TestRVPCSite2SiteVpn>:teardownError604.00test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setupError604.03test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setupError604.08test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setupError604.10test_vpc_vpn.py

@harikrishna-patnala

Copy link
Copy Markdown
Member

@blueorangutan package

@apacheapache deleted a comment from blueorangutanSep 11, 2020
@apacheapache deleted a comment from blueorangutanSep 11, 2020
@blueorangutan

Copy link
Copy Markdown

@harikrishna-patnala a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@borisstoyanov

Copy link
Copy Markdown
Contributor

Kubernetes cluster test seems to be present in other PRs as well, I've run this on the test env we have with this PR and got

=== TestName: test_01_deploy_kubernetes_cluster | Status : SUCCESS ===
=== TestName: test_02_invalid_upgrade_kubernetes_cluster | Status : SUCCESS ===
=== TestName: test_03_deploy_and_upgrade_kubernetes_cluster | Status : FAILED ===
=== TestName: test_04_deploy_and_scale_kubernetes_cluster | Status : SUCCESS ===
=== TestName: test_05_delete_kubernetes_cluster | Status : SUCCESS ===
=== TestName: test_06_deploy_invalid_kubernetes_ha_cluster | Status : SUCCESS ===
=== TestName: test_07_deploy_kubernetes_ha_cluster | Status : SUCCESS ===
=== TestName: test_08_deploy_and_upgrade_kubernetes_ha_cluster | Status : SUCCESS ===
=== TestName: test_09_delete_kubernetes_ha_cluster | Status : SUCCESS ===

Then the same error with test03 has been seen here as well:
#4382 (comment)

The storage policy failure is related to setup and using an incompatible policy with our infra, we'll fix the test now to evade this action and we'll look to fix this one later as it's not breaking anything.

I think we're in a good position to merge this!

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2277

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@borisstoyanov

Copy link
Copy Markdown
Contributor

cat /marvin//MarvinLogs/test_storage_policy_DFIU0C/results.txt
Test VMWare storage policies ... === TestName: test_01_import_storage_policies | Status : SUCCESS ===
ok


Ran 1 test in 70.694s

OK
LGTM, locally tested

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2278

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan test matrix

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan test centos7 vmware-67u3

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

@borisstoyanov
borisstoyanov merged commit 1a75872 into apache:masterOct 26, 2020
@nvazquez
nvazquez deleted the ovfprops-and-vsphere-adv-together branch October 26, 2020 13:44
@weizhouapache

Copy link
Copy Markdown
Member

@borisstoyanov@nvazquez
I think we normally merge a pr with only 1 commit.

@borisstoyanov

borisstoyanov commented Oct 26, 2020

Copy link
Copy Markdown
Contributor

Sorry about that Wei, my bad

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3077)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33433 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4307-t3077-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_03_deploy_and_upgrade_kubernetes_clusterFailure248.51test_kubernetes_clusters.py

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3079)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37821 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4307-t3079-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_03_deploy_and_upgrade_kubernetes_clusterFailure791.03test_kubernetes_clusters.py

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3078)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38296 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4307-t3078-vmware-65u2.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_03_deploy_and_upgrade_kubernetes_clusterFailure782.42test_kubernetes_clusters.py

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3076)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 79402 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4307-t3076-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_public_ip_range.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_storage_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 64 look OK, 22 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
ContextSuite context=TestResetVmOnReboot>:setupError0.00test_reset_vm_on_reboot.py
ContextSuite context=TestRouterDHCPHosts>:setupError0.00test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setupError0.00test_router_dhcphosts.py
ContextSuite context=TestRAMCPUResourceAccounting>:setupError0.00test_resource_accounting.py
ContextSuite context=TestRouterDns>:setupError0.00test_router_dns.py
ContextSuite context=TestRouterDnsService>:setupError0.00test_router_dnsservice.py
ContextSuite context=TestRouterIpTablesPolicies>:setupError0.00test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setupError0.00test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setupError0.00test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setupError0.00test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setupError0.00test_routers.py
ContextSuite context=TestScaleVm>:setupError0.00test_scale_vm.py
ContextSuite context=TestServiceOfferings>:setupError1513.62test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setupError0.00test_snapshots.py
test_01_list_sec_storage_vmFailure0.04test_ssvm.py
test_02_list_cpvm_vmFailure0.04test_ssvm.py
test_03_ssvm_internalsFailure0.04test_ssvm.py
test_04_cpvm_internalsFailure0.04test_ssvm.py
test_05_stop_ssvmFailure0.04test_ssvm.py
test_06_stop_cpvmFailure0.04test_ssvm.py
test_07_reboot_ssvmFailure0.04test_ssvm.py
test_08_reboot_cpvmFailure0.04test_ssvm.py
test_09_destroy_ssvmFailure0.05test_ssvm.py
test_10_destroy_cpvmFailure0.04test_ssvm.py
ContextSuite context=TestVMWareStoragePolicies>:setupError0.00test_storage_policy.py
test_02_create_template_with_checksum_sha1Error65.49test_templates.py
test_03_create_template_with_checksum_sha256Error65.51test_templates.py
test_04_create_template_with_checksum_md5Error65.52test_templates.py
test_05_create_template_with_no_checksumError65.49test_templates.py
ContextSuite context=TestTemplates>:setupError2.35test_templates.py
ContextSuite context=TestISOUsage>:setupError0.00test_usage.py
ContextSuite context=TestLBRuleUsage>:setupError0.00test_usage.py
ContextSuite context=TestNatRuleUsage>:setupError0.00test_usage.py
ContextSuite context=TestPublicIPUsage>:setupError0.00test_usage.py
ContextSuite context=TestSnapshotUsage>:setupError0.00test_usage.py
ContextSuite context=TestVmUsage>:setupError0.00test_usage.py
ContextSuite context=TestVolumeUsage>:setupError0.00test_usage.py
ContextSuite context=TestVpnUsage>:setupError0.00test_usage.py
ContextSuite context=Test01DeployVM>:setupError0.00test_vm_life_cycle.py
ContextSuite context=Test02VMLifeCycle>:setupError0.00test_vm_life_cycle.py
ContextSuite context=TestChangeServiceOfferingForVmWithSnapshots>:setupError0.00test_vm_snapshots.py
ContextSuite context=TestVmSnapshot>:setupError0.00test_vm_snapshots.py
ContextSuite context=TestCreateVolume>:setupError0.00test_volumes.py
ContextSuite context=TestVolumes>:setupError0.00test_volumes.py
ContextSuite context=TestVPCRedundancy>:setupError0.00test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setupError0.00test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setupError0.00test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setupError0.00test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setupError0.00test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setupError0.00test_vpc_vpn.py
test_02_cancel_host_maintenace_with_migration_jobsError1513.72test_host_maintenance.py

detachVolumesFromVm(allVolumes);
} else {
detachVolumesFromVm(volumesToBeDeleted);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these changes are reverted in the PR #4493 , as the data disks (which are not to be deleted) are being detached on destroy VM, which is breaking the existing behavior.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

14 participants

@nvazquez@blueorangutan@yadvr@harikrishna-patnala@borisstoyanov@DaanHoogland@andrijapanicsb@alexandremattioli@weizhouapache@sureshanaparti@PaulAngus@vladimirpetrov@ACSGitBot@davidjumani