component test ports/fixes in python3

[DaanHoogland]

Description

This PR fixes marvin component tests to run in ported marvin framework for python3

part of #3195

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Work to do:

new failures in python3

• test_egress_fw_rules.py
• test_configdrive.py
• test_lb_secondary_ip.py
• test_deploy_vm_userdata_reg.py
• test_multiple_ips_per_nic.py
• test_ps_domain_limits.py
• test_ps_limits.py
• test_ps_resize_volume.py
• test_snapshots.py
• test_ss_max_limits.py
• test_volume_destroy_recover.py
• test_vpc_network_lbrules.py
• test_vpc_network_pfrules.py
• test_vpc_network.py
• test_vpc_network_staticnatrule.py
• test_vpc_offerings.py
• test_vpc_on_host_maintenance.py

different results:

• test_configdrive.py
• test_ps_resource_limits_volume.py
• test_vpc_network_internal_lbrules.py
• test_vpc_network_pfrules.py
• test_vpc_network.py
• test_vpc_network_staticnatrule.py
• test_vpc_offerings.py

different times:

• test_egress_fw_rules.py
• test_add_remove_network.py
• test_affinity_groups_projects.py
• test_affinity_groups.py
• test_base_image_updation.py
• test_blocker_bugs.py
• test_implicit_planner.py
• test_browse_templates.py
• test_configdrive.py
• test_escalation_listTemplateDomainAdmin.py
• test_escalations_ipaddresses.py
• test_escalations_networks.py
• test_escalations_volumes.py
• test_escalations_vpncustomergateways.py
• test_multiple_public_interfaces.py
• test_organization_states.py
• test_persistent_networks.py
• test_portable_ip.py
• test_redundant_router_cleanups.py
• test_reset_ssh_keypair.py
• test_rootvolume_resize.py
• test_shared_networks.py
• test_snapshots.py
• test_ss_limits.py
• test_ss_max_limits.py
• test_ss_project_limits.py
• test_stopped_vm.py
• test_storage_motion.py
• test_tags.py
• test_volumes.py
• test_vpc_network_internal_lbrules.py
• test_vpc_vms_deployment.py
• test_vpn_service.py
• test_vpn_users.py
• test_vr_metadata.py
• test_bugs.py
• test_dedicate_guest_vlan_ranges.py
• test_dedicate_public_ip_range.py
• test_escalations_hosts.py
• test_escalation_templates.py
• test_explicit_dedication.py
• test_ha_pool_maintenance.py
• test_high_availability.py
• test_ip_resource_count_for_vpc.py
• test_vpc_vms_deployment.py
• test_redundant_router_deployment_planning.py
• test_redundant_router_network_rules.py
• test_redundant_router.py

exact matches:

• test_dhcp_dns_offload.py
• test_browse_volumes.py
• test_escalations_networks.py
• test_multiple_public_interfaces.py
• test_organization_states.py
• test_vpc_routers.py
• test_vpc_vm_life_cycle.py
• test_vpc_host_maintenance.py
• test_vpc_vms_deployment.py
• test_zone_level_local_storage_setting.py

####### only in python2:

• test_browse_templates2.py
• test_deploy_vgpu_vm.py
• test_escalations_volumes.py
• test_snapshots.py
• test_ss_limits.py
• test_dedicate_public_ip_range.py

Screenshots (if appropriate):

How Has This Been Tested?

[DaanHoogland]

initial test results:

Test completed. 104 look ok, 68 have errors

however

# ls tests/component/ | wc
159 159 4106

still to investigate the discrepency between 104+68 and 159 ???

cc @borisstoyanov @rhtyd

[DaanHoogland]

still to investigate the discrepency between 104+68 and 159 ???

found it but it yield a discrepency the other direction:

# ls -RF1 | grep "test_.*.py" | wc
187 187 4993

seems 15 test-suites either don't have tests or are not being run.

[DaanHoogland]

compared to a python2 baseline the differences are:
python2: Test completed. 112 look ok, 60 have errors
python3: Test completed. 104 look ok, 68 have errors

Test completed here means 8 test-suites. The actual difference in number of tests failing or erroring are

$ wc -l py2-components.txt py3-components.txt 198 py2-components.txt
251 py3-components.txt

hence 53.

There are however tests that failed in python2 and passed in python3 as well, so it is a bit more complicated. The categories of differnces are:

• failure in both python2 and python3
• • shorter duration in python3
• • shorter duration in pyhton2
• • not the same duration but less than 50% and a second difference
• failure in python2 and error in python3
• error in python2 and failure in python3
• failure only in python3
• error only in python3

and last and least

• faileure or error only in python2 (these are less interesting for us)

I'll edit this comment with updates

tables with differences in decreasing level of interest

failures only in python3:

Test	Result	Time in py3	Test File
test_06_egress_fr6	Error	893.64	test_egress_fw_rules.py
test_configdrive_shared_network	Failure	1.64	test_configdrive.py
test_13_delete_lb_rule	Error	1300.28	test_lb_secondary_ip.py
test_deployvm_userdata_post	Failure	58.73	test_deploy_vm_userdata_reg.py
test_delete_vm_2_SHARED	Failure	1.50	test_multiple_ips_per_nic.py
test_01_multiple_domains_primary_storage_limits	Failure	7.82	test_ps_domain_limits.py
test_03_multiple_domains_multiple_volumes	Error	313.00	test_ps_domain_limits.py
test_deploy_multiple_vm_1_root_domain_admin	Failure	161.50	test_ps_limits.py
test_deploy_multiple_vm_2_child_domain_admin	Failure	168.32	test_ps_limits.py
test_01_increase_volume_size_within_account_limit	Failure	2.65	test_ps_resize_volume.py
test_02_increase_volume_size_above_account_limit	Failure	2.53	test_ps_resize_volume.py
test_03_increase_volume_size_above_domain_limit	Failure	2.52	test_ps_resize_volume.py
test_08_increase_volume_size_within_account_limit	Failure	1.57	test_rootvolume_resize.py
test_08_increase_volume_size_within_account_limit	Error	1.57	test_rootvolume_resize.py
test_01_createVM_snapshotTemplate	Failure	217.92	test_snapshots.py
test_02_snapshot_data_disk	Failure	177.39	test_snapshots.py
test_03_deploy_vm_project_limit_reached	Failure	31.09	test_ss_max_limits.py
test_01_create_vm_with_data_disk	Failure	61.77	test_volume_destroy_recover.py
test_02_destroy_allocated_volume	Failure	1.10	test_volume_destroy_recover.py
test_03_destroy_detached_volume	Failure	34.26	test_volume_destroy_recover.py
test_04_recover_root_volume_after_restorevm	Failure	34.26	test_volume_destroy_recover.py
test_05_extract_root_volume_and_destroy_vm	Failure	34.23	test_volume_destroy_recover.py
test_06_delete_network	Error	1.06	test_volume_destroy_recover.py
test_02_VPC_CreateLBRuleInMultipleNetworks	Failure	182.86	test_vpc_network_lbrules.py
test_03_VPC_CreateLBRuleInMultipleNetworksVRStoppedState	Failure	325.07	test_vpc_network_lbrules.py
test_04_VPC_CreateLBRuleInMultipleNetworksVRStoppedState	Failure	185.61	test_vpc_network_lbrules.py
test_05_VPC_CreateAndDeleteLBRule	Failure	187.39	test_vpc_network_lbrules.py
test_06_VPC_CreateAndDeleteLBRuleVRStopppedState	Failure	334.09	test_vpc_network_lbrules.py
test_07_VPC_CreateAndDeleteAllLBRule	Failure	192.68	test_vpc_network_lbrules.py
test_08_VPC_CreateAndDeleteAllLBRuleVRStoppedState	Failure	191.49	test_vpc_network_lbrules.py
test_09_VPC_LBRuleCreateFailMultipleVPC	Failure	248.62	test_vpc_network_lbrules.py
test_11_VPC_LBRuleCreateNotAllowed	Failure	218.06	test_vpc_network_lbrules.py
test_01_network_services_VPC_StopCreatePF	Error	140.11	test_vpc_network_pfrules.py
test_02_network_services_VPC_CreatePF	Error	105.30	test_vpc_network_pfrules.py
test_03_network_services_VPC_StopCreateMultiplePF	Error	275.35	test_vpc_network_pfrules.py
test_04_network_services_VPC_CreateMultiplePF	Error	152.66	test_vpc_network_pfrules.py
ContextSuite context=TestVPCNetworkPFRules>:teardown	Error	3.62	test_vpc_network_pfrules.py
ContextSuite context=TestRouterOperations>:setup	Error	0.00	test_vpc_network.py
test_01_create_network_1_network_offering	Error	2.58	test_vpc_network.py
test_02_create_network_fail_1_network_offering	Error	2.55	test_vpc_network.py
test_07_create_network_unsupported_services	Error	4.69	test_vpc_network.py
test_08_create_network_without_sourceNAT	Error	4.67	test_vpc_network.py
test_09_create_network_shared_nwoff_1_network_off_shared	Error	2.55	test_vpc_network.py
ContextSuite context=TestVPCNetworkGc>:setup	Error	4.15	test_vpc_network.py
test_01_create_network_outside_range_1_network_offering	Error	2.56	test_vpc_network.py
test_02_create_network_outside_range	Error	4.71	test_vpc_network.py
test_03_create_network_inside_range_1_network_offering	Error	2.53	test_vpc_network.py
test_04_create_network_overlapping_range_1_network_offering	Error	2.65	test_vpc_network.py
test_05_create_network_diff_account_1_network_offering	Error	2.53	test_vpc_network.py
test_02_network_vpcvr2vr_upgrade	Error	4.69	test_vpc_network.py
test_01_VPC_StaticNatRuleCreateStoppedState	Error	3.59	test_vpc_network_staticnatrule.py
test_02_VPC_CreateStaticNatRule	Error	3.61	test_vpc_network_staticnatrule.py
test_03_VPC_StopCreateMultipleStaticNatRuleStopppedState	Error	2.58	test_vpc_network_staticnatrule.py
test_04_VPC_CreateMultipleStaticNatRule	Error	3.58	test_vpc_network_staticnatrule.py
test_03_vpc_off_without_lb	Error	4.89	test_vpc_offerings.py
test_04_vpc_off_without_static_nat	Error	6.02	test_vpc_offerings.py
test_05_vpc_off_without_pf	Error	4.93	test_vpc_offerings.py
test_07_update_vpc_off	Error	6.82	test_vpc_offerings.py
test_09_create_redundant_vpc_offering	Error	4.71	test_vpc_offerings.py
ContextSuite context=TestVPCHostMaintenance>:setup	Error	0.00	test_vpc_on_host_maintenance.py

different results:

Test	Result in py2	result in py3	Time in py2	time in py3	Test File
test_configdrive_isolated_network	Error	Failure	71.35	2.53	test_configdrive.py
test_attach_volume_exceeding_primary_limits	Failure	Error	188.77	140.57	test_ps_resource_limits_volume.py
test_04_internallb_appliance_operations_traffic	Failure	Error	256.79	957.08	test_vpc_network_internal_lbrules.py
test_05_network_services_VPC_StopDeletePF	Failure	Error	117.51	118.20	test_vpc_network_pfrules.py
test_06_network_services_VPC_DeletePF	Failure	Error	115.48	71.78	test_vpc_network_pfrules.py
test_07_network_services_VPC_StopDeleteAllPF	Failure	Error	110.20	3.58	test_vpc_network_pfrules.py
test_08_network_services_VPC_DeleteAllPF	Failure	Error	113.39	3.56	test_vpc_network_pfrules.py
test_09_network_services_VPC_StopDeleteAllMultiplePF	Failure	Error	226.63	3.59	test_vpc_network_pfrules.py
test_10_network_services_VPC_DeleteAllMultiplePF	Failure	Error	233.75	3.57	test_vpc_network_pfrules.py
test_01_network_services_upgrade	Failure	Error	110.54	2.54	test_vpc_network.py
test_05_network_services_VPC_DeleteAllPF	Failure	Error	107.70	2.60	test_vpc_network_staticnatrule.py
test_06_network_services_VPC_DeleteAllMultiplePF	Failure	Error	205.96	2.58	test_vpc_network_staticnatrule.py
test_02_deploy_vms_in_vpc_nw	Failure	Error	115.86	5.70	test_vpc_offerings.py

different times:

Test	Result	Time py2	time py3	Test File
test_01_2_egress_fr1	Failure	112.85	130.03	test_egress_fw_rules.py
test_14_add_nw_different_account_1_isolated	Failure	43.54	29.71	test_add_remove_network.py
test_14_add_nw_different_account_2_shared	Failure	33.29	30.21	test_add_remove_network.py
ContextSuite context=TestAddNetworkToVirtualMachine>:teardown	Error	141.05	144.66	test_add_remove_network.py
test_30_remove_nic_reattach	Failure	218.65	265.34	test_add_remove_network.py
test_02_deploy_vm_anti_affinity_group_fail_on_not_enough_hosts	Error	72.94	71.89	test_affinity_groups_projects.py
test_04_update_aff_grp_remove_all	Error	0.22	0.12	test_affinity_groups.py
test_06_update_aff_grp_invalid_args	Error	161.51	161.91	test_affinity_groups.py
ContextSuite context=TestBaseImageUpdate>:teardown	Error	4345.55	4349.63	test_base_image_updation.py
ContextSuite context=TestDataPersistency>:teardown	Error	190.89	180.68	test_blocker_bugs.py
test_01_deploy_vm_with_implicit_planner	Error	1.74	1.77	test_implicit_planner.py
test_01_Browser_template_Life_cycle_tpath	Failure	6.04	4.45	test_browse_templates.py
test_02_SSVM_Life_Cycle_With_Browser_Template_TPath	Failure	106.10	764.96	test_browse_templates.py
test_04_Browser_template_ResetVM_With_Deleted_Template	Failure	1.91	301.58	test_browse_templates.py
test_05_Browser_Upload_Template_with_all_API_parameters	Failure	1.79	200.48	test_browse_templates.py
test_06_Browser_Upload_template_resource_limits	Failure	1.77	257.55	test_browse_templates.py
test_07_Browser_Upload_template_secondary_storage_resource_limits	Failure	2.01	301.63	test_browse_templates.py
test_08_Browser_Upload_template_resource_limits_after_deletion	Failure	1.82	175.60	test_browse_templates.py
test_09_Browser_Upload_Volume_secondary_storage_resource_limits_after_deletion	Failure	2.39	246.93	test_browse_templates.py
test_browser_upload_template_incomplete	Failure	184.50	95.40	test_browse_templates.py
test_configdrive_isolated_network_hypervisor_hostname_exposed	Error	60.91	100.96	test_configdrive.py
test_configdrive_vpc_network	Error	76.58	129.44	test_configdrive.py
test_configdrive_vpc_network_verify_metadata	Error	70.70	119.08	test_configdrive.py
test_listtemplate	Failure	30.43	41.43	test_escalation_listTemplateDomainAdmin.py
test_listtemplate	Error	31.53	41.44	test_escalation_listTemplateDomainAdmin.py
test_05_create_delete_lbrule_forvpc	Error	26.27	26.13	test_escalations_ipaddresses.py
test_10_create_delete_portforwarding_forvpc	Error	27.25	28.23	test_escalations_ipaddresses.py
test_15_enable_disable_staticnat_forvpc	Error	28.25	25.09	test_escalations_ipaddresses.py
ContextSuite context=TestIpAddresses>:teardown	Error	1.82	0.53	test_escalations_ipaddresses.py
test_06_create_network_in_vpc	Error	24.68	26.62	test_escalations_networks.py
test_11_update_vpc	Failure	0.03	0.04	test_escalations_networks.py
test_19_create_list_reset_delete_vpnconnections	Error	26.88	27.82	test_escalations_networks.py
test_02_list_volume_byid	Failure	63.61	73.72	test_escalations_volumes.py
test_05_volume_snapshot	Error	71.86	82.37	test_escalations_volumes.py
test_01_list_vpncustomergateways_pagination	Error	0.07	1.07	test_escalations_vpncustomergateways.py
test_02_update_vpncustomergateways	Error	0.07	0.05	test_escalations_vpncustomergateways.py
test_iptable_rules	Error	0.03	0.06	test_multiple_public_interfaces.py
test_routing_tables	Error	0.03	0.02	test_multiple_public_interfaces.py
test_static_nat_on_ip_from_non_src_nat_ip_range	Error	0.02	0.03	test_multiple_public_interfaces.py
test_network_services_VPC_CreatePF	Error	65.66	76.73	test_multiple_public_interfaces.py
test_network_services_VPC_CreatePF	Error	65.59	76.76	test_multiple_public_interfaces.py
test_12_disableZone_admin_deployVM	Error	1.17	1.15	test_organization_states.py
test_13_disableZone_admin_stop_startVM	Error	40.08	38.89	test_organization_states.py
test_15_disableZone_user_stop_startVM	Error	38.15	40.31	test_organization_states.py
test_22_disablePod_admin_deployVM	Error	1.17	1.15	test_organization_states.py
test_23_disablePod_admin_stop_startVM	Error	4.20	3.17	test_organization_states.py
test_25_disablePod_user_stop_startVM	Error	3.21	4.21	test_organization_states.py
test_33_disableCluster_admin_stop_startVM	Error	4.20	3.19	test_organization_states.py
test_35_disableCluster_user_stop_startVM	Error	3.22	3.21	test_organization_states.py
test_36_enableCluster	Failure	0.01	0.02	test_organization_states.py
test_43_disableHost_admin_deployVM	Failure	1.13	1.11	test_organization_states.py
test_44_disableHost_admin_stop_startVM	Failure	2.16	4.19	test_organization_states.py
test_delete_account	Failure	112.07	108.16	test_persistent_networks.py
test_vpc_delete_account	Failure	1408.23	1483.98	test_persistent_networks.py
test_associate_ip_address_services_enable_disable	Failure	1322.77	1327.87	test_portable_ip.py
test_list_portable_ip_range_non_root_admin	Failure	1253.16	1253.91	test_portable_ip.py
test_restart_network_with_destroyed_masterVR	Failure	94.07	103.10	test_redundant_router_cleanups.py
test_01_reset_keypair_normal_user	Failure	827.64	832.27	test_reset_ssh_keypair.py
test_02_reset_keypair_domain_admin	Failure	815.36	826.25	test_reset_ssh_keypair.py
test_03_reset_keypair_root_admin	Failure	817.26	826.21	test_reset_ssh_keypair.py
test_01_reset_ssh_keys	Failure	795.34	799.42	test_reset_ssh_keypair.py
test_02_reset_ssh_key_password_enabled_template	Failure	808.71	793.20	test_reset_ssh_keypair.py
test_03_reset_ssh_with_no_key	Failure	791.01	790.27	test_reset_ssh_keypair.py
test_04_reset_key_passwd_enabled_no_key	Failure	794.77	792.19	test_reset_ssh_keypair.py
test_03_vmsnapshot__on_resized_rootvolume_vm	Error	105.56	118.83	test_rootvolume_resize.py
test_7_usage_events_after_rootvolume_resized_	Error	135.87	145.96	test_rootvolume_resize.py
test_acquire_ip	Failure	95.94	170.11	test_shared_networks.py
test_01_volume_from_snapshot	Failure	201.92	299.55	test_snapshots.py
test_01_register_template_1_root_domain_admin	Failure	147.07	159.59	test_ss_limits.py
test_01_register_template_2_child_domain_admin	Failure	146.97	150.10	test_ss_limits.py
test_02_create_template_snapshot_1_root_domain_admin	Failure	321.82	345.17	test_ss_limits.py
test_02_create_template_snapshot_2_child_domain_admin	Failure	321.90	341.98	test_ss_limits.py
test_03_register_iso_1_root_domain_admin	Failure	60.69	61.60	test_ss_limits.py
test_03_register_iso_2_child_domain_admin	Failure	60.60	62.72	test_ss_limits.py
test_01_deploy_vm_domain_limit_reached	Failure	28.00	41.28	test_ss_max_limits.py
test_02_deploy_vm_account_limit_reached	Failure	28.04	31.10	test_ss_max_limits.py
test_01_register_template_with_project	Failure	27.86	32.08	test_ss_project_limits.py
test_02_register_iso	Failure	63.17	67.27	test_ss_project_limits.py
test_09_stop_vm_migrate_vol	Error	1.65	7.74	test_stopped_vm.py
test_02_migrate_volume	Error	1.15	1.12	test_storage_motion.py
test_07_iso_tag	Failure	0.46	0.19	test_tags.py
test_16_query_tags_other_account	Failure	1.44	3.06	test_tags.py
test_17_query_tags_admin_account	Failure	1.02	1.57	test_tags.py
test_23_list_untagged_host_for_vm_migration	Error	1.63	1.27	test_tags.py
test_24_public_ip_tag	Error	2.00	9.12	test_tags.py
test_01_migrateVolume	Error	4.26	3.26	test_volumes.py
test_02_internallb_rules_traffic	Failure	430.55	438.38	test_vpc_network_internal_lbrules.py
test_03_internallb_rules_vpc_network_restarts_traffic	Failure	235.32	260.83	test_vpc_network_internal_lbrules.py
test_08_ip_reallocation_CS5986	Error	1.49	2.53	test_vpc_vms_deployment.py
test_01_VPN_service	Error	3.73	4.67	test_vpn_service.py
test_01_VPN_user_limit	Error	4.70	4.60	test_vpn_users.py
test_02_use_vpn_port	Error	3.62	5.72	test_vpn_users.py
test_03_enable_vpn_use_port	Error	3.77	3.64	test_vpn_users.py
test_04_add_new_users	Error	4.76	4.60	test_vpn_users.py
test_05_add_duplicate_user	Error	5.70	5.65	test_vpn_users.py
test_06_add_VPN_user_global_admin	Error	3.65	3.58	test_vpn_users.py
test_07_add_VPN_user_domain_admin	Error	4.68	4.59	test_vpn_users.py
test_08_add_TCP_PF_Rule_In_VPN	Error	3.62	3.66	test_vpn_users.py
test_deployVM_verify_metadata_in_VR	Error	4.62	5.56	test_vr_metadata.py
test_deployVM_verify_metadata_in_VR	Error	5.70	5.57	test_vr_metadata.py
test_deployVM_verify_metadata_in_VR_after_migration	Error	4.57	5.55	test_vr_metadata.py
test_deployVM_verify_metadata_in_VR_after_migration	Error	5.63	5.55	test_vr_metadata.py
test_CLOUDSTACK_6181_stoppedvm_root_resize	Error	6.65	14.65	test_bugs.py
test_CLOUDSTACK_6181_vm_root_resize	Error	4.25	4.20	test_bugs.py
test_es_1863_register_template_s3_domain_admin_user	Error	0.68	3.60	test_bugs.py
test_01_dedicate_guest_vlan_range_root_domain	Error	2.29	5.23	test_dedicate_guest_vlan_ranges.py
test_02_dedicate_guest_vlan_range_user_domain	Error	2.29	6.30	test_dedicate_guest_vlan_ranges.py
test_03_multiple_guest_netwoks	Error	1.73	4.80	test_dedicate_guest_vlan_ranges.py
test_02_delete_dedicated_vlan_range_vlan_in_use	Error	1.71	3.75	test_dedicate_guest_vlan_ranges.py
test_03_delete_account	Error	1.67	3.67	test_dedicate_guest_vlan_ranges.py
test_05_release_range_vlan_in_use	Error	1.77	3.69	test_dedicate_guest_vlan_ranges.py
test_05_acquire_ip_single_guest_network	Error	3.12	3.97	test_dedicate_public_ip_range.py
test_06_acquire_ip_multiple_guest_network	Error	3.10	4.09	test_dedicate_public_ip_range.py
test_08_acquire_ip_in_vpc	Error	1.72	3.77	test_dedicate_public_ip_range.py
test_14_release_ip_from_dedicated_range	Error	0.73	1.73	test_dedicate_public_ip_range.py
test_16_delete_ip_range_no_network_rules	Error	0.75	1.72	test_dedicate_public_ip_range.py
test_17_delete_ip_range_network_rules	Error	0.74	1.84	test_dedicate_public_ip_range.py
test_18_delete_network	Error	0.68	2.68	test_dedicate_public_ip_range.py
test_01_op_host_capacity_disable_cluster	Failure	0.15	0.05	test_escalations_hosts.py
test_04_disable_host_unmanage_cluster_check_hosts_status	Error	200.21	200.20	test_escalations_hosts.py
test_01_CS40139_listtemplate_with_different_pagesize	Error	1.06	1.05	test_escalation_templates.py
test_02_template_permissions	Error	99.80	95.70	test_escalation_templates.py
test_02_template_permissions	Error	109.0	895.70	test_escalation_templates.py
ContextSuite context=TestlistTemplates>:teardown	Error	111.2	497.95	test_escalation_templates.py
test_01_deploy_vm_with_explicit_dedication	Error	2.71	2.73	test_explicit_dedication.py
test_ha_with_storage_maintenance	Error	1.70	1.64	test_ha_pool_maintenance.py
test_ha_with_storage_maintenance	Error	1.70	1.64	test_ha_pool_maintenance.py
test_01_host_maintenance_mode	Error	5.80	12.16	test_high_availability.py
test_02_host_maintenance_mode_with_activities	Error	4.73	10.85	test_high_availability.py
test_01_ip_resouce_count_vpc_network	Error	3.85	4.71	test_ip_resource_count_for_vpc.py
test_01_deploy_vms_in_network	Error	2.63	4.64	test_vpc_vms_deployment.py
test_02_deploy_vms_delete_network	Error	2.64	2.84	test_vpc_vms_deployment.py
test_03_deploy_vms_delete_add_network	Error	2.64	4.99	test_vpc_vms_deployment.py
test_04_deploy_vms_delete_add_network_noLb	Error	2.67	4.66	test_vpc_vms_deployment.py
test_05_create_network_max_limit	Error	2.75	4.66	test_vpc_vms_deployment.py
test_06_delete_network_vm_running	Error	1.75	4.70	test_vpc_vms_deployment.py
test_RvR_multicluster	Error	3.94	11.05	test_redundant_router_deployment_planning.py
test_RvR_multihosts	Error	1.86	7.95	test_redundant_router_deployment_planning.py
test_RvR_multiprimarystorage	Error	1.77	7.87	test_redundant_router_deployment_planning.py
test_applyNetworkRules_MasterDown_deleteNetworkRules	Error	4.91	10.90	test_redundant_router_network_rules.py
test_applyRules_restartRvRNetwork	Error	4.78	9.83	test_redundant_router_network_rules.py
test_apply_and__delete_NetworkRulesOnRvR	Error	4.85	9.90	test_redundant_router_network_rules.py
test_networkRules_afterRebootRouters	Error	4.72	9.84	test_redundant_router_network_rules.py
test_createRvRNetwork	Error	5.02	9.94	test_redundant_router.py
test_createRvRNetwork	Error	3.82	9.91	test_redundant_router.py
test_redundantVR_internals	Error	4.84	10.90	test_redundant_router.py
test_01_stopMasterRvR	Error	5.83	3.70	test_redundant_router.py
test_02_stopBackupRvR	Error	5.78	4.65	test_redundant_router.py
test_03_rebootMasterRvR	Error	3.68	3.62	test_redundant_router.py
test_04_rebootBackupRvR	Error	3.67	3.66	test_redundant_router.py
test_05_stopBackupRvR_startInstance	Error	4.89	4.64	test_redundant_router.py
ContextSuite context=TestRvRRedundancy>:teardown	Error	5.06	4.75	test_redundant_router.py

exact matches:

Test	Result (py2/py3)	Time (py2/py3)	Test File
ContextSuite context=TestDeployVMs>:setup	Error	0.00	test_dhcp_dns_offload.py
ContextSuite context=TestBrowseUploadVolume>:setup	Error	0.00	test_browse_volumes.py
test_05_list_network_offerings_with_and_without_vpc	Error	0.15	test_escalations_networks.py
test_07_create_delete_network	Error	0.02	test_escalations_networks.py
test_08_update_network	Error	0.02	test_escalations_networks.py
test_10_list_networks_in_vpc	Failure	0.04	test_escalations_networks.py
test_12_list_create_delete_networkACL	Failure	0.03	test_escalations_networks.py
test_port_forwarding_on_ip_from_non_src_nat_ip_range	Error	0.02	test_multiple_public_interfaces.py
test_31_disableCluster	Failure	0.02	test_organization_states.py
test_32_disableCluster_admin_deployVM	Error	1.13	test_organization_states.py
test_42_disableHost_admin_deployVM	Error	0.02	test_organization_states.py
test_46_disableHost_user_stop_startVM	Failure	3.20	test_organization_states.py
ContextSuite context=TestVPCRouterOneNetwork>:setup	Error	0.00	test_vpc_routers.py
ContextSuite context=TestVPCRoutersBasic>:setup	Error	0.00	test_vpc_routers.py
ContextSuite context=TestVMLifeCycleBothIsolated>:setup	Error	0.00	test_vpc_vm_life_cycle.py
ContextSuite context=TestVMLifeCycleDiffHosts>:setup	Error	0.00	test_vpc_vm_life_cycle.py
ContextSuite context=TestVMLifeCycleSharedNwVPC>:setup	Error	0.00	test_vpc_vm_life_cycle.py
ContextSuite context=TestVMLifeCycleStoppedVPCVR>:setup	Error	0.00	test_vpc_vm_life_cycle.py
ContextSuite context=TestVMLifeCycleVPC>:setup	Error	0.00	test_vpc_vm_life_cycle.py
ContextSuite context=TestVMLifeCycleHostmaintenance>:setup	Error	0.00	test_vpc_host_maintenance.py
test_07_delete_network_with_rules	Error	3.70	test_vpc_vms_deployment.py
ContextSuite context=TestSystemVmLocalStorage>:setup	Error	0.00	test_zone_level_local_storage_setting.py

only in python2:

Test	Result	Time	Test File
test_browser_upload_template_incomplete	Failure	304.53	test_browse_templates2.py
ContextSuite context=TestvGPUWindowsVm>:setup	Error	0.00	test_deploy_vgpu_vm.py
test_10_volume_snapshots_pagination	Error	73.08	test_escalations_volumes.py
test_07_template_from_snapshot	Error	234.40	test_snapshots.py
test_04_copy_template_1_root_domain_admin	Failure	26.79	test_ss_limits.py
test_04_copy_template_2_child_domain_admin	Failure	26.88	test_ss_limits.py
test_20_release_range_not_dedicated	Error	0.22	test_dedicate_public_ip_range.py

[DaanHoogland]

reconsiddering (and re-running) configdrive tests on py2 and py3 I get

• py2:
  Test | Result | Time (s) | Test File
  --- | --- | --- | ---
  test_configdrive_isolated_network | Failure | 0.99 | test_configdrive.py
  test_configdrive_isolated_network_hypervisor_hostname_exposed | Error | 3.48 | test_configdrive.py
  test_configdrive_shared_network | Failure | 2.17 | test_configdrive.py
  test_configdrive_vpc_network | Failure | 6.96 | test_configdrive.py

• py3
  Test | Result | Time (s) | Test File
  --- | --- | --- | ---
  test_configdrive_isolated_network | Failure | 1.73 | test_configdrive.py
  test_configdrive_isolated_network_hypervisor_hostname_exposed | Error | 6.95 | test_configdrive.py
  test_configdrive_shared_network | Failure | 5.17 | test_configdrive.py
  test_configdrive_vpc_network | Failure | 17.17 | test_configdrive.py
  test_configdrive_vpc_network_verify_metadata | Failure | 9.70 | test_configdrive.py

these are too similar to give it priority now (@rhtyd)

[DaanHoogland]

test_egress_fw_rules.py in a new env shows the new failure succeeding, skipping this test (for now):

Test Create Egress rule without CIDR ... === TestName: test_06_egress_fr6 | Status : SUCCESS ===
ok

the difference in time in the first comparison is now an exception in py2 and still a failure in py3:

Test egress rule with /32 CIDR of a VM, and check other VM in the ... === TestName: test_01_2_egress_fr1 | Status : EXCEPTION ===
ERROR

vs

Test egress rule with /32 CIDR of a VM, and check other VM in the ... === TestName: test_01_2_egress_fr1 | Status : FAILED ===
FAIL

addendum; re-test succeeds:

Test egress rule with /32 CIDR of a VM, and check other VM in the ... === TestName: test_01_2_egress_fr1 | Status : SUCCESS ===
ok

[DaanHoogland]

multiple ips per nic:

python2:

Add secondary IP to NIC of a VM ... === TestName: test_add_ip_to_nic_1_SHARED | Status : SUCCESS ===
ok
Test listing nics associated with the ip address ... === TestName: test_list_nics_1_ISOLATED | Status : SUCCESS ===
ok
Test listing nics associated with the ip address ... === TestName: test_list_nics_2_SHARED | Status : SUCCESS ===
ok
Test listing nics associated with the ip address ... === TestName: test_list_nics_3_VPC | Status : SUCCESS ===
ok
Test basic operations using non root admin apii client ... === TestName: test_operations_non_root_admin_api_client_1_ISOLATED | Status : SUCCESS ===
ok
Test basic operations using non root admin apii client ... === TestName: test_operations_non_root_admin_api_client_2_SHARED | Status : SUCCESS ===
ok
Test basic operations using non root admin apii client ... === TestName: test_operations_non_root_admin_api_client_3_VPC | Status : SUCCESS ===
ok
Remove invalid ip ... === TestName: test_remove_invalid_ip | Status : SUCCESS ===
ok
Remove secondary IP from NIC of a VM ... === TestName: test_remove_ip_from_nic_1_ISOLATED | Status : SUCCESS ===
ok
Remove secondary IP from NIC of a VM ... === TestName: test_remove_ip_from_nic_2_SHARED | Status : SUCCESS ===
ok
Remove secondary IP from NIC of a VM ... === TestName: test_remove_ip_from_nic_3_VPC | Status : EXCEPTION ===
ERROR
=== TestName: test_remove_ip_from_nic_3_VPC | Status : EXCEPTION ===
ERROR
=== TestName: test_remove_ip_from_nic_3_VPC | Status : EXCEPTION ===
ERROR
=== TestName: test_remove_ip_from_nic_3_VPC | Status : EXCEPTION ===
ERROR

and py3

Add secondary IP to NIC of a VM ... === TestName: test_add_ip_to_nic_1_SHARED | Status : SUCCESS ===
ok
Test listing nics associated with the ip address ... === TestName: test_list_nics_1_ISOLATED | Status : SUCCESS ===
ok
Test listing nics associated with the ip address ... === TestName: test_list_nics_2_SHARED | Status : SUCCESS ===
ok
Test listing nics associated with the ip address ... === TestName: test_list_nics_3_VPC | Status : SUCCESS ===
ok
Test basic operations using non root admin apii client ... === TestName: test_operations_non_root_admin_api_client_1_ISOLATED | Status : SUCCESS ===
ok
Test basic operations using non root admin apii client ... === TestName: test_operations_non_root_admin_api_client_2_SHARED | Status : SUCCESS ===
ok
Test basic operations using non root admin apii client ... === TestName: test_operations_non_root_admin_api_client_3_VPC | Status : SUCCESS ===
ok
Remove invalid ip ... === TestName: test_remove_invalid_ip | Status : SUCCESS ===
ok
Remove secondary IP from NIC of a VM ... === TestName: test_remove_ip_from_nic_1_ISOLATED | Status : SUCCESS ===
ok
Remove secondary IP from NIC of a VM ... === TestName: test_remove_ip_from_nic_2_SHARED | Status : SUCCESS ===
ok
Remove secondary IP from NIC of a VM ... === TestName: test_remove_ip_from_nic_3_VPC | Status : SUCCESS ===
ok
Add secondary IP to NIC of a VM ... === TestName: test_add_PF_rule_1_ISOLATED | Status : FAILED ===
FAIL
Add secondary IP to NIC of a VM ... === TestName: test_add_PF_rule_2_SHARED | Status : FAILED ===
FAIL
Add secondary IP to NIC of a VM ... === TestName: test_add_PF_rule_3_VPC | Status : FAILED ===
FAIL
Add secondary IP to NIC of a VM ... === TestName: test_add_static_nat_rule_1_ISOLATED | Status : FAILED ===
FAIL
=== TestName: test_add_static_nat_rule_1_ISOLATED | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_add_static_nat_rule_2_SHARED | Status : EXCEPTION ===
ERROR
=== TestName: test_add_static_nat_rule_2_SHARED | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_add_static_nat_rule_3_VPC | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_delete_PF_nat_rule_1_ISOLATED | Status : EXCEPTION ===
ERROR
=== TestName: test_delete_PF_nat_rule_1_ISOLATED | Status : EXCEPTION ===
Add secondary IP to NIC of a VM ... === TestName: test_delete_PF_nat_rule_2_SHARED | Status : EXCEPTION ===
ERROR
=== TestName: test_delete_PF_nat_rule_2_SHARED | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_delete_PF_nat_rule_3_VPC | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_disable_static_nat_1_ISOLATED | Status : EXCEPTION ===
ERROR
=== TestName: test_disable_static_nat_1_ISOLATED | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_disable_static_nat_2_SHARED | Status : EXCEPTION ===
ERROR
=== TestName: test_disable_static_nat_2_SHARED | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_disable_static_nat_3_VPC | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_disassociate_ip_mapped_to_secondary_ip_through_PF_rule_1_ISOLATED | Status : EXCEPTION ===
ERROR
=== TestName: test_disassociate_ip_mapped_to_secondary_ip_through_PF_rule_1_ISOLATED | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_disassociate_ip_mapped_to_secondary_ip_through_PF_rule_2_SHARED | Status : EXCEPTION ===
ERROR
=== TestName: test_disassociate_ip_mapped_to_secondary_ip_through_PF_rule_2_SHARED | Status : EXCEPTION ===
ERROR
Add secondary IP to NIC of a VM ... === TestName: test_disassociate_ip_mapped_to_secondary_ip_through_PF_rule_3_VPC | Status : EXCEPTION ===
ERROR
=== TestName: test_disassociate_ip_mapped_to_secondary_ip_through_PF_rule_3_VPC | Status : EXCEPTION ===
ERROR
Test delete VM and verify network rules are cleaned up ... === TestName: test_delete_vm_1_ISOLATED | Status : EXCEPTION ===
ERROR
=== TestName: test_delete_vm_1_ISOLATED | Status : EXCEPTION ===
ERROR
Test delete VM and verify network rules are cleaned up ... === TestName: test_delete_vm_2_SHARED | Status : EXCEPTION ===
ERROR
Test delete VM and verify network rules are cleaned up ... === TestName: test_delete_vm_3_VPC | Status : EXCEPTION ===
ERROR
Test network restart (cleanup True) with VM having secondary IPs and related network rules ... === TestName: test_network_restart_cleanup_false_1_ISOLATED | Status : EXCEPTION ===
ERROR
=== TestName: test_network_restart_cleanup_false_1_ISOLATED | Status : EXCEPTION ===
ERROR
Test network restart (cleanup True) with VM having secondary IPs and related network rules ... === TestName: test_network_restart_cleanup_false_2_SHARED | Status : EXCEPTION ===
ERROR
Test network restart (cleanup True) with VM having secondary IPs and related network rules ... === TestName: test_network_restart_cleanup_false_3_VPC | Status : EXCEPTION ===
ERROR
Test network restart (cleanup True) with VM having secondary IPs and related network rules ... === TestName: test_network_restart_cleanup_true_1_ISOLATED | Status : EXCEPTION ===
ERROR
=== TestName: test_network_restart_cleanup_true_1_ISOLATED | Status : EXCEPTION ===
ERROR
Test network restart (cleanup True) with VM having secondary IPs and related network rules ... === TestName: test_network_restart_cleanup_true_2_SHARED | Status : EXCEPTION ===
ERROR
Test network restart (cleanup True) with VM having secondary IPs and related network rules ... === TestName: test_network_restart_cleanup_true_3_VPC | Status : EXCEPTION ===
ERROR
Test reboot router and persistence of network rules ... === TestName: test_reboot_router_VM_1_ISOLATED | Status : EXCEPTION ===
ERROR
=== TestName: test_reboot_router_VM_1_ISOLATED | Status : EXCEPTION ===
ERROR
Test reboot router and persistence of network rules ... === TestName: test_reboot_router_VM_2_SHARED | Status : EXCEPTION ===
ERROR
Test reboot router and persistence of network rules ... === TestName: test_reboot_router_VM_3_VPC | Status : EXCEPTION ===
ERROR
Test recover VM operation with VM having secondary IPs ... === TestName: test_recover_vm_1_ISOLATED | Status : EXCEPTION ===
ERROR
=== TestName: test_recover_vm_1_ISOLATED | Status : EXCEPTION ===
ERROR
Test recover VM operation with VM having secondary IPs ... === TestName: test_recover_vm_2_SHARED | Status : EXCEPTION ===
ERROR
Test recover VM operation with VM having secondary IPs ... === TestName: test_recover_vm_3_VPC | Status : EXCEPTION ===
ERROR

So in conclusion more successes in absolute numbers but much more are run and a lot of exceptions extra. We'll have to revisit this.

[DaanHoogland]

test_templates brought up to par; py2:

Test create VM, Snapshot and Template ... === TestName: test_01_createVM_snapshotTemplate | Status : SUCCESS ===
ok
Test snapshot events ... === TestName: test_05_snapshot_events | Status : SUCCESS ===
ok
Test Creating snapshot from volume having spaces in name(KVM) ... === TestName: test_01_volume_from_snapshot | Status : FAILED ===
FAIL
Test Snapshot Data Disk ... === TestName: test_02_snapshot_data_disk | Status : SUCCESS ===
ok
Test snapshot from detached disk ... === TestName: test_03_snapshot_detachedDisk | Status : SUCCESS ===
ok
Test Delete Snapshot ... === TestName: test_04_delete_snapshot | Status : SUCCESS ===
ok
Create Template from snapshot ... === TestName: test_07_template_from_snapshot | Status : EXCEPTION ===
ERROR

and py3:

Test create VM, Snapshot and Template ... === TestName: test_01_createVM_snapshotTemplate | Status : SUCCESS ===
ok
Test snapshot events ... === TestName: test_05_snapshot_events | Status : SUCCESS ===
ok
Test Creating snapshot from volume having spaces in name(KVM) ... === TestName: test_01_volume_from_snapshot | Status : FAILED ===
FAIL
Test Snapshot Data Disk ... === TestName: test_02_snapshot_data_disk | Status : SUCCESS ===
ok
Test snapshot from detached disk ... === TestName: test_03_snapshot_detachedDisk | Status : SUCCESS ===
ok
Test Delete Snapshot ... === TestName: test_04_delete_snapshot | Status : SUCCESS ===
ok
Create Template from snapshot ... === TestName: test_07_template_from_snapshot | Status : SUCCESS ===
ok

The failure is strange and I am not sure but I think it is in the test, have spent too much time on this one for now, and the goal is being on par with py2 for now.

[DaanHoogland]

the test_vpc_* failures show a failure to ssh into a vm or router or do a wget that fails.
in python2 the wget gives no error but this seems a false positive. When removing the leanup from the tests the remaining envs do not allow for wget and on the marvin box a file named test.html remains with the contents,

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

, indicating that the wget indeed failed even though the test reports success. I'm going to rebase the full test suites for both python2 and python3 and re-run the full comparison.

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 673

[DaanHoogland]

@GabrielBrascher @rhtyd I am running a new comparison between the py2 and py3 tests now, to verify we are at least at par. will let you know.
can you please review?
thanks

[yadvr]

Will review soon, but +1 great work @DaanHoogland we've always had components test but never run them. With your work, we finally wil have verified list of tests/cases that we expect to work which will further help with running such component tests on weekly basis to review/monitor branch health.

I suppose there are still some work left but I would want to get something merged soon that could help 4.16, for example:

• limit scope of the remaining component tests and fix that limited scope of tests, such that they pass for kvm/vmware/xenserver at least with adv zone (preferably the case of adv zone + SG too)
• move all failing tests that can't be fixed (takes another couple of years :) ) with a different attribute to be fixed in future
• add/update component tests that missing to Travis
• integrate fix BO to run component tests (that are known to pass), something like @bo test componentmatrix to kick component only tests across kvm/vmware/xenserver?

[DaanHoogland]

• limit scope of the remaining component tests and fix that limited scope of tests, such that they pass for kvm/vmware/xenserver at least with adv zone (preferably the case of adv zone + SG too)

this scope is not yet clear and I would like to do that in a future PR.

• move all failing tests that can't be fixed (takes another couple of years :) ) with a different attribute to be fixed in future

good idea, will do

• add/update component tests that missing to Travis

Travis is already taking a long time (and we are probably not like by travis or the rest of apache) Let's discuss this

• integrate fix BO to run component tests (that are known to pass), something like @bo test componentmatrix to kick component only tests across kvm/vmware/xenserver?

yes, but this also seems a separate PR. (and internal to BO mostly, not ACS)

thanks for the suggestions @rhtyd

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 752

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 1061

[yadvr]

LGTM - I'm +1 to get the first pass of component tests merged that show siginificant progress in which ones can we run now (and they pass). The next steps:

1. check/update these against Travis yml i.e. check and include any tests which aren't already added to the list which pass in Travis
2. Figure out a subset that can be worked upon as phase2

[DaanHoogland]

smoke tests and java code have not changed, but still:
@blueorangutan test matrix

[blueorangutan]

@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

Trillian Build Failed (tid-1860)

[blueorangutan]

Trillian Build Failed (tid-1862)

[blueorangutan]

Trillian Build Failed (tid-1861)

[DaanHoogland]

@blueorangutan test matrix keepEnv

[blueorangutan]

@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

Trillian Build Failed (tid-1866)

[blueorangutan]

Trillian Build Failed (tid-1864)

[blueorangutan]

Trillian Build Failed (tid-1865)

[nvazquez]

@blueorangutan test matrix keepEnv

[blueorangutan]

@nvazquez a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-1873)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 52415 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5082-t1873-xenserver-71.zip
Smoke tests completed. 90 look OK, 3 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_deploy_vm_start_failure	Error	99.97	test_deploy_vm.py
test_deploy_vm_volume_creation_failure	Error	87.69	test_deploy_vm.py
test_vm_ha	Error	86.81	test_vm_ha.py
test_vm_sync	Error	171.74	test_vm_sync.py

[blueorangutan]

Trillian test result (tid-1874)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 54761 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5082-t1874-kvm-centos7.zip
Smoke tests completed. 90 look OK, 3 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_deploy_vm_start_failure	Error	61.25	test_deploy_vm.py
test_deploy_vm_volume_creation_failure	Error	79.68	test_deploy_vm.py
test_vm_ha	Error	67.11	test_vm_ha.py
test_vm_sync	Error	132.83	test_vm_sync.py

[blueorangutan]

Trillian test result (tid-1875)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 55486 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5082-t1875-vmware-65u2.zip
Smoke tests completed. 90 look OK, 3 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_deploy_vm_start_failure	Error	96.04	test_deploy_vm.py
test_deploy_vm_volume_creation_failure	Error	116.33	test_deploy_vm.py
test_vm_ha	Error	112.64	test_vm_ha.py
test_vm_sync	Error	182.41	test_vm_sync.py

[GabrielBrascher]

LGTM based on the CI test results and overall code review.
Agree with the idea of merging how it is and work incrementally on new PR(s).

[yadvr]

LGTM as Gabriel and others have said let's merge this. We don't normally run component tests, but getting incremental PRs merged is a good approach. Smoketests on this have passed (the errors are not related to this PR, these are tests that shouldn't run).

[yadvr]

@DaanHoogland pl create another PR to continue any further work. Thanks.