Skip to content

CLOUDSTACK-10102: New network type (L2) - #2281

Merged
yadvr merged 5 commits into
apache:masterfrom
shapeblue:roadmap-newl2networktype
Dec 20, 2017
Merged

CLOUDSTACK-10102: New network type (L2)#2281
yadvr merged 5 commits into
apache:masterfrom
shapeblue:roadmap-newl2networktype

Conversation

@nvazquez

Copy link
Copy Markdown
Contributor

JIRA Ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-10102
Feature Specification: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74680920

Introduction

This feature allows CloudStack administrators to create layer 2 networks on CloudStack. As these networks are purely layer 2, they don't require IP addresses or Virtual Router, only VLAN is necessary (provided by administrator or assigned by CloudStack). Also, network services should be handled externally, e.g. DNS, DHCP, as they are not provided by L2 networks.
As a consequence, a new Guest Network type is created within CloudStack: L2

Description

  • Network offerings and networks support new guest type: L2.
  • L2 Network offering creation allows administrator to select Specify VLAN or let CloudStack assign it dynamically.
  • L2 Network creation allows administrator to specify VLAN tag (if network offerings allows it) or simply create network.
  • VM deployments on L2 networks:
    • VMs should not IP addresses or any network service
    • No Virtual Router deployed on network
    • If Specify VLAN = true for network offering, network gets implemented using a dynamically assigned VLAN

UI changes

A new button is added on Networks tab, available for admins, to allow L2 networks creation

@yadvr

yadvr commented Oct 6, 2017

Copy link
Copy Markdown
Member

Did not test it, but good work @nvazquez
@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-1135

@nvazquez
nvazquezforce-pushed the roadmap-newl2networktype branch from 3a9fb97 to df47971CompareOctober 9, 2017 12:22
@borisstoyanov

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-1573)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 29725 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2281-t1573-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermitten failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermitten failure detected: /marvin/tests/smoke/test_network.py
Intermitten failure detected: /marvin/tests/smoke/test_password_server.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dns.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermitten failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermitten failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermitten failure detected: /marvin/tests/smoke/test_routers.py
Intermitten failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_ssvm.py
Intermitten failure detected: /marvin/tests/smoke/test_templates.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_volumes.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Test completed. 41 look OK, 21 have error(s)

TestResultTime (s)Test File
test_01_vpc_site2site_vpnFailure15.40test_vpc_vpn.py
test_01_vpc_remote_access_vpnFailure5.22test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpnFailure10.65test_vpc_vpn.py
test_04_rvpc_privategw_static_routesFailure10.95test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanupFailure10.92test_privategw_acl.py
test_02_vpc_privategw_static_routesFailure10.94test_privategw_acl.py
test_01_vpc_privategw_aclFailure11.02test_privategw_acl.py
test_02_port_fwd_on_non_src_natFailure737.91test_network.py
test_02_VPC_default_routesError10.96test_vpc_router_nics.py
test_01_VPC_nics_after_destroyError10.94test_vpc_router_nics.py
test_05_rvpc_multi_tiersError10.96test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nicsError10.99test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routersError5.93test_vpc_redundant.py
test_02_redundant_VPC_default_routesError10.96test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACLError15.97test_vpc_redundant.py
ContextSuite context=TestVolumes>:setupError0.00test_volumes.py
ContextSuite context=TestCreateVolume>:setupError0.00test_volumes.py
ContextSuite context=TestVmSnapshot>:setupError6.03test_vm_snapshots.py
ContextSuite context=TestVMLifeCycle>:setupError0.00test_vm_life_cycle.py
ContextSuite context=TestDeployVM>:setupError0.00test_vm_life_cycle.py
ContextSuite context=TestTemplates>:setupError0.00test_templates.py
ContextSuite context=TestSnapshotRootDisk>:setupError0.00test_snapshots.py
ContextSuite context=TestServiceOfferings>:setupError6.24test_service_offerings.py
test_03_RVR_Network_check_router_stateError0.28test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_falseError5.44test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_falseError0.27test_routers_network_ops.py
test_01_isolate_network_FW_PF_default_routes_egress_trueError5.46test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_trueError0.27test_routers_network_ops.py
test_02_routervm_iptables_policiesError5.32test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policiesError5.28test_routers_iptables_default_policy.py
ContextSuite context=TestRouterServices>:setupError0.00test_routers.py
ContextSuite context=TestRouterDnsService>:setupError0.00test_router_dnsservice.py
ContextSuite context=TestRouterDns>:setupError0.00test_router_dns.py
ContextSuite context=TestRouterDHCPOpts>:setupError0.00test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPHosts>:setupError0.00test_router_dhcphosts.py
ContextSuite context=TestResetVmOnReboot>:setupError0.00test_reset_vm_on_reboot.py
test_isolate_network_password_serverError0.09test_password_server.py
test_releaseIPError11.08test_network.py
test_reboot_routerError242.14test_network.py
test_delete_accountError86.57test_network.py
test_02_port_fwd_on_non_src_natError743.02test_network.py
test_01_port_fwd_on_src_natError0.09test_network.py
ContextSuite context=TestRouterRules>:setupError17.09test_network.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfacesError242.74test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfacesError147.06test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80Error223.14test_internal_lb.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80Error142.86test_internal_lb.py
test_06_verify_guest_lspci_againError721.06test_deploy_virtio_scsi_vm.py
test_04_verify_guest_lspciError715.51test_deploy_virtio_scsi_vm.py
test_change_service_offering_for_vm_with_snapshotsSkipped0.00test_vm_snapshots.py
ContextSuite context=TestCreateTemplate>:setupSkipped0.00test_templates.py
ContextSuite context=TestCopyDeleteTemplate>:setupSkipped0.00test_templates.py
test_static_role_account_aclsSkipped0.03test_staticroles.py
test_11_ss_nfs_version_on_ssvmSkipped0.02test_ssvm.py
test_01_scale_vmSkipped0.00test_scale_vm.py
test_01_primary_storage_iscsiSkipped0.10test_primary_storage.py
test_vm_nic_adapter_vmxnet3Skipped0.00test_nic_adapter_type.py
test_nested_virtualization_vmwareSkipped0.00test_nested_virtualization.py
test_06_copy_isoSkipped0.00test_iso.py
test_list_ha_for_host_validSkipped0.02test_hostha_simulator.py
test_list_ha_for_host_invalidSkipped0.02test_hostha_simulator.py
test_list_ha_for_hostSkipped0.02test_hostha_simulator.py
test_hostha_enable_feature_without_setting_providerSkipped0.02test_hostha_simulator.py
test_hostha_enable_feature_validSkipped0.03test_hostha_simulator.py
test_hostha_disable_feature_validSkipped0.03test_hostha_simulator.py
test_hostha_configure_invalid_providerSkipped0.02test_hostha_simulator.py
test_hostha_configure_default_driverSkipped0.03test_hostha_simulator.py
test_ha_verify_fsm_recoveringSkipped0.02test_hostha_simulator.py
test_ha_verify_fsm_fencedSkipped0.02test_hostha_simulator.py
test_ha_verify_fsm_degradedSkipped0.02test_hostha_simulator.py
test_ha_verify_fsm_availableSkipped0.03test_hostha_simulator.py
test_ha_multiple_mgmt_server_ownershipSkipped0.03test_hostha_simulator.py
test_ha_list_providersSkipped0.03test_hostha_simulator.py
test_ha_enable_feature_invalidSkipped0.03test_hostha_simulator.py
test_ha_disable_feature_invalidSkipped0.03test_hostha_simulator.py
test_ha_configure_enabledisable_across_clusterzonesSkipped0.02test_hostha_simulator.py
test_configure_ha_provider_validSkipped0.02test_hostha_simulator.py
test_configure_ha_provider_invalidSkipped0.03test_hostha_simulator.py
test_deploy_vgpu_enabled_vmSkipped0.03test_deploy_vgpu_enabled_vm.py
test_3d_gpu_supportSkipped0.04test_deploy_vgpu_enabled_vm.py

@yadvryadvr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall nice changes, I've not tested it though. Does it make sense to add some marvin test?

nicSearch.and().op("ip4Address", nicSearch.entity().getIPv4Address(), SearchCriteria.Op.NNULL);
nicSearch.or("ip6Address", nicSearch.entity().getIPv6Address(), SearchCriteria.Op.NNULL);
nicSearch.cp();
if (! network.getGuestType().equals(Network.GuestType.L2)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Space after ! may be removed

addr.setState(assign ? IpAddress.State.Allocated : IpAddress.State.Allocating);

if (vlanUse != VlanType.DirectAttached) {
if (assign && vlanUse != VlanType.DirectAttached) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this cause a backward compatibility issue?

}
}


Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove extra newline?

Comment threadui/scripts/configuration.js Outdated
$useVpc.css('display', 'inline-block');
$supportedServices.css('display', 'inline-block');
}
else if ($guestTypeField.val() == 'L2') {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put } else if ... on same line as line 2413.

Comment threadui/scripts/configuration.js Outdated
} else if ($(this).val() == "Isolated") {
$form.find('.form-item[rel=specifyVlan]').find('input[type=checkbox]').removeAttr("disabled"); //make it editable
$form.find('.form-item[rel=isPersistent]').find('input[type=checkbox]').removeAttr("disabled");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove extra new line

Comment threadui/scripts/configuration.js Outdated


} else { //$(this).val() == "Isolated"
} else if ($(this).val() == "Isolated") {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can both the cases for L2 and Isolated be collapsed, as both are using same code?

Comment threadui/scripts/configuration.js Outdated
@@ -3315,6 +3328,13 @@
delete inputData.isPersistent; //if Persistent checkbox is unchecked, do not pass isPersistent parameter to API call since we need to keep API call's size as small as possible (p.s. isPersistent is defaulted as false at server-side)
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put } else if.. on same line.

@borisstoyanov

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1170

@borisstoyanov

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-1595)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 17701 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2281-t1595-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermitten failure detected: /marvin/tests/smoke/test_host_annotations.py
Intermitten failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Intermitten failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermitten failure detected: /marvin/tests/smoke/test_network.py
Intermitten failure detected: /marvin/tests/smoke/test_password_server.py
Intermitten failure detected: /marvin/tests/smoke/test_primary_storage.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dns.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermitten failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermitten failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermitten failure detected: /marvin/tests/smoke/test_routers.py
Intermitten failure detected: /marvin/tests/smoke/test_secondary_storage.py
Intermitten failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_templates.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_volumes.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Test completed. 38 look OK, 25 have error(s)

TestResultTime (s)Test File
test_01_vpc_site2site_vpnFailure10.38test_vpc_vpn.py
test_01_vpc_remote_access_vpnFailure5.17test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpnFailure10.41test_vpc_vpn.py
test_01_sys_vm_startFailure0.07test_secondary_storage.py
test_04_rvpc_privategw_static_routesFailure5.95test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanupFailure10.93test_privategw_acl.py
test_02_vpc_privategw_static_routesFailure5.92test_privategw_acl.py
test_01_vpc_privategw_aclFailure5.93test_privategw_acl.py
test_02_VPC_default_routesError11.15test_vpc_router_nics.py
test_01_VPC_nics_after_destroyError10.93test_vpc_router_nics.py
test_05_rvpc_multi_tiersError11.07test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nicsError10.99test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routersError10.97test_vpc_redundant.py
test_02_redundant_VPC_default_routesError10.96test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACLError16.04test_vpc_redundant.py
ContextSuite context=TestVolumes>:setupError0.00test_volumes.py
ContextSuite context=TestCreateVolume>:setupError0.00test_volumes.py
ContextSuite context=TestVmSnapshot>:setupError6.01test_vm_snapshots.py
ContextSuite context=TestVMLifeCycle>:setupError0.00test_vm_life_cycle.py
ContextSuite context=TestDeployVM>:setupError0.00test_vm_life_cycle.py
ContextSuite context=TestTemplates>:setupError47.28test_templates.py
ContextSuite context=TestSnapshotRootDisk>:setupError0.00test_snapshots.py
ContextSuite context=TestServiceOfferings>:setupError6.18test_service_offerings.py
test_03_RVR_Network_check_router_stateError0.51test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_falseError5.38test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_falseError0.54test_routers_network_ops.py
test_01_isolate_network_FW_PF_default_routes_egress_trueError5.43test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_trueError0.43test_routers_network_ops.py
test_02_routervm_iptables_policiesError5.27test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policiesError15.33test_routers_iptables_default_policy.py
ContextSuite context=TestRouterServices>:setupError0.00test_routers.py
ContextSuite context=TestRouterDnsService>:setupError0.00test_router_dnsservice.py
ContextSuite context=TestRouterDns>:setupError0.00test_router_dns.py
ContextSuite context=TestRouterDHCPOpts>:setupError0.00test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPHosts>:setupError0.00test_router_dhcphosts.py
ContextSuite context=TestResetVmOnReboot>:setupError0.00test_reset_vm_on_reboot.py
test_01_primary_storage_nfsError0.16test_primary_storage.py
test_01_add_primary_storage_disabled_hostError42.00test_primary_storage.py
ContextSuite context=TestStorageTags>:setupError0.28test_primary_storage.py
test_isolate_network_password_serverError0.08test_password_server.py
test_releaseIPError6.07test_network.py
test_reboot_routerError166.53test_network.py
test_public_ip_user_accountError0.05test_network.py
test_public_ip_admin_accountError0.06test_network.py
test_delete_accountError41.30test_network.py
test_02_port_fwd_on_non_src_natError0.05test_network.py
test_01_port_fwd_on_src_natError0.08test_network.py
ContextSuite context=TestRouterRules>:setupError12.14test_network.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfacesError126.82test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfacesError91.78test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80Error132.70test_internal_lb.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80Error92.38test_internal_lb.py
test_hostha_enable_ha_when_host_in_maintenanceError20.92test_hostha_kvm.py
test_05_add_annotation_for_invvalid_entityTypeError0.05test_host_annotations.py
test_06_verify_guest_lspci_againError715.55test_deploy_virtio_scsi_vm.py
test_04_verify_guest_lspciError715.48test_deploy_virtio_scsi_vm.py
test_change_service_offering_for_vm_with_snapshotsSkipped0.00test_vm_snapshots.py
ContextSuite context=TestCreateTemplate>:setupSkipped0.00test_templates.py
ContextSuite context=TestCopyDeleteTemplate>:setupSkipped0.00test_templates.py
test_static_role_account_aclsSkipped0.02test_staticroles.py
test_11_ss_nfs_version_on_ssvmSkipped0.02test_ssvm.py
test_01_scale_vmSkipped0.00test_scale_vm.py
test_01_primary_storage_iscsiSkipped0.09test_primary_storage.py
test_vm_nic_adapter_vmxnet3Skipped0.00test_nic_adapter_type.py
test_nested_virtualization_vmwareSkipped0.00test_nested_virtualization.py
test_06_copy_isoSkipped0.00test_iso.py
test_list_ha_for_host_validSkipped0.02test_hostha_simulator.py
test_list_ha_for_host_invalidSkipped0.03test_hostha_simulator.py
test_list_ha_for_hostSkipped0.02test_hostha_simulator.py
test_hostha_enable_feature_without_setting_providerSkipped0.02test_hostha_simulator.py
test_hostha_enable_feature_validSkipped0.02test_hostha_simulator.py
test_hostha_disable_feature_validSkipped0.02test_hostha_simulator.py
test_hostha_configure_invalid_providerSkipped0.03test_hostha_simulator.py
test_hostha_configure_default_driverSkipped0.03test_hostha_simulator.py
test_ha_verify_fsm_recoveringSkipped0.03test_hostha_simulator.py
test_ha_verify_fsm_fencedSkipped0.04test_hostha_simulator.py
test_ha_verify_fsm_degradedSkipped0.07test_hostha_simulator.py
test_ha_verify_fsm_availableSkipped0.03test_hostha_simulator.py
test_ha_multiple_mgmt_server_ownershipSkipped0.02test_hostha_simulator.py
test_ha_list_providersSkipped0.02test_hostha_simulator.py
test_ha_enable_feature_invalidSkipped0.02test_hostha_simulator.py
test_ha_disable_feature_invalidSkipped0.02test_hostha_simulator.py
test_ha_configure_enabledisable_across_clusterzonesSkipped0.03test_hostha_simulator.py
test_configure_ha_provider_validSkipped0.02test_hostha_simulator.py
test_configure_ha_provider_invalidSkipped0.03test_hostha_simulator.py
test_hostha_kvm_host_recoveringSkipped56.24test_hostha_kvm.py
test_hostha_kvm_host_fencingSkipped36.11test_hostha_kvm.py
test_hostha_kvm_host_degradedSkipped66.36test_hostha_kvm.py
test_deploy_vgpu_enabled_vmSkipped0.03test_deploy_vgpu_enabled_vm.py
test_3d_gpu_supportSkipped0.04test_deploy_vgpu_enabled_vm.py

@borisstoyanov

Copy link
Copy Markdown
Contributor

@DaanHoogland I see this error here when listing hosts:

2017-10-19 09:45:50,498 ERROR [c.c.a.ApiServer] (catalina-exec-8:ctx-6542176f ctx-66e777cb) (logid:cc54a5b2) unhandled exception executing api command: [Ljava.lang.String;@5aa032ab
com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@49891c22: SELECT host_view.id, host_view.uuid, host_view.name, host_view.status, host_view.type, host_view.private_ip_address, host_view.disconnected, host_view.version, host_view.hypervisor_type, host_view.hypervisor_version, host_view.capabilities, host_view.last_ping, host_view.created, host_view.removed, host_view.oobm_enabled, host_view.oobm_power_state, host_view.ha_enabled, host_view.ha_state, host_view.ha_provider, host_view.resource_state, host_view.mgmt_server_id, host_view.cpu_sockets, host_view.cpus, host_view.speed, host_view.ram, host_view.cluster_id, host_view.cluster_uuid, host_view.cluster_name, host_view.cluster_type, host_view.data_center_id, host_view.data_center_uuid, host_view.data_center_name, host_view.pod_id, host_view.pod_uuid, host_view.pod_name, host_view.guest_os_category_id, host_view.guest_os_category_uuid, host_view.guest_os_category_name, host_view.tag, host_view.memory_used_capacity, host_view.memory_reserved_capacity, host_view.cpu_used_capacity, host_view.cpu_reserved_capacity, host_view.job_id, host_view.job_uuid, host_view.job_status, host_view.annotation, host_view.last_annotated, host_view.username FROM host_view WHERE host_view.id IN (1,5)
at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:428)
at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:364)
at com.cloud.api.query.dao.HostJoinDaoImpl.searchByIds(HostJoinDaoImpl.java:447)
at sun.reflect.GeneratedMethodAccessor221.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy298.searchByIds(Unknown Source)
at com.cloud.api.query.QueryManagerImpl.searchForServersInternal(QueryManagerImpl.java:1683)
at com.cloud.api.query.QueryManagerImpl.searchForServers(QueryManagerImpl.java:1562)
at org.apache.cloudstack.api.command.admin.host.ListHostsCmd.getHostResponses(ListHostsCmd.java:208)
at org.apache.cloudstack.api.command.admin.host.ListHostsCmd.execute(ListHostsCmd.java:233)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:728)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:552)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:297)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:129)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:126)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:88)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1770)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1729)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'host_view.annotation' in 'field list'
at sun.reflect.GeneratedConstructorAccessor138.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
... 64 more

Does this looks like a db upgrade failure in annotations PR to you?

@borisstoyanov

Copy link
Copy Markdown
Contributor

@nvazquez can you please rebase and push

@nvazquez
nvazquezforce-pushed the roadmap-newl2networktype branch from df47971 to 78f1ecaCompareOctober 19, 2017 12:56
@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@borisstoyanov sure, done.

@borisstoyanov

Copy link
Copy Markdown
Contributor

Thanks @nvazquez, @blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1174

@borisstoyanov

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-1598)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 20180 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2281-t1598-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
Intermitten failure detected: /marvin/tests/smoke/test_host_annotations.py
Intermitten failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermitten failure detected: /marvin/tests/smoke/test_iso.py
Intermitten failure detected: /marvin/tests/smoke/test_network.py
Intermitten failure detected: /marvin/tests/smoke/test_password_server.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dns.py
Intermitten failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermitten failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermitten failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermitten failure detected: /marvin/tests/smoke/test_routers.py
Intermitten failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_templates.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_volumes.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Test completed. 41 look OK, 22 have error(s)

TestResultTime (s)Test File
test_01_vpc_site2site_vpnFailure10.30test_vpc_vpn.py
test_01_vpc_remote_access_vpnFailure5.17test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpnFailure5.31test_vpc_vpn.py
test_04_rvpc_privategw_static_routesFailure11.03test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanupFailure10.91test_privategw_acl.py
test_02_vpc_privategw_static_routesFailure10.92test_privategw_acl.py
test_01_vpc_privategw_aclFailure10.91test_privategw_acl.py
test_02_port_fwd_on_non_src_natFailure730.81test_network.py
test_02_VPC_default_routesError5.87test_vpc_router_nics.py
test_01_VPC_nics_after_destroyError10.92test_vpc_router_nics.py
test_05_rvpc_multi_tiersError10.93test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nicsError10.92test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routersError10.93test_vpc_redundant.py
test_02_redundant_VPC_default_routesError10.94test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACLError10.93test_vpc_redundant.py
ContextSuite context=TestVolumes>:setupError0.00test_volumes.py
ContextSuite context=TestCreateVolume>:setupError0.00test_volumes.py
ContextSuite context=TestVmSnapshot>:setupError5.97test_vm_snapshots.py
ContextSuite context=TestVMLifeCycle>:setupError0.00test_vm_life_cycle.py
ContextSuite context=TestDeployVM>:setupError0.00test_vm_life_cycle.py
ContextSuite context=TestTemplates>:setupError47.21test_templates.py
ContextSuite context=TestSnapshotRootDisk>:setupError0.00test_snapshots.py
ContextSuite context=TestServiceOfferings>:setupError6.15test_service_offerings.py
test_03_RVR_Network_check_router_stateError0.25test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_falseError5.35test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_falseError0.24test_routers_network_ops.py
test_01_isolate_network_FW_PF_default_routes_egress_trueError5.37test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_trueError0.25test_routers_network_ops.py
test_02_routervm_iptables_policiesError5.26test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policiesError10.33test_routers_iptables_default_policy.py
ContextSuite context=TestRouterServices>:setupError0.00test_routers.py
ContextSuite context=TestRouterDnsService>:setupError0.00test_router_dnsservice.py
ContextSuite context=TestRouterDns>:setupError0.00test_router_dns.py
ContextSuite context=TestRouterDHCPOpts>:setupError0.00test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPHosts>:setupError0.00test_router_dhcphosts.py
ContextSuite context=TestResetVmOnReboot>:setupError0.00test_reset_vm_on_reboot.py
test_isolate_network_password_serverError0.07test_password_server.py
test_releaseIPError5.98test_network.py
test_reboot_routerError226.82test_network.py
test_delete_accountError56.39test_network.py
test_02_port_fwd_on_non_src_natError735.94test_network.py
test_01_port_fwd_on_src_natError0.08test_network.py
ContextSuite context=TestRouterRules>:setupError11.97test_network.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfacesError156.91test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfacesError116.77test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80Error162.79test_internal_lb.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80Error107.26test_internal_lb.py
test_05_add_annotation_for_invalid_entityTypeError0.06test_host_annotations.py
test_06_verify_guest_lspci_againError715.45test_deploy_virtio_scsi_vm.py
test_04_verify_guest_lspciError715.75test_deploy_virtio_scsi_vm.py
test_change_service_offering_for_vm_with_snapshotsSkipped0.00test_vm_snapshots.py
ContextSuite context=TestCreateTemplate>:setupSkipped0.00test_templates.py
ContextSuite context=TestCopyDeleteTemplate>:setupSkipped0.00test_templates.py
test_static_role_account_aclsSkipped0.02test_staticroles.py
test_11_ss_nfs_version_on_ssvmSkipped0.02test_ssvm.py
test_01_scale_vmSkipped0.00test_scale_vm.py
test_01_primary_storage_iscsiSkipped0.08test_primary_storage.py
test_vm_nic_adapter_vmxnet3Skipped0.00test_nic_adapter_type.py
test_nested_virtualization_vmwareSkipped0.00test_nested_virtualization.py
test_06_copy_isoSkipped0.00test_iso.py
test_list_ha_for_host_validSkipped0.01test_hostha_simulator.py
test_list_ha_for_host_invalidSkipped0.01test_hostha_simulator.py
test_list_ha_for_hostSkipped0.02test_hostha_simulator.py
test_hostha_enable_feature_without_setting_providerSkipped0.02test_hostha_simulator.py
test_hostha_enable_feature_validSkipped0.02test_hostha_simulator.py
test_hostha_disable_feature_validSkipped0.02test_hostha_simulator.py
test_hostha_configure_invalid_providerSkipped0.02test_hostha_simulator.py
test_hostha_configure_default_driverSkipped0.01test_hostha_simulator.py
test_ha_verify_fsm_recoveringSkipped0.01test_hostha_simulator.py
test_ha_verify_fsm_fencedSkipped0.01test_hostha_simulator.py
test_ha_verify_fsm_degradedSkipped0.01test_hostha_simulator.py
test_ha_verify_fsm_availableSkipped0.02test_hostha_simulator.py
test_ha_multiple_mgmt_server_ownershipSkipped0.01test_hostha_simulator.py
test_ha_list_providersSkipped0.01test_hostha_simulator.py
test_ha_enable_feature_invalidSkipped0.01test_hostha_simulator.py
test_ha_disable_feature_invalidSkipped0.01test_hostha_simulator.py
test_ha_configure_enabledisable_across_clusterzonesSkipped0.02test_hostha_simulator.py
test_configure_ha_provider_validSkipped0.02test_hostha_simulator.py
test_configure_ha_provider_invalidSkipped0.02test_hostha_simulator.py
test_deploy_vgpu_enabled_vmSkipped0.03test_deploy_vgpu_enabled_vm.py
test_3d_gpu_supportSkipped0.03test_deploy_vgpu_enabled_vm.py

@borisstoyanov

Copy link
Copy Markdown
Contributor

@nvazquez can you please see the failing tests

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@borisstoyanov sure, I'll work on fixing failures

@nvazquez
nvazquezforce-pushed the roadmap-newl2networktype branch from 78f1eca to 9bad6fcCompareOctober 26, 2017 20:08
@nvazquez
nvazquezforce-pushed the roadmap-newl2networktype branch from 9bad6fc to 3e4a3eaCompareNovember 8, 2017 17:32
@nvazquez
nvazquezforce-pushed the roadmap-newl2networktype branch from fa8778f to 405ca44CompareDecember 11, 2017 22:22
@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@rhtyd @borisstoyanov master rebased, Jenkins and Travis are green now

@yadvr

Copy link
Copy Markdown
Member

Additional review from others request.
@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1367

@yadvr

Copy link
Copy Markdown
Member

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1394

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-1821)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 29591 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2281-t1821-kvm-centos7.zip
Smoke tests completed. 64 look OK, 3 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_01_vpc_privategw_aclFailure51.18test_privategw_acl.py
test_02_vpc_privategw_static_routesFailure287.67test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanupFailure106.75test_privategw_acl.py
test_04_rvpc_privategw_static_routesFailure317.73test_privategw_acl.py
test_02_create_template_with_checksum_sha1Error5.15test_templates.py
test_03_create_template_with_checksum_sha256Error5.15test_templates.py
test_04_create_template_with_checksum_md5Error5.16test_templates.py
test_01_vpc_remote_access_vpnFailure45.58test_vpc_vpn.py

@yadvr

Copy link
Copy Markdown
Member

@nvazquez Tests LGTM, additional review requested following which this can be merged.

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

Thanks @rhtyd
@DaanHoogland@rafaelweingartner@marcaurele can you please review?

@rafaelweingartnerrafaelweingartner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nvazquez I only have one or two remarks here.
Everything else seems to be ok

nicSearch.and().op("ip4Address", nicSearch.entity().getIPv4Address(), SearchCriteria.Op.NNULL);
nicSearch.or("ip6Address", nicSearch.entity().getIPv6Address(), SearchCriteria.Op.NNULL);
nicSearch.cp();
if (!network.getGuestType().equals(Network.GuestType.L2)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the method getGuestType return Null?
If not, the code is ok. Otherwise, it might be more interesting to do Network.GuestType.L2.equals(network.getGuestType())

Or even using ==, if it is an enum type, it is ok to use ==

@Override
public Set<Long> getAvailableIps(Network network, String requestedIp) {
if (network.getCidr() == null) {
return null;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of null, what about an empty set here?

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

Thanks @rafaelweingartner, I pushed a commit with suggested improvements

@nvazquez
nvazquezforce-pushed the roadmap-newl2networktype branch from 6748208 to 40fda18CompareDecember 19, 2017 13:44
@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1428

@nvazquez

Copy link
Copy Markdown
ContributorAuthor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-1832)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 28508 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2281-t1832-kvm-centos7.zip
Smoke tests completed. 63 look OK, 4 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_01_vpc_privategw_aclFailure57.00test_privategw_acl.py
test_02_vpc_privategw_static_routesFailure208.74test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanupFailure153.34test_privategw_acl.py
test_04_rvpc_privategw_static_routesFailure294.56test_privategw_acl.py
test_02_create_template_with_checksum_sha1Error5.29test_templates.py
test_03_create_template_with_checksum_sha256Error5.22test_templates.py
test_04_create_template_with_checksum_md5Error5.23test_templates.py
test_06_download_detached_volumeFailure147.53test_volumes.py
test_01_vpc_remote_access_vpnError50.81test_vpc_vpn.py

@yadvr

Copy link
Copy Markdown
Member

Tests LGTM, others are known failures. This is ready for merging.

@yadvr
yadvr merged commit 13c325a into apache:masterDec 20, 2017
@nvazquez
nvazquez deleted the roadmap-newl2networktype branch April 6, 2020 14:53
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@nvazquez@yadvr@blueorangutan@borisstoyanov@rafaelweingartner