CLOUDSTACK-9245 - Cannot delete non-attached ACL that contains items

[wilderrodrigues]

This PR fixes the issue when trying to delete ACL lists which contain item.

It seemed it was not thought of when the ACS project started, when most user were relying on the UI to execute those tasks. Nowadays, with automation all over the place and ACL lists containing hundreds of items, it's very hard to have to delete them 1 by 1 either via the UI. Writing scripts to do so might be a solution, but it would be much simpler to just delete non-attached ACLs with all its items in one go.

Also, destroying a VPC that contains ACL lists was "succeeding", but after that the ACL list/items were messing up:

list networkacls aclid=920d74b6-4d15-454f-b3a6-61e7a6ffd1a4
Error 431: Unable to find VPC associated with acl
{
"cserrorcode": 4350,
"errorcode": 431,
"errortext": "Unable to find VPC associated with acl",
"uuidList": []
}

So, it also cleans up ACLs when destroying VPCs

[wilderrodrigues]

Ping @remibergsma@borisroman@michaelandersen

Manual tests:

• Deleting attached ACL

• ACL with items

• Deleting attached ACL

• Using cloud-monkey to delete attached ACL

(local) 🐵 > delete networkacllist id=98a50aad-52c4-4196-b71c-c414eb939435 accountid = 0223d070-c0e1-11e5-902d-5254001daa61
cmd = org.apache.cloudstack.api.command.user.network.DeleteNetworkACLListCmd
created = 2016-01-22T09:30:04+0000
jobid = 2b134fd9-a8cf-49e4-96fc-b25b0046ab64
jobprocstatus = 0
jobresult:
success = True
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 0223deab-c0e1-11e5-902d-5254001daa61
(local) 🐵 > 

[wilderrodrigues]

Found and issue when destroying the VPC. Will fix it and update the PR.

(local) 🐵 > delete vpc id=be232d77-495e-4f89-9b89-972db6a2efc5 Error Async job f5640572-cbfb-4501-96ea-042ca1b0c921 failed
Error 530, Command failed due to Internal Server Error

[wilderrodrigues]

Problem fixed!

Ping @remibergsma@borisroman

• Delete ACL that contains items via Cloud Monkey

(local) 🐵 > delete networkacllist id=e4ade8e1-9e69-489a-8642-b895f730090d accountid = 795ac80c-c0fc-11e5-902d-5254001daa61
cmd = org.apache.cloudstack.api.command.user.network.DeleteNetworkACLListCmd
created = 2016-01-22T12:11:01+0000
jobid = aa6835e8-efb0-4e27-9191-b6d738b8cf99
jobprocstatus = 0
jobresult:
success = True
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 795ada3a-c0fc-11e5-902d-5254001daa61
(local) 🐵 >

• Destroy VPC that has ACL with items via Cloud Monkey

(local) 🐵 > delete vpc id=77338add-a3d4-4c07-8ff3-b37100a8dc28 accountid = 795ac80c-c0fc-11e5-902d-5254001daa61
cmd = org.apache.cloudstack.api.command.user.vpc.DeleteVPCCmd
created = 2016-01-22T12:13:50+0000
jobid = 7cb18193-6d4f-4a0b-afd4-96c25ab7731e
jobprocstatus = 0
jobresult:
success = True
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 795ada3a-c0fc-11e5-902d-5254001daa61
(local) 🐵 > 

• List ACLs after destroying the VPC

(local) 🐵 > list networkacls
count = 4
networkacl:
id = 7a955f27-c0fc-11e5-902d-5254001daa61
aclid = 7a955664-c0fc-11e5-902d-5254001daa61
action = Allow
cidrlist = 0.0.0.0/0
fordisplay = True
number = 2
protocol = all
state = Active
tags:
traffictype = Egress
================================================================================
id = 7a955aae-c0fc-11e5-902d-5254001daa61
aclid = 7a955664-c0fc-11e5-902d-5254001daa61
action = Allow
cidrlist = 0.0.0.0/0
fordisplay = True
number = 1
protocol = all
state = Active
tags:
traffictype = Ingress
================================================================================
id = 7a9551d2-c0fc-11e5-902d-5254001daa61
aclid = 7a954338-c0fc-11e5-902d-5254001daa61
action = Deny
cidrlist = 0.0.0.0/0
fordisplay = True
number = 2
protocol = all
state = Active
tags:
traffictype = Egress
================================================================================
id = 7a954c65-c0fc-11e5-902d-5254001daa61
aclid = 7a954338-c0fc-11e5-902d-5254001daa61
action = Deny
cidrlist = 0.0.0.0/0
fordisplay = True
number = 1
protocol = all
state = Active
tags:
traffictype = Ingress
(local) 🐵 > 

[remibergsma]

LGTM based on these tests:

nosetests --with-marvin --marvin-config=${marvinCfg} -s -a tags=advanced,required_hardware=true \
component/test_vpc_redundant.py \
component/test_routers_iptables_default_policy.py \
component/test_routers_network_ops.py \
component/test_vpc_router_nics.py \
smoke/test_loadbalance.py \
smoke/test_internal_lb.py \
smoke/test_ssvm.py \
smoke/test_network.py

Result:

Check the password file in the Router VM ... === TestName: test_isolate_network_password_server | Status : SUCCESS ===
ok
Create a redundant VPC with two networks with two VMs in each network ... === TestName: test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Status : SUCCESS ===
ok
Create a redundant VPC with two networks with two VMs in each network and check default routes ... === TestName: test_02_redundant_VPC_default_routes | Status : SUCCESS ===
ok
Create a redundant VPC with two networks with two VMs in each network ... === TestName: test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | Status : SUCCESS ===
ok
Create a redundant VPC with 1 Tier, 1 VM, 1 ACL, 1 PF and test Network GC Nics ... === TestName: test_04_rvpc_network_garbage_collector_nics | Status : SUCCESS ===
ok
Create a redundant VPC with 1 Tier, 1 VM, 1 ACL, 1 PF and test Network GC Nics ... === TestName: test_05_rvpc_multi_tiers | Status : SUCCESS ===
ok
Test iptables default INPUT/FORWARD policy on RouterVM ... === TestName: test_02_routervm_iptables_policies | Status : SUCCESS ===
ok
Test iptables default INPUT/FORWARD policies on VPC router ... === TestName: test_01_single_VPC_iptables_policies | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_01_isolate_network_FW_PF_default_routes_egress_true | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_02_isolate_network_FW_PF_default_routes_egress_false | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_03_RVR_Network_check_router_state | Status : SUCCESS ===
ok
Create a VPC with two networks with one VM in each network and test nics after destroy ... === TestName: test_01_VPC_nics_after_destroy | Status : SUCCESS ===
ok
Create a VPC with two networks with one VM in each network and test default routes ... === TestName: test_02_VPC_default_routes | Status : SUCCESS ===
ok
Check that the /etc/dhcphosts.txt doesn't contain duplicate IPs ... === TestName: test_router_dhcphosts | Status : SUCCESS ===
ok
Test to create Load balancing rule with source NAT ... === TestName: test_01_create_lb_rule_src_nat | Status : SUCCESS ===
ok
Test to create Load balancing rule with non source NAT ... === TestName: test_02_create_lb_rule_non_nat | Status : SUCCESS ===
ok
Test for assign & removing load balancing rule ... === TestName: test_assign_and_removal_lb | Status : SUCCESS ===
ok
Test create, assign, remove of an Internal LB with roundrobin http traffic to 3 vm's in a Single VPC ... === TestName: test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | Status : SUCCESS ===
ok
Test create, assign, remove of an Internal LB with roundrobin http traffic to 3 vm's in a Redundant VPC ... === TestName: test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | Status : SUCCESS ===
ok
Test to verify access to loadbalancer haproxy admin stats page ... === TestName: test_03_vpc_internallb_haproxy_stats_on_all_interfaces | Status : SUCCESS ===
ok
Test to verify access to loadbalancer haproxy admin stats page ... === TestName: test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | Status : SUCCESS ===
ok
Test SSVM Internals ... === TestName: test_03_ssvm_internals | Status : SUCCESS ===
ok
Test CPVM Internals ... === TestName: test_04_cpvm_internals | Status : SUCCESS ===
ok
Test stop SSVM ... === TestName: test_05_stop_ssvm | Status : SUCCESS ===
ok
Test stop CPVM ... === TestName: test_06_stop_cpvm | Status : SUCCESS ===
ok
Test reboot SSVM ... === TestName: test_07_reboot_ssvm | Status : SUCCESS ===
ok
Test reboot CPVM ... === TestName: test_08_reboot_cpvm | Status : SUCCESS ===
ok
Test destroy SSVM ... === TestName: test_09_destroy_ssvm | Status : SUCCESS ===
ok
Test destroy CPVM ... === TestName: test_10_destroy_cpvm | Status : SUCCESS ===
ok
test_01_vpc_privategw_acl (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_01_vpc_privategw_acl | Status : SUCCESS ===
ok
test_02_vpc_privategw_static_routes (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_02_vpc_privategw_static_routes | Status : SUCCESS ===
ok
test_03_vpc_privategw_restart_vpc_cleanup (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_03_vpc_privategw_restart_vpc_cleanup | Status : SUCCESS ===
ok
test_04_rvpc_privategw_static_routes (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_04_rvpc_privategw_static_routes | Status : SUCCESS ===
ok
Test for port forwarding on source NAT ... === TestName: test_01_port_fwd_on_src_nat | Status : SUCCESS ===
ok
Test for port forwarding on non source NAT ... === TestName: test_02_port_fwd_on_non_src_nat | Status : SUCCESS ===
ok
Test for reboot router ... === TestName: test_reboot_router | Status : SUCCESS ===
ok
Test for Router rules for network rules on acquired public IP ... === TestName: test_network_rules_acquired_public_ip_1_static_nat_rule | Status : SUCCESS ===
ok
Test for Router rules for network rules on acquired public IP ... === TestName: test_network_rules_acquired_public_ip_2_nat_rule | Status : SUCCESS ===
ok
Test for Router rules for network rules on acquired public IP ... === TestName: test_network_rules_acquired_public_ip_3_Load_Balancer_Rule | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 44 tests in 21351.823s
OK

And:

nosetests --with-marvin --marvin-config=${marvinCfg} -s -a tags=advanced,required_hardware=false \
smoke/test_routers.py \
smoke/test_network_acl.py \
smoke/test_privategw_acl.py \
smoke/test_reset_vm_on_reboot.py \
smoke/test_vm_life_cycle.py \
smoke/test_vpc_vpn.py \
smoke/test_service_offerings.py \
component/test_vpc_offerings.py \
component/test_vpc_routers.py

Result:

Test router internal advanced zone ... === TestName: test_02_router_internal_adv | Status : SUCCESS ===
ok
Test restart network ... === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
ok
Test router basic setup ... === TestName: test_05_router_basic | Status : SUCCESS ===
ok
Test router advanced setup ... === TestName: test_06_router_advanced | Status : SUCCESS ===
ok
Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
ok
Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
ok
Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS ===
ok
Test reset virtual machine on reboot ... === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===
ok
Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
ok
Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS ===
ok
Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
ok
Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS ===
ok
Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS ===
ok
Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS ===
ok
Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS ===
ok
Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS ===
ok
Test migrate VM ... === TestName: test_08_migrate_vm | Status : SUCCESS ===
ok
Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS ===
ok
Test to create service offering ... === TestName: test_01_create_service_offering | Status : SUCCESS ===
ok
Test to update existing service offering ... === TestName: test_02_edit_service_offering | Status : SUCCESS ===
ok
Test to delete service offering ... === TestName: test_03_delete_service_offering | Status : SUCCESS ===
ok
Test for delete account ... === TestName: test_delete_account | Status : SUCCESS ===
ok
Test for Associate/Disassociate public IP address for admin account ... === TestName: test_public_ip_admin_account | Status : SUCCESS ===
ok
Test for Associate/Disassociate public IP address for user account ... === TestName: test_public_ip_user_account | Status : SUCCESS ===
ok
Test for release public IP address ... === TestName: test_releaseIP | Status : SUCCESS ===
ok
Test create VPC offering ... === TestName: test_01_create_vpc_offering | Status : SUCCESS ===
ok
Test VPC offering without load balancing service ... === TestName: test_03_vpc_off_without_lb | Status : SUCCESS ===
ok
Test VPC offering without static NAT service ... === TestName: test_04_vpc_off_without_static_nat | Status : SUCCESS ===
ok
Test VPC offering without port forwarding service ... === TestName: test_05_vpc_off_without_pf | Status : SUCCESS ===
ok
Test VPC offering with invalid services ... === TestName: test_06_vpc_off_invalid_services | Status : SUCCESS ===
ok
Test update VPC offering ... === TestName: test_07_update_vpc_off | Status : SUCCESS ===
ok
Test list VPC offering ... === TestName: test_08_list_vpc_off | Status : SUCCESS ===
ok
test_09_create_redundant_vpc_offering (integration.component.test_vpc_offerings.TestVPCOffering) ... === TestName: test_09_create_redundant_vpc_offering | Status : SUCCESS ===
ok
Test start/stop of router after addition of one guest network ... === TestName: test_01_start_stop_router_after_addition_of_one_guest_network | Status : SUCCESS ===
ok
Test reboot of router after addition of one guest network ... === TestName: test_02_reboot_router_after_addition_of_one_guest_network | Status : SUCCESS ===
ok
Test to change service offering of router after addition of one guest network ... === TestName: test_04_chg_srv_off_router_after_addition_of_one_guest_network | Status : SUCCESS ===
ok
Test destroy of router after addition of one guest network ... === TestName: test_05_destroy_router_after_addition_of_one_guest_network | Status : SUCCESS ===
ok
Test to stop and start router after creation of VPC ... === TestName: test_01_stop_start_router_after_creating_vpc | Status : SUCCESS ===
ok
Test to reboot the router after creating a VPC ... === TestName: test_02_reboot_router_after_creating_vpc | Status : SUCCESS ===
ok
Tests to change service offering of the Router after ... === TestName: test_04_change_service_offerring_vpc | Status : SUCCESS ===
ok
Test to destroy the router after creating a VPC ... === TestName: test_05_destroy_router_after_creating_vpc | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 41 tests in 7065.735s
OK

[remibergsma]

Ping @borisroman

[borisroman]

LGTM 👍

Based on integration tests and code-review.

[wilderrodrigues]

Thanks @borisroman and @remibergsma .

Cheers,
Wilder