cleanup processed files in VR

[DaanHoogland]

Description

When a PR is long running, the directory /var/cache/cloud/processed filles up and might glock the filesystem, rendring the VR unusable.
This PR implements a mech for cleaning that directory on demand.
(to be triggered yet, by API/schedule/configurable timer???)

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

[codecov]

Codecov Report

Merging #7632 (38089c7) into 4.18 (e4117e6) will decrease coverage by 0.04%.
The diff coverage is 54.76%.

❗ Current head 38089c7 differs from pull request most recent head 555e054. Consider uploading reports for the commit 555e054 to get more accurate results

@@ Coverage Diff @@## 4.18 #7632 +/- ##
============================================
- Coverage 13.05% 13.02% -0.04% + Complexity 9083 9032 -51 
============================================
Files 2720 2720 Lines 257370 257067 -303 Branches 40124 40087 -37 ============================================
- Hits 33612 33476 -136 + Misses 219537 219387 -150 + Partials 4221 4204 -17 

Files Changed	Coverage Δ
...va/com/cloud/agent/manager/DirectAgentAttache.java	9.83% <0.00%> (ø)
...tack/engine/orchestration/NetworkOrchestrator.java	6.06% <0.00%> (ø)
...ork/router/VirtualNetworkApplianceManagerImpl.java	2.72% <0.00%> (-0.01%)	⬇️
...network/router/VirtualNetworkApplianceManager.java	100.00% <100.00%> (ø)

... and 29 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[DaanHoogland]

nts: does this mean the answer should also inherit (from NetworkElementAnswer for instance)

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6690

[DaanHoogland]

@blueorangutan test keepEnv

[blueorangutan]

@DaanHoogland a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-7316)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39009 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7632-t7316-kvm-centos7.zip
Smoke tests completed. 95 look OK, 13 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_network_acl	Error	39.99	test_network_acl.py
test_01_vpc_privategw_acl	Failure	42.66	test_privategw_acl.py
test_02_vpc_privategw_static_routes	Failure	39.65	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	Failure	42.62	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	Failure	46.12	test_privategw_acl.py
test_10_vpc_tier_kubernetes_cluster	Error	82.22	test_kubernetes_clusters.py
test_02_native_to_native_vpc_migration	Error	40.80	test_migration.py
test_03_create_vpc_domain_vpc_offering	Error	51.28	test_domain_vpc_offerings.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80	Failure	46.52	test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80	Failure	46.54	test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces	Failure	44.63	test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces	Failure	46.49	test_internal_lb.py
test_01_single_VPC_iptables_policies	Error	41.16	test_routers_iptables_default_policy.py
test_07_autoscaling_vmgroup_on_vpc_network	Error	43.05	test_vm_autoscaling.py
test_01_migrate_VM_and_root_volume	Error	81.75	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	Error	53.38	test_vm_life_cycle.py
test_01_verify_ipv6_vpc	Error	41.15	test_vpc_ipv6.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	Error	41.79	test_vpc_redundant.py
test_02_redundant_VPC_default_routes	Error	42.82	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	Error	45.82	test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics	Error	43.77	test_vpc_redundant.py
test_05_rvpc_multi_tiers	Error	41.72	test_vpc_redundant.py
test_01_VPC_nics_after_destroy	Error	39.56	test_vpc_router_nics.py
test_02_VPC_default_routes	Error	40.68	test_vpc_router_nics.py
test_01_redundant_vpc_site2site_vpn	Failure	42.20	test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options	Failure	43.21	test_vpc_vpn.py
test_01_vpc_remote_access_vpn	Failure	34.87	test_vpc_vpn.py
test_01_vpc_site2site_vpn	Failure	44.36	test_vpc_vpn.py

[DaanHoogland]

vpc creation failes after these changes: needs work

[weizhouapache]

@DaanHoogland
there was a commit for similar issue see #5331
the script runs once every day. we could let it run every hour or every minute.

[DaanHoogland]

@DaanHoogland there was a commit for similar issue see #5331 the script runs once every day. we could let it run every hour or every minute.

let's push this forward then (marking 4.18.2 for now and we can discuss if it is still applicable)

[DaanHoogland]

@blueorangutan LLpackage

[blueorangutan]

@DaanHoogland a [LL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [LL]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6191

[DaanHoogland]

@blueorangutan LLtest matrix

[blueorangutan]

@DaanHoogland a [LL] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

[LL] Trillian Build Failed (tid-6763)

[DaanHoogland]

no interest