cleanup some smoke tests by DaanHoogland · Pull Request #8196 · apache/cloudstack

DaanHoogland · 2023-11-08T08:40:58Z

Description

This PR splits the test_vm_life_cycle.py test file and cleans some of the conditional test skipping. This all in an attempt to reduce intermittent integration test failures.

The reason is that a lot of tests are not basic life cycle tests and differnt failures on different platform are mostly pre-conditional instead of genuine positives.

During this work I also noticed that we might gain by eliminating tests as it seems a lot of the same functionalities are tested at different locations. I will not investigate that further or address it in this PR.

I will look at other intermittent failures as well and try to address those as well. This may happen in this PR or in separate PRs.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov · 2023-11-08T11:42:46Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (08749d8) 13.12% compared to head (251a913) 13.13%.

Additional details and impacted files

@@ Coverage Diff @@## 4.18 #8196 +/- ##
=========================================
Coverage 13.12% 13.13% - Complexity 9141 9142 +1 
=========================================
Files 2720 2720 Lines 257726 257726 Branches 40177 40177 =========================================
+ Hits 33838 33847 +9 + Misses 219598 219588 -10 - Partials 4290 4291 +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

codecov-commenter · 2024-04-19T09:34:21Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 16.58%. Comparing base (3e3a0c0) to head (bcc129c).
⚠️ Report is 276 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #8196 +/- ##
=========================================
Coverage 16.58% 16.58% Complexity 13989 13989 =========================================
Files 5743 5743 Lines 510706 510706 Branches 62119 62119 =========================================
Hits 84689 84689 Misses 416543 416543 Partials 9474 9474

Flag	Coverage Δ
uitests	`3.90% <ø> (ø)`
unittests	`17.48% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2024-06-07T12:04:26Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions · 2024-08-20T10:43:48Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland · 2024-08-23T10:37:08Z

@blueorangutan package

blueorangutan · 2024-08-23T10:38:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-08-23T11:33:27Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10779

DaanHoogland · 2024-08-26T07:29:43Z

@blueorangutan test

blueorangutan · 2024-08-26T07:32:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2024-08-26T22:45:55Z

[SF] Trillian test result (tid-11173)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 52255 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8196-t11173-kvm-ol8.zip
Smoke tests completed. 137 look OK, 7 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_primary_storage_disabled_host	`Error`	0.23	test_primary_storage.py
test_03_migration_options_storage_tags	`Error`	0.02	test_primary_storage.py
test_01_secure_vm_migration	`Error`	191.29	test_secured_vm_migration.py
test_02_unsecure_vm_migration	`Error`	450.63	test_secured_vm_migration.py
test_04_nonsecured_to_secured_vm_migration	`Error`	378.93	test_secured_vm_migration.py
ContextSuite context=TestCpuCapServiceOfferings>:setup	`Error`	0.00	test_service_offerings.py
test_02_list_snapshots_with_removed_data_store	`Error`	10.64	test_snapshots.py
test_02_list_snapshots_with_removed_data_store	`Error`	10.64	test_snapshots.py
test_01_deploy_vm_on_specific_host	`Error`	0.10	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	`Error`	0.15	test_vm_deployment_planner.py
test_08_migrate_vm	`Error`	0.07	test_vm_life_cycle.py
test_13_destroy_and_expunge_vm	`Error`	0.03	test_vm_life_cycle.py
test_01_migrate_vm_strict_tags_success	`Error`	0.30	test_vm_strict_host_tags.py
test_02_migrate_vm_strict_tags_failure	`Error`	0.30	test_vm_strict_host_tags.py
test_01_restore_vm_strict_tags_success	`Error`	0.30	test_vm_strict_host_tags.py
test_02_restore_vm_strict_tags_failure	`Error`	0.30	test_vm_strict_host_tags.py
test_01_scale_vm_strict_tags_success	`Error`	0.29	test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure	`Error`	0.31	test_vm_strict_host_tags.py
test_01_deploy_vm_on_specific_host_without_strict_tags	`Error`	0.27	test_vm_strict_host_tags.py
test_02_deploy_vm_on_any_host_without_strict_tags	`Error`	5.72	test_vm_strict_host_tags.py
test_03_deploy_vm_on_specific_host_with_strict_tags_success	`Error`	0.24	test_vm_strict_host_tags.py
test_04_deploy_vm_on_any_host_with_strict_tags_success	`Error`	5.70	test_vm_strict_host_tags.py
test_05_deploy_vm_on_specific_host_with_strict_tags_failure	`Failure`	0.28	test_vm_strict_host_tags.py

github-actions · 2024-09-09T13:31:09Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland · 2024-09-20T10:15:10Z

@blueorangutan package

blueorangutan · 2024-09-20T10:16:04Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-09-20T11:17:10Z

Packaging result [SF]: ✔️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 11160

blueorangutan · 2024-09-20T15:41:46Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11165

DaanHoogland · 2024-09-23T07:19:03Z

@blueorangutan test

blueorangutan · 2024-09-23T07:20:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2024-09-24T00:14:32Z

[SF] Trillian test result (tid-11534)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 58592 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8196-t11534-kvm-ol8.zip
Smoke tests completed. 139 look OK, 7 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_primary_storage_disabled_host	`Error`	336.07	test_primary_storage.py
test_01_add_primary_storage_disabled_host	`Error`	336.09	test_primary_storage.py
test_03_migration_options_storage_tags	`Error`	0.02	test_primary_storage.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true	`Failure`	203.14	test_routers_network_ops.py
test_03_secured_to_nonsecured_vm_migration	`Error`	114.71	test_secured_vm_migration.py
test_03_secured_to_nonsecured_vm_migration	`Error`	114.71	test_secured_vm_migration.py
test_04_nonsecured_to_secured_vm_migration	`Error`	0.01	test_secured_vm_migration.py
test_02_list_snapshots_with_removed_data_store	`Error`	8.51	test_snapshots.py
test_02_list_snapshots_with_removed_data_store	`Error`	8.52	test_snapshots.py
ContextSuite context=TestISOUsage>:setup	`Error`	0.00	test_usage.py
test_08_migrate_vm	`Error`	45.02	test_vm_life_cycle.py
test_13_destroy_and_expunge_vm	`Error`	0.02	test_vm_life_cycle.py
test_01_migrate_vm_strict_tags_success	`Error`	98.02	test_vm_strict_host_tags.py

github-actions · 2025-09-17T09:42:05Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland · 2025-09-17T12:17:08Z

will have to be redone at some time

boring-cyborgBot added component:integration-test Python Warning... Python code Ahead! labels Nov 8, 2023

apache deleted a comment from blueorangutan Nov 8, 2023

DaanHoogland commented Nov 9, 2023
View reviewed changes

Comment threadtest/integration/smoke/test_kvm_lifemigration.py
Comment threadtest/integration/smoke/test_secured_vm_migration.py

DaanHoogland mentioned this pull request Nov 14, 2023
Fix deploy as is VM start after template deletion #8115
Merged
11 tasks

DaanHoogland changed the title ~~cleanup vm life cycle test file~~cleanup some smoke testsNov 14, 2023

apache deleted a comment from blueorangutan Nov 14, 2023

DaanHoogland force-pushed the smoke_tests_cleanup branch from 9402195 to 2a7a40cCompare November 14, 2023 15:36

DaanHoogland mentioned this pull request Nov 14, 2023
cleanup test resources #3693
Open
37 tasks

github-actionsBot added the status:has-conflicts label Dec 21, 2023

DaanHoogland force-pushed the smoke_tests_cleanup branch from f4b56ea to 251a913Compare January 3, 2024 09:35

github-actionsBot removed the status:has-conflicts label Jan 3, 2024

DaanHoogland force-pushed the smoke_tests_cleanup branch from 251a913 to cdf41eaCompare April 19, 2024 09:33

DaanHoogland force-pushed the smoke_tests_cleanup branch from cdf41ea to 7442520Compare April 22, 2024 08:06

DaanHoogland marked this pull request as draft April 22, 2024 08:07

apache deleted a comment from blueorangutan Apr 22, 2024

github-actionsBot added the status:has-conflicts label Jun 7, 2024

github-actionsBot removed the status:has-conflicts label Jun 7, 2024

github-actionsBot added the status:has-conflicts label Aug 20, 2024

DaanHoogland force-pushed the smoke_tests_cleanup branch from c352ebe to 3a7484cCompare August 23, 2024 10:36

github-actionsBot removed the status:has-conflicts label Aug 23, 2024

github-actionsBot added the status:has-conflicts label Sep 9, 2024

DaanHoogland force-pushed the smoke_tests_cleanup branch from 3a7484c to 29cc73fCompare September 20, 2024 10:14

github-actionsBot removed the status:has-conflicts label Sep 20, 2024

cleanup vm life cycle test file
bcc129c

DaanHoogland force-pushed the smoke_tests_cleanup branch from 29cc73f to bcc129cCompare July 4, 2025 11:51

github-actionsBot added the status:has-conflicts label Sep 17, 2025

DaanHoogland closed this Sep 17, 2025

Uh oh!

Conversation

DaanHoogland commented Nov 8, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecovBot commented Nov 8, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Apr 19, 2024• edited by codecov BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actionsBot commented Jun 7, 2024

Uh oh!

github-actionsBot commented Aug 20, 2024

Uh oh!

DaanHoogland commented Aug 23, 2024

Uh oh!

blueorangutan commented Aug 23, 2024

Uh oh!

blueorangutan commented Aug 23, 2024

Uh oh!

DaanHoogland commented Aug 26, 2024

Uh oh!

blueorangutan commented Aug 26, 2024

Uh oh!

blueorangutan commented Aug 26, 2024

Uh oh!

github-actionsBot commented Sep 9, 2024

Uh oh!

DaanHoogland commented Sep 20, 2024

Uh oh!

blueorangutan commented Sep 20, 2024

Uh oh!

blueorangutan commented Sep 20, 2024

Uh oh!

blueorangutan commented Sep 20, 2024

Uh oh!

DaanHoogland commented Sep 23, 2024

Uh oh!

blueorangutan commented Sep 23, 2024

Uh oh!

blueorangutan commented Sep 24, 2024

Uh oh!

github-actionsBot commented Sep 17, 2025

Uh oh!

DaanHoogland commented Sep 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

DaanHoogland commented Nov 8, 2023•
edited
Loading

codecovBot commented Nov 8, 2023•
edited
Loading

codecov-commenter commented Apr 19, 2024•
edited by codecov Bot
Loading