Support to enable/disable VM High Availability manager by sureshanaparti · Pull Request #10118 · apache/cloudstack

sureshanaparti · 2024-12-17T11:55:58Z

Description

This PR adds support to enable/disable VM High Availability manager.

New config 'vm.ha.enabled' with Zone scope is added, to enable/disable VM High Availability manager. This is enable by default (for backward compatibilty). When enabled, the VM HA WorkItems (for VM Stop, Restart, Migration, Destroy) can be created and the scheduled items are executed. When disabled, new VM HA WorkItems are not allowed and the scheduled items are retried until max retries configured at 'vm.ha.migration.max.retries' (executed in case HA is re-enabled during retry attempts), and then purged after 'time.between.failures' by the cleanup thread that runs regularly at 'time.between.cleanup'.
New config 'vm.ha.alerts.enabled' with Zone scope is added, to enable/disable alerts for the VM HA operations. This is enabled by default.

Both these config settings can defined at zone/global level.

Doc PR: apache/cloudstack-documentation#464

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

New Settings =>

Sample Alert when 'vm.ha.enabled' is false =>

How Has This Been Tested?

Manually tested the VM HA related operations on host maintenance and during host down/alert, enabling & disabling new config 'vm.ha.enabled'.

…erts - Adds new config 'vm.ha.enabled' with Zone scope, to enable/disable VM High Availability manager. This is enable by default (for backward compatibilty). When enabled, the VM HA WorkItems (for VM Stop, Restart, Migration, Destroy) can be created and the scheduled items are executed. When disabled, new VM HA WorkItems are not allowed and the scheduled items are retried until max retries configured at 'vm.ha.migration.max.retries' (executed in case HA is re-enabled during retry attempts), and then purged after 'time.between.failures' by the cleanup thread that runs regularly at 'time.between.cleanup'. - Adds new config 'vm.ha.alerts.enabled' with Zone scope, to enable/disable alerts for the VM HA operations. This is enabled by default.

sureshanaparti · 2024-12-17T11:56:51Z

@blueorangutan package

codecov · 2024-12-17T12:03:58Z

Codecov Report

Attention: Patch coverage is 53.08642% with 38 lines in your changes missing coverage. Please review.

Project coverage is 16.04%. Comparing base (ac19379) to head (819f72a).
Report is 11 commits behind head on 4.20.

Files with missing lines	Patch %	Lines
...java/com/cloud/ha/HighAvailabilityManagerImpl.java	55.26%	21 Missing and 13 partials ⚠️
...n/java/com/cloud/resource/ResourceManagerImpl.java	0.00%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## 4.20 #10118 +/- ##
=========================================
Coverage 16.03% 16.04% - Complexity 12814 12829 +15 
=========================================
Files 5637 5637 Lines 493507 493574 +67 Branches 59831 59848 +17 =========================================
+ Hits 79131 79188 +57 + Misses 405600 405592 -8 - Partials 8776 8794 +18

Flag	Coverage Δ
uitests	`4.02% <ø> (ø)`
unittests	`16.88% <53.08%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

weizhouapache

code lgtm

sureshanaparti · 2024-12-17T12:54:12Z

@blueorangutan package

sureshanaparti · 2024-12-17T13:57:51Z

@blueorangutan package

sureshanaparti · 2024-12-18T06:24:19Z

@blueorangutan package

blueorangutan · 2024-12-18T08:20:06Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-12-18T09:33:11Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11839

blueorangutan · 2024-12-18T09:33:12Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11838

sureshanaparti · 2024-12-19T17:10:53Z

@blueorangutan package

blueorangutan · 2024-12-19T17:12:04Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-12-19T18:16:48Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11853

sureshanaparti · 2024-12-19T18:53:35Z

@blueorangutan test

blueorangutan · 2024-12-19T18:54:04Z

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2024-12-20T09:40:57Z

[SF] Trillian test result (tid-11956)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 50877 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10118-t11956-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

kiranchavala

LGTM , tested manually and the feature works fine

Test Case Execution	Result
Verify that high availability tasks for VMs work as usual when 'vm.ha.enabled' is enabled	Pass
Verify no alerts are sent to operators when scheduling of VM HA operations fails when 'vm.ha.alerts.enabled' is disabled	Pass
Exception message should be shown when vm.ha.enabled is disabled and host is kept in maintainence mode	Pass
The entires in op_ha_work should get purged by cleanup threads	Pass
Verify that certain tasks are not allowed when 'vm.ha.enabled' is disabled	Pass
Verify alerts are sent to operators when scheduling of VM HA operations fails when 'vm.ha.alerts.enabled' is enabled	Pass

* 4.20: VR: fix site-2-site VPN if split connections is enabled (#10067) UI: fix cannot open 'Edit tags' modal for static routes (#10065) Update ownership selection component to be language independent (#10052) Support to enable/disable VM High Availability manager and related alerts (#10118)

…erts (apache#10118) - Adds new config 'vm.ha.enabled' with Zone scope, to enable/disable VM High Availability manager. This is enable by default (for backward compatibilty). When enabled, the VM HA WorkItems (for VM Stop, Restart, Migration, Destroy) can be created and the scheduled items are executed. When disabled, new VM HA WorkItems are not allowed and the scheduled items are retried until max retries configured at 'vm.ha.migration.max.retries' (executed in case HA is re-enabled during retry attempts), and then purged after 'time.between.failures' by the cleanup thread that runs regularly at 'time.between.cleanup'. - Adds new config 'vm.ha.alerts.enabled' with Zone scope, to enable/disable alerts for the VM HA operations. This is enabled by default.

sureshanaparti added this to the 4.20.1 milestone Dec 17, 2024

sureshanaparti added the status:needs-testing label Dec 17, 2024

sureshanaparti requested a review from weizhouapache December 17, 2024 12:08

weizhouapache approved these changes Dec 17, 2024
View reviewed changes

apache deleted a comment from blueorangutan Dec 19, 2024

kiranchavala approved these changes Dec 24, 2024
View reviewed changes

sureshanaparti marked this pull request as ready for review December 26, 2024 08:14

yadvr removed the status:needs-testing label Dec 26, 2024

yadvr merged commit 330ed25 into apache:4.20Dec 26, 2024

yadvr deleted the cs-vm-high-availablity-on-off branch December 26, 2024 12:15

Pearl1594 added this to ACS 4.20.1 Mar 17, 2025

Pearl1594 moved this to Done in ACS 4.20.1 Mar 17, 2025

Uh oh!

Conversation

sureshanaparti commented Dec 17, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

sureshanaparti commented Dec 17, 2024

Uh oh!

codecovBot commented Dec 17, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

sureshanaparti commented Dec 17, 2024

Uh oh!

sureshanaparti commented Dec 17, 2024

Uh oh!

sureshanaparti commented Dec 18, 2024

Uh oh!

blueorangutan commented Dec 18, 2024

Uh oh!

blueorangutan commented Dec 18, 2024

Uh oh!

blueorangutan commented Dec 18, 2024

Uh oh!

sureshanaparti commented Dec 19, 2024

Uh oh!

blueorangutan commented Dec 19, 2024

Uh oh!

blueorangutan commented Dec 19, 2024

Uh oh!

sureshanaparti commented Dec 19, 2024

Uh oh!

blueorangutan commented Dec 19, 2024

Uh oh!

blueorangutan commented Dec 20, 2024

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sureshanaparti commented Dec 17, 2024•
edited
Loading

codecovBot commented Dec 17, 2024•
edited
Loading