Skip to content

[Veeam] Check for failures in the restore process - #7224

Merged
sureshanaparti merged 4 commits into
apache:4.19from
scclouds:veeam-check-failure-when-restoring
Jun 25, 2024
Merged

[Veeam] Check for failures in the restore process#7224
sureshanaparti merged 4 commits into
apache:4.19from
scclouds:veeam-check-failure-when-restoring

Conversation

@SadiJr

@SadiJrSadiJr commented Feb 14, 2023

Copy link
Copy Markdown
Contributor

Description

Using the VMware hypervisor with the Veeam plugin active, when restoring a backup, ACS only verifies if the restore has finished successfully, ignoring any failure and making the user wait the timeout defined in backup.plugin.veeam.restore.timeout. This behavior has been fixed by this PR.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

It was tested in a local lab:

  1. I created one VM, added this VM to one backup offering, and made one manual backup;
  2. I tested restoring this backup, and canceled the restore in Veeam server;
  3. Before, I needed to wait for the timeout of the restore process to be reached before I was notified that the restore failed;
  4. Now, the ACS notifies me when it receives the failure message from Veeam.

@codecov

codecovBot commented Feb 14, 2023•
edited
Loading

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 4.18%. Comparing base (8b02624) to head (d432997).
Report is 9 commits behind head on 4.19.

Additional details and impacted files
@@ Coverage Diff @@## 4.19 #7224 +/- ##
============================================
- Coverage 14.95% 4.18% -10.77% 
============================================
Files 5380 369 -5011 Lines 470044 30284 -439760 Branches 60808 5359 -55449 ============================================
- Hits 70295 1268 -69027 + Misses 391965 28872 -363093 + Partials 7784 144 -7640
FlagCoverage Δ
uitests4.18% <ø> (-0.11%)⬇️
unittests?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

DaanHoogland
DaanHoogland reviewed Feb 15, 2023
View reviewed changes
Comment threadplugins/backup/veeam/src/main/java/org/apache/cloudstack/backup/veeam/VeeamClient.java Outdated
@sonarqubecloud

sonarqubecloudBot commented Feb 21, 2023

Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed! Quality Gate passed

BugA0 Bugs
VulnerabilityA0 Vulnerabilities
Security HotspotA0 Security Hotspots
Code SmellA2 Code Smells

58.0%58.0% Coverage
0.0%0.0% Duplication

DaanHoogland
DaanHoogland reviewed Feb 22, 2023
View reviewed changes
Comment threadplugins/backup/veeam/src/main/java/org/apache/cloudstack/backup/veeam/VeeamClient.java Outdated
@github-actions

github-actionsBot commented Mar 27, 2023

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@sonarqubecloud

sonarqubecloudBot commented Mar 28, 2023

Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed! Quality Gate passed

BugA0 Bugs
VulnerabilityA0 Vulnerabilities
Security HotspotA0 Security Hotspots
Code SmellA2 Code Smells

58.0%58.0% Coverage
0.0%0.0% Duplication

@DaanHooglandDaanHoogland added this to the 4.19.0.0 milestone Jun 22, 2023
JoaoJandre
JoaoJandre approved these changes Sep 28, 2023
View reviewed changes

@JoaoJandreJoaoJandre left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLTM, didn't test it.

DaanHoogland
DaanHoogland approved these changes Oct 31, 2023
View reviewed changes

@DaanHooglandDaanHoogland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLGTM

@DaanHoogland

DaanHoogland commented Oct 31, 2023

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

blueorangutan commented Oct 31, 2023

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

blueorangutan commented Oct 31, 2023

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7581

@DaanHoogland

DaanHoogland commented Nov 1, 2023

Copy link
Copy Markdown
Contributor

@blueorangutan test alma8 vmware-67u3

@blueorangutan

blueorangutan commented Nov 1, 2023

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (alma8 mgmt + vmware-67u3) has been kicked to run smoke tests

@blueorangutan

blueorangutan commented Nov 16, 2023

Copy link
Copy Markdown

[LL]Trillian test result (tid-6894)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server a8
Total time taken: 58684 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7224-t6894-vmware-70u3.zip
Smoke tests completed. 101 look OK, 14 have errors, 0 did not run
Only failed and skipped tests results shown below:

TestResultTime (s)Test File
test_02_balanced_drs_algorithmError420.04test_cluster_drs.py
test_list_system_vms_metrics_historyFailure0.21test_metrics_api.py
test_list_vms_metrics_adminError3606.15test_metrics_api.py
test_list_vms_metrics_historyError4.46test_metrics_api.py
test_list_volumes_metrics_historyError3621.13test_metrics_api.py
test_01_deploy_vm_on_specific_hostError2.24test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_clusterError3602.93test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_podError1.28test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_clusterError2.27test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_podError2.23test_vm_deployment_planner.py
test_09_expunge_vmFailure425.60test_vm_life_cycle.py
test_01_unmanage_vm_cycleError0.25test_vm_lifecycle_unmanage_import.py
ContextSuite context=TestUnmanageVM>:teardownError0.27test_vm_lifecycle_unmanage_import.py
ContextSuite context=TestVMSchedule>:setupError0.00test_vm_schedule.py
test_change_service_offering_for_vm_with_snapshotsError1.27test_vm_snapshots.py
ContextSuite context=TestVmSnapshot>:setupError9.00test_vm_snapshots.py
test_04_deploy_vnf_applianceError22.93test_vnf_templates.py
test_04_deploy_vnf_applianceError22.93test_vnf_templates.py
test_05_delete_vnf_templateError0.05test_vnf_templates.py
ContextSuite context=TestVnfTemplates>:teardownError1.14test_vnf_templates.py
ContextSuite context=TestCreateVolume>:setupError0.00test_volumes.py
ContextSuite context=TestVolumes>:setupError8.80test_volumes.py
test_01_verify_ipv6_vpcError3.31test_vpc_ipv6.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACLError4.72test_vpc_redundant.py
test_02_redundant_VPC_default_routesError4.69test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routersError4.68test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nicsError4.70test_vpc_redundant.py
test_05_rvpc_multi_tiersError4.67test_vpc_redundant.py
test_01_VPC_nics_after_destroyError4.67test_vpc_router_nics.py
test_02_VPC_default_routesError4.82test_vpc_router_nics.py
test_01_redundant_vpc_site2site_vpnFailure2.32test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_optionsFailure3.32test_vpc_vpn.py
test_01_vpc_remote_access_vpnFailure1.15test_vpc_vpn.py
test_01_vpc_site2site_vpnFailure2.30test_vpc_vpn.py
test_01_cancel_host_maintenace_with_no_migration_jobsError0.05test_host_maintenance.py
test_02_cancel_host_maintenace_with_migration_jobsError0.04test_host_maintenance.py
test_03_cancel_host_maintenace_with_migration_jobs_failureError0.05test_host_maintenance.py

@DaanHoogland

DaanHoogland commented Nov 16, 2023

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

blueorangutan commented Nov 16, 2023

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

blueorangutan commented Nov 16, 2023

Copy link
Copy Markdown

[SF] Trillian test result (tid-8333)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 48009 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7224-t8333-kvm-centos7.zip
Smoke tests completed. 114 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

TestResultTime (s)Test File
test_08_upgrade_kubernetes_ha_clusterFailure694.93test_kubernetes_clusters.py

@DaanHoogland

DaanHoogland commented Nov 17, 2023

Copy link
Copy Markdown
Contributor

@weizhouapache can you look at this?

@shwstppr

shwstppr commented Dec 10, 2023

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

blueorangutan commented Dec 10, 2023

Copy link
Copy Markdown

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

blueorangutan commented Dec 10, 2023

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8009

@shwstppr

shwstppr commented Dec 10, 2023

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

blueorangutan commented Dec 10, 2023

Copy link
Copy Markdown

@shwstppr a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@DaanHoogland
DaanHoogland changed the base branch from 4.19 to mainMarch 15, 2024 13:44
@JoaoJandre

JoaoJandre commented Mar 20, 2024

Copy link
Copy Markdown
Contributor

Hey @sureshanaparti, @SadiJr is focusing in other tasks, I will be working on this PR from now on. I'll try to address the reviews when I can and rebase with 4.19 if possible as well. It might take me some time to get to this.

@sureshanaparti

sureshanaparti commented Jun 20, 2024

Copy link
Copy Markdown
Contributor

Hi @JoaoJandre Is this PR targeted / ready for 4.19.1? If so, please rebase with 4.19, address any comments and change target branch to 4.19.

@JoaoJandre

JoaoJandre commented Jun 20, 2024

Copy link
Copy Markdown
Contributor

Hi @JoaoJandre Is this PR targeted / ready for 4.19.1? If so, please rebase with 4.19, address any comments and change target branch to 4.19.

Hey @sureshanaparti, It is ready, but I'm still checking if @weizhouapache 's suggestion is feasible. In any case the PR is working. I'll rebase with 4.19.

@JoaoJandre
JoaoJandreforce-pushed the veeam-check-failure-when-restoring branch from bfd1579 to d432997CompareJune 20, 2024 18:16
@JoaoJandre
JoaoJandre changed the base branch from main to 4.19June 20, 2024 18:16
@DaanHoogland

DaanHoogland commented Jun 21, 2024

Copy link
Copy Markdown
Contributor

@weizhouapache I know you are working with Veeam as well. Is this alright by you?

@sureshanaparti

sureshanaparti commented Jun 21, 2024

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

blueorangutan commented Jun 21, 2024

Copy link
Copy Markdown

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

blueorangutan commented Jun 21, 2024

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10048

@weizhouapache

weizhouapache commented Jun 21, 2024

Copy link
Copy Markdown
Member

@weizhouapache I know you are working with Veeam as well. Is this alright by you?

codewise looks good to me

I do not have a valid veeam license so I am not able to test now.

@sureshanaparti

sureshanaparti commented Jun 21, 2024

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

blueorangutan commented Jun 21, 2024

Copy link
Copy Markdown

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

blueorangutan commented Jun 21, 2024

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10057

@DaanHoogland

DaanHoogland commented Jun 21, 2024

Copy link
Copy Markdown
Contributor

@blueorangutan test matrix

@blueorangutan

blueorangutan commented Jun 21, 2024

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan

blueorangutan commented Jun 22, 2024

Copy link
Copy Markdown

[SF] Trillian test result (tid-10557)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45881 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7224-t10557-kvm-centos7.zip
Smoke tests completed. 131 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

TestResultTime (s)Test File

@blueorangutan

blueorangutan commented Jun 22, 2024

Copy link
Copy Markdown

[SF] Trillian test result (tid-10555)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46095 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7224-t10555-xenserver-71.zip
Smoke tests completed. 128 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

TestResultTime (s)Test File
test_12_destroy_cpvmError6.26test_ssvm.py
test_CreateTemplateWithDuplicateNameError195.18test_templates.py
test_02_create_template_with_checksum_sha1Error65.52test_templates.py
test_03_create_template_with_checksum_sha256Error65.52test_templates.py
test_04_create_template_with_checksum_md5Error65.52test_templates.py
test_05_create_template_with_no_checksumError65.55test_templates.py
test_03_delete_templateError1.07test_templates.py
test_04_extract_templateError1.07test_templates.py
test_09_list_templates_download_detailsFailure0.07test_templates.py
test_04_deploy_vnf_applianceError311.46test_vnf_templates.py

@blueorangutan

blueorangutan commented Jun 22, 2024

Copy link
Copy Markdown

[SF] Trillian test result (tid-10556)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 71748 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7224-t10556-vmware-67u3.zip
Smoke tests completed. 128 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

TestResultTime (s)Test File
test_01_restore_vmError3604.45test_restore_vm.py
test_02_restore_vm_allocated_rootError8.82test_restore_vm.py
test_01_deploy_vm_on_specific_hostError15.63test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_clusterError3605.45test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_podError1.33test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_clusterError2.38test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_podError1.31test_vm_deployment_planner.py
test_09_expunge_vmFailure424.63test_vm_life_cycle.py

sureshanaparti
sureshanaparti approved these changes Jun 23, 2024
View reviewed changes

@sureshanapartisureshanaparti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@sureshanaparti

sureshanaparti commented Jun 25, 2024

Copy link
Copy Markdown
Contributor

Merging based on the reviews & tests (these failures are not related to the changes here).

@sureshanaparti
sureshanaparti merged commit 7f0d9a0 into apache:4.19Jun 25, 2024
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Jul 2, 2024
@SadiJrSadiJr
@JoaoJandre@dhslove
[Veeam] Check for failures in the restore process (apache#7224)
c6dd7ea
* Validate failure state in Veeam restore process
* Address Daan review, and properly call method
* Address bryan's reviews
* remove return
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shwstpprshwstpprshwstppr left review comments
@JoaoJandreJoaoJandreJoaoJandre approved these changes
@weizhouapacheweizhouapacheweizhouapache left review comments
@DaanHooglandDaanHooglandDaanHoogland approved these changes
@sureshanapartisureshanapartisureshanaparti approved these changes
@BryanMLimaBryanMLimaBryanMLima approved these changes
@harikrishna-patnalaharikrishna-patnalaAwaiting requested review from harikrishna-patnala

Assignees

@weizhouapacheweizhouapache

Labels

component:agentcomponent:apicomponent:backupcomponent:buildcomponent:integrationcomponent:networking

Projects

No open projects
Apache CloudStack 4.19.1
Status: Done

Milestone

4.19.1.0

Development

Successfully merging this pull request may close these issues.

8 participants

@SadiJr@DaanHoogland@blueorangutan@shwstppr@weizhouapache@sureshanaparti@JoaoJandre@BryanMLima