Uh oh!
There was an error while loading. Please reload this page.
nasbackup.sh: add bandwidth throttle via -b flag - #12846
Closed
jmsperu wants to merge 1 commit into
Closed
Conversation
Add -b/--bandwidth flag (MiB/s) to limit backup I/O impact on production workloads. For running VMs: uses virsh blockjob --bandwidth to throttle the QEMU push backup job per disk. For stopped VMs: uses qemu-img convert -r rate limit and ionice -c 3 (idle I/O class) to minimize impact on other VMs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@## 4.20 #12846 +/- ##
=========================================
Coverage 16.24% 16.24% Complexity 13411 13411 =========================================
Files 5664 5664 Lines 500463 500463 Branches 60779 60779 =========================================
+ Hits 81308 81312 +4 + Misses 410059 410058 -1 + Partials 9096 9093 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds optional bandwidth throttling to the KVM NAS backup script to reduce backup I/O impact during running-VM and stopped-VM backups.
Changes:
- Introduces a
-b|--bandwidthCLI option and updates usage text accordingly. - Applies bandwidth limiting to running-VM backups via
virsh blockjob --bandwidth. - Applies bandwidth limiting to stopped-VM backups via
qemu-img convertrate limiting andionice.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
+108
to
+111
| for disk in $(virsh -c qemu:///system domblklist $VM --details 2>/dev/null | awk '/disk/{print$3}'); do | ||
| virsh -c qemu:///system blockjob $VM $disk --bandwidth "${BANDWIDTH}" 2>/dev/null || true | ||
| done | ||
| log -ne "Backup bandwidth limited to ${BANDWIDTH} MiB/s per disk for $VM" |
Comment on lines
+106
to
+110
| # Throttle backup bandwidth if requested (MiB/s per disk) | ||
| if [[ -n "$BANDWIDTH" ]]; then | ||
| for disk in $(virsh -c qemu:///system domblklist $VM --details 2>/dev/null | awk '/disk/{print$3}'); do | ||
| virsh -c qemu:///system blockjob $VM $disk --bandwidth "${BANDWIDTH}" 2>/dev/null || true | ||
| done |
| for disk in $DISK_PATHS; do | ||
| volUuid="${disk##*/}" | ||
| qemu-img convert -O qcow2 $disk $dest/$name.$volUuid.qcow2 | tee -a "$logFile" | ||
| ionice -c 3 qemu-img convert $([[ -n "$BANDWIDTH" ]] && echo "-r" "${BANDWIDTH}M") -O qcow2 $disk $dest/$name.$volUuid.qcow2 | tee -a "$logFile" |
| for disk in $DISK_PATHS; do | ||
| volUuid="${disk##*/}" | ||
| qemu-img convert -O qcow2 $disk $dest/$name.$volUuid.qcow2 | tee -a "$logFile" | ||
| ionice -c 3 qemu-img convert $([[ -n "$BANDWIDTH" ]] && echo "-r" "${BANDWIDTH}M") -O qcow2 $disk $dest/$name.$volUuid.qcow2 | tee -a "$logFile" |
abh1sar
commented
Mar 24, 2026
Contributor
@jmsperu Please see #12848 (comment) |
jmsperu added a commit
to jmsperu/cloudstack
that referenced
this pull request
Mar 26, 2026
… integrity check Adds four optional features to NAS backup operations, configurable at zone scope via CloudStack global settings: - Compression (-c): qcow2 internal compression of backup files Config: nas.backup.compression.enabled (default: false) - LUKS Encryption (-e): encrypt backup files at rest using qemu-img Config: nas.backup.encryption.enabled (default: false) Config: nas.backup.encryption.passphrase (Secure category) - Bandwidth Throttle (-b): limit backup I/O bandwidth via virsh blockjob for running VMs or qemu-img -r for stopped VMs Config: nas.backup.bandwidth.limit.mbps (default: 0/unlimited) - Integrity Check (--verify): qemu-img check after backup creation Config: nas.backup.integrity.check (default: false) All features are disabled by default and fully backward compatible. Settings are read from zone-scoped ConfigKeys in NASBackupProvider, passed to the KVM agent via TakeBackupCommand details map, and translated to nasbackup.sh CLI flags in LibvirtTakeBackupCommandWrapper. Changes: - nasbackup.sh: add -c, -b, -e, --verify flags with encrypt_backup() and verify_backup() helper functions - TakeBackupCommand.java: add details map for passing config to agent - NASBackupProvider.java: add 5 ConfigKeys, populate command details - LibvirtTakeBackupCommandWrapper.java: extract details, build CLI args, handle passphrase temp file lifecycle Combines and supersedes PRs apache#12844, apache#12846, apache#12848, apache#12845
9 tasks
jmsperu
commented
Mar 26, 2026
CollaboratorAuthor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
-b/--bandwidthflag (MiB/s) to limit backup I/O impact on production workloadsvirsh blockjob --bandwidthper diskqemu-img convert -rrate limit +ionice -c 3(idle I/O class)Motivation
Backup jobs on shared storage can saturate I/O bandwidth and degrade performance for production VMs on the same host or NAS. This is especially problematic during business hours or when multiple VMs are being backed up concurrently.
The bandwidth limit can be passed by the CloudStack agent based on a global or per-repository setting, giving admins control over backup impact.
Test plan
-b— verify no throttling, identical to current behavior-b 50— verifyblockjob --bandwidthapplied, checkdomjobinfoshows throttled rate-b 50— verifyionice -c 3and-r 50Min process list