Add support for providing userdata to system VMs by vishesh92 · Pull Request #11654 · apache/cloudstack

vishesh92 · 2025-09-17T08:25:03Z

Description

This PR adds support for specifying user data (cloud-init) for system VMs via Zone Scoped global settings. This allows the operators to customize the System VMs and setup monitoring, logging or execute any custom commands.

We set the user data from the global setting in /var/cache/cloud/cmdline, and use the NoCloud datasource to process user data. cloud-init service is still disabled in the system VMs and it's executed as part of the cloud-postinit service which executes the postinit.sh script.

Added global settings:
systemvm.userdata.enabled - Disabled by default. Needs to be enabled to utilize the feature.
console.proxy.vm.userdata - UUID of the User data to be used for Console Proxy
secstorage.vm.userdata - UUID of the User data to be used for Secondary Storage VM
virtual.router.userdata - UUID of the User data to be used for Virtual Routers

Docs PR: apache/cloudstack-documentation#567

Generated summary

This pull request introduces support for providing user data to system VMs (Console Proxy VM, Secondary Storage VM, and Virtual Router) via global settings, enabling cloud-init configuration through user-supplied data. The changes include new configuration keys, logic to inject user data into VM boot arguments (encoded in base64), and updates to the system VM template to better support cloud-init.

System VM user data support:

Added systemvm.userdata.enabled config key to globally enable user data for system VMs, and made it available in the VirtualMachineManager and related implementations. [1][2]
Introduced new config keys for user data:
- consoleproxy.userdata for Console Proxy VMs
- secstorage.userdata for Secondary Storage VMs
- router.userdata for Virtual Routers
Updated getConfigKeys() methods in relevant managers to include the new config keys. [1][2][3]

User data injection logic:

In each system VM manager (ConsoleProxyManagerImpl, SecondaryStorageManagerImpl, VirtualNetworkApplianceManagerImpl), added logic to:
- Check if user data is enabled for the zone and, if so, retrieve and base64-encode the configured user data string.
- Inject the encoded user data into the VM's boot arguments as userdata=.... [1][2][3]

System VM template / cloud-init configuration:

Updated configure_systemvm_services.sh to change the default cloud-init configuration, switching the data source to NoCloud and disabling automatic cloud-init services to allow manual control by CloudStack.

These changes collectively enable administrators to inject custom cloud-init user data into system VMs via global settings, improving flexibility and automation for system VM initialization.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

vishesh92 · 2025-09-17T08:27:50Z

@blueorangutan package

blueorangutan · 2025-09-17T08:30:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-09-17T08:33:15Z

Codecov Report

❌ Patch coverage is 60.33058% with 48 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.54%. Comparing base (3159fa7) to head (422e448).
⚠️ Report is 22 commits behind head on main.

Files with missing lines	Patch %	Lines
.../network/lb/InternalLoadBalancerVMManagerImpl.java	0.00%	10 Missing ⚠️
...om/cloud/consoleproxy/ConsoleProxyManagerImpl.java	0.00%	10 Missing ⚠️
.../secondarystorage/SecondaryStorageManagerImpl.java	0.00%	10 Missing ⚠️
...oud/network/lb/ElasticLoadBalancerManagerImpl.java	0.00%	9 Missing ⚠️
...ork/router/VirtualNetworkApplianceManagerImpl.java	0.00%	9 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #11654 +/- ##
============================================
+ Coverage 17.50% 17.54% +0.03% - Complexity 15426 15465 +39 
============================================
Files 5894 5897 +3 Lines 526847 527340 +493 Branches 64335 64390 +55 ============================================
+ Hits 92232 92516 +284 - Misses 424238 424429 +191 - Partials 10377 10395 +18

Flag	Coverage Δ
uitests	`3.60% <ø> (-0.01%)`	⬇️
unittests	`18.60% <60.33%> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2025-09-17T10:04:39Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15055

Copilot

Pull Request Overview

This PR adds support for providing user data (cloud-init) to system VMs via Zone Scoped global settings, enabling operators to customize Console Proxy VMs, Secondary Storage VMs, and Virtual Routers with monitoring, logging, or custom commands.

Added new global settings to enable user data feature and configure user data for each system VM type
Implemented user data injection logic in system VM managers to encode and append user data to boot arguments
Updated system VM template cloud-init configuration to use NoCloud datasource and disable automatic cloud-init services

Reviewed Changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
engine/api/src/main/java/com/cloud/vm/VirtualMachineManager.java	Added global setting to enable user data for system VMs
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java	Updated config keys array to include the new system VM user data setting
server/src/main/java/com/cloud/consoleproxy/ConsoleProxyManager.java	Added configuration key for console proxy user data
server/src/main/java/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java	Implemented user data injection logic for console proxy VMs
server/src/main/java/com/cloud/network/router/VirtualNetworkApplianceManager.java	Added configuration key for virtual router user data
server/src/main/java/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java	Implemented user data injection logic for virtual router VMs
server/src/main/java/com/cloud/storage/secondary/SecondaryStorageVmManager.java	Added configuration key for secondary storage user data
services/secondary-storage/controller/src/main/java/org/apache/cloudstack/secondarystorage/SecondaryStorageManagerImpl.java	Implemented user data injection logic for secondary storage VMs
tools/appliance/systemvmtemplate/scripts/configure_systemvm_services.sh	Updated cloud-init configuration to use NoCloud datasource and disabled automatic services

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

vishesh92 · 2025-09-17T10:34:31Z

@blueorangutan package

blueorangutan · 2025-09-17T10:36:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-09-17T12:31:04Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✖️ debian ✔️ suse15. SL-JID 15057

vishesh92 · 2025-09-17T16:54:15Z

@blueorangutan package

blueorangutan · 2025-09-17T16:56:04Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-09-17T18:29:27Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15064

harikrishna-patnala

@vishesh92 is it a good idea to add some validations to these userdata configs, to avoid any kind of security risks ?

vishesh92 · 2025-09-18T08:41:24Z

@vishesh92 is it a good idea to add some validations to these userdata configs, to avoid any kind of security risks ?

IMO, it would be difficult to do that. Technically the user data could be a script which fetches a binary (malicious) from internet and executes it on the system VM and we won't be able to validate this.
But since the global settings are scoped at zone level, only the operators can set the user data.

I have also added a new global setting, systemvm.userdata.enabled, which is set to false by default. Unless this is set to true, the other global settings are ignored.

vishesh92 · 2025-09-30T10:37:09Z

@blueorangutan package

blueorangutan · 2025-09-30T10:38:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-09-30T11:59:11Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15235

borisstoyanov · 2025-09-30T12:24:36Z

@blueorangutan test matrix

blueorangutan · 2025-09-30T12:26:04Z

@borisstoyanov a [SL] Trillian-Jenkins matrix job (EL8 mgmt + EL8 KVM, Ubuntu22 mgmt + Ubuntu22 KVM, EL8 mgmt + VMware 7.0u3, EL9 mgmt + XCP-ng 8.2 ) has been kicked to run smoke tests

blueorangutan · 2025-10-01T04:45:44Z

[SF] Trillian test result (tid-14482)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 55364 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11654-t14482-kvm-ol8.zip
Smoke tests completed. 147 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_enableHumanReadableLogs	`Error`	0.32	test_human_readable_logs.py

blueorangutan · 2025-10-01T05:11:45Z

[SF] Trillian test result (tid-14483)
Environment: kvm-ubuntu22 (x2), zone: Advanced Networking with Mgmt server u22
Total time taken: 57645 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11654-t14483-kvm-ubuntu22.zip
Smoke tests completed. 147 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_enableHumanReadableLogs	`Error`	0.18	test_human_readable_logs.py

blueorangutan · 2025-10-01T08:33:38Z

[SF] Trillian test result (tid-14484)
Environment: vmware-70u3 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 69517 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11654-t14484-vmware-70u3.zip
Smoke tests completed. 147 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	3679.89	test_kubernetes_clusters.py

blueorangutan · 2025-10-01T11:58:10Z

[SF] Trillian test result (tid-14485)
Environment: xcpng82 (x2), zone: Advanced Networking with Mgmt server ol9
Total time taken: 82150 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11654-t14485-xcpng82.zip
Smoke tests completed. 142 look OK, 6 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	235.80	test_events_resource.py
test_01_non_strict_host_anti_affinity	`Error`	218.18	test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity	`Error`	82.12	test_nonstrict_affinity_group.py
test_01_primary_storage_iscsi	`Error`	0.87	test_primary_storage.py
test_01_vpn_usage	`Error`	1.15	test_usage.py
test_11_destroy_vm_and_volumes	`Error`	26.36	test_vm_life_cycle.py
test_01_migrate_vm_strict_tags_success	`Error`	73.84	test_vm_strict_host_tags.py

borisstoyanov

LGTM, the Xen failures does not seem related

harikrishna-patnala · 2025-10-07T07:58:12Z

@blueorangutan package

blueorangutan · 2025-10-07T08:00:04Z

@harikrishna-patnala a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-10-07T09:14:46Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15315

vishesh92 · 2025-10-07T09:17:15Z

@blueorangutan test

blueorangutan · 2025-10-07T09:18:04Z

@vishesh92 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-10-08T01:12:09Z

[SF] Trillian test result (tid-14552)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 54188 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11654-t14552-kvm-ol8.zip
Smoke tests completed. 149 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

This PR adds support for specifying user data (cloud-init) for system VMs via Zone Scoped global settings. This allows the operators to customize the System VMs and setup monitoring, logging or execute any custom commands. We set the user data from the global setting in /var/cache/cloud/cmdline, and use the NoCloud datasource to process user data. cloud-init service is still disabled in the system VMs and it's executed as part of the cloud-postinit service which executes the postinit.sh script. Added global settings: systemvm.userdata.enabled - Disabled by default. Needs to be enabled to utilize the feature. console.proxy.vm.userdata - UUID of the User data to be used for Console Proxy secstorage.vm.userdata - UUID of the User data to be used for Secondary Storage VM virtual.router.userdata - UUID of the User data to be used for Virtual Routers

boring-cyborgBot added component:compute component:networking component:orchestration component:secondary-storage component:ssvm component:storage component:templates component:virtual-router systemvm-template-change labels Sep 17, 2025

vishesh92 commented Sep 17, 2025
View reviewed changes

Comment threadtools/appliance/systemvmtemplate/scripts/configure_systemvm_services.sh Outdated

vishesh92 commented Sep 17, 2025
View reviewed changes

Comment threadsystemvm/debian/opt/cloud/bin/setup/postinit.sh Outdated

vishesh92 commented Sep 17, 2025
View reviewed changes

Comment threadtools/appliance/systemvmtemplate/scripts/configure_systemvm_services.sh Outdated

vishesh92 requested a review from Copilot September 17, 2025 10:14

CopilotAI reviewed Sep 17, 2025
View reviewed changes

Comment threadtools/appliance/systemvmtemplate/scripts/configure_systemvm_services.sh Outdated

vishesh92 force-pushed the userdata-support-systemvms branch from 9cc8655 to 7b6881bCompare September 17, 2025 10:25

apache deleted a comment from blueorangutan Sep 17, 2025

vishesh92 force-pushed the userdata-support-systemvms branch from 7b6881b to f5eeca0Compare September 17, 2025 12:10

weizhouapache reviewed Sep 17, 2025
View reviewed changes

Comment threadsystemvm/debian/opt/cloud/bin/setup/postinit.sh
Comment threadtools/appliance/systemvmtemplate/scripts/configure_systemvm_services.sh
Comment threadsystemvm/debian/opt/cloud/bin/setup/postinit.sh

harikrishna-patnala reviewed Sep 18, 2025
View reviewed changes

vishesh92 added 4 commits September 30, 2025 16:04

minor code refactoring
2a4a281

Add unit tests
2cc1772

Add smoke test for ssvm
219d4bf

Add tests userdata in cpvm & vr
d769a9b

vishesh92 force-pushed the userdata-support-systemvms branch from 56f60ec to d769a9bCompare September 30, 2025 10:34

github-actionsBot removed the status:has-conflicts label Sep 30, 2025

apache deleted a comment from blueorangutan Sep 30, 2025

weizhouapache mentioned this pull request Oct 1, 2025
Shared Filesystem support on Config Drive Networks #11726
Merged
14 tasks

borisstoyanov approved these changes Oct 2, 2025
View reviewed changes

Change global settings name
422e448

harikrishna-patnala merged commit d2615bb into apache:mainOct 8, 2025
28 checks passed

harikrishna-patnala deleted the userdata-support-systemvms branch October 8, 2025 05:14

harikrishna-patnala mentioned this pull request Nov 19, 2025
[WIP] Centralized logging capability. Follow up on the #4108 #5296
Closed
7 tasks

Uh oh!

Conversation

vishesh92 commented Sep 17, 2025• edited by harikrishna-patnala Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

vishesh92 commented Sep 17, 2025

Uh oh!

blueorangutan commented Sep 17, 2025

Uh oh!

codecovBot commented Sep 17, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Sep 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

vishesh92 commented Sep 17, 2025

Uh oh!

blueorangutan commented Sep 17, 2025

Uh oh!

blueorangutan commented Sep 17, 2025

Uh oh!

vishesh92 commented Sep 17, 2025

Uh oh!

blueorangutan commented Sep 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blueorangutan commented Sep 17, 2025

Uh oh!

harikrishna-patnala left a comment

Choose a reason for hiding this comment

Uh oh!

vishesh92 commented Sep 18, 2025

Uh oh!

vishesh92 commented Sep 30, 2025

Uh oh!

blueorangutan commented Sep 30, 2025

Uh oh!

blueorangutan commented Sep 30, 2025

Uh oh!

borisstoyanov commented Sep 30, 2025

Uh oh!

blueorangutan commented Sep 30, 2025

Uh oh!

blueorangutan commented Oct 1, 2025

Uh oh!

blueorangutan commented Oct 1, 2025

Uh oh!

blueorangutan commented Oct 1, 2025

Uh oh!

blueorangutan commented Oct 1, 2025

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

harikrishna-patnala commented Oct 7, 2025

Uh oh!

blueorangutan commented Oct 7, 2025

Uh oh!

blueorangutan commented Oct 7, 2025

Uh oh!

vishesh92 commented Oct 7, 2025

vishesh92 commented Sep 17, 2025•
edited by harikrishna-patnala
Loading

codecovBot commented Sep 17, 2025•
edited
Loading