Add config keys for controlling public/private template secondary storage replica counts by Damans227 · Pull Request #12877 · apache/cloudstack

Damans227 · 2026-03-23T16:12:29Z

Description

Adds two new operator-level configuration keys to control the number of secondary storage copies made for public and private templates, decoupling replica count from template visibility.

secstorage.public.template.copy.max (default: 0 = all stores, preserving existing behavior)
secstorage.private.template.copy.max (default: 1, preserving existing behavior)

Doc PR: apache/cloudstack-documentation#643

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

How Has This Been Tested?

Unit tests in HypervisorTemplateAdapterTest covering the new replica-limit logic in isZoneAndImageStoreAvailable.
Manually verified on a zone with multiple secondary storage pools.

…rage replica counts

codecov · 2026-03-23T16:22:13Z

Codecov Report

❌ Patch coverage is 21.39535% with 169 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.94%. Comparing base (82986f6) to head (655ce0c).

Files with missing lines	Patch %	Lines
.../cloudstack/storage/image/TemplateServiceImpl.java	20.49%	121 Missing and 7 partials ⚠️
...n/java/com/cloud/template/TemplateManagerImpl.java	0.00%	26 Missing ⚠️
...com/cloud/storage/ImageStoreUploadMonitorImpl.java	0.00%	6 Missing ⚠️
.../com/cloud/template/HypervisorTemplateAdapter.java	37.50%	5 Missing ⚠️
...n/java/com/cloud/template/TemplateAdapterBase.java	66.66%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #12877 +/- ##
============================================
+ Coverage 3.51% 18.94% +15.42% - Complexity 0 18373 +18373 
============================================
Files 469 6192 +5723 Lines 40471 556550 +516079 Branches 7615 67954 +60339 ============================================
+ Hits 1424 105431 +104007 - Misses 38856 439541 +400685 - Partials 191 11578 +11387

Flag	Coverage Δ
uitests	`3.51% <ø> (ø)`
unittests	`20.15% <21.39%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…vate templates

Damans227 · 2026-03-23T16:48:59Z

@blueorangutan package

blueorangutan · 2026-03-23T16:50:05Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-03-23T17:33:35Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17224

Copilot

Pull request overview

This PR introduces new template-related configuration keys intended to control how many secondary storage replicas are created for public vs private templates, decoupling replication behavior from template visibility.

Changes:

Adds two new Zone-scoped ConfigKey<Integer> settings: public.template.secstorage.copy and private.template.secstorage.copy.
Reworks secondary storage allocation logic to enforce a per-zone replica limit using a zone→count map.
Updates unit tests to match the new method signatures/replica-limit behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`engine/components-api/src/main/java/com/cloud/template/TemplateManager.java`	Adds new Zone-scoped config keys for public/private template replica limits.
`server/src/main/java/com/cloud/template/TemplateManagerImpl.java`	Registers the new config keys via `getConfigKeys()`.
`server/src/main/java/com/cloud/template/TemplateAdapterBase.java`	Replaces zone-set visibility logic with zone copy counting + replica limit enforcement.
`server/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java`	Threads replica-limit and zone copy count through template creation/allocation paths.
`server/src/test/java/com/cloud/template/HypervisorTemplateAdapterTest.java`	Updates tests for the new signatures and adds coverage for replica-limit behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-25T16:17:19Z

 /**
 * If the template/ISO is marked as private, then it is allocated to a random secondary storage; otherwise, allocates to every storage pool in every zone given by the
 * {@link TemplateProfile#getZoneIdList()}.
 */
 protected void postUploadAllocation(List<DataStore> imageStores, VMTemplateVO template, List<TemplateOrVolumePostUploadCommand> payloads) {


This method JavaDoc no longer matches the behavior: allocation is now governed by replica-limit config keys (public/private can both be limited), not strictly by template visibility (private=random vs public=all). Please update the comment to reflect the new replica-limit semantics (and that the limits are zone-scoped).

Copilot · 2026-03-25T16:17:20Z

+ ConfigKey<Integer> PublicTemplateSecStorageCopy = new ConfigKey<Integer>("Advanced", Integer.class,
+ PublicTemplateSecStorageCopyCK, "0",
+ "Maximum number of secondary storage pools to which a public template is copied. " +
+ "0 means copy to all secondary storage pools (default behavior).",
+ true, ConfigKey.Scope.Zone);
+
+ ConfigKey<Integer> PrivateTemplateSecStorageCopy = new ConfigKey<Integer>("Advanced", Integer.class,
+ PrivateTemplateSecStorageCopyCK, "1",
+ "Maximum number of secondary storage pools to which a private template is copied. " +
+ "Default is 1 to preserve existing behavior.",
+ true, ConfigKey.Scope.Zone);


The config key descriptions don’t mention that the limits are evaluated per-zone (and the keys themselves are zone-scoped). Consider clarifying the wording (e.g., “per zone”) so operators understand how replica limits are applied across multiple zones/image stores.

…c values

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Damans227 · 2026-04-15T11:46:44Z

@blueorangutan package

blueorangutan · 2026-04-15T11:48:04Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-15T12:47:54Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17501

sureshanaparti · 2026-04-27T12:35:10Z

@Damans227 update the description with update config names.

sureshanaparti

clgtm

Damans227 · 2026-04-27T13:40:36Z

@Damans227 update the description with update config names.

Done. Thanks!

Damans227 · 2026-05-27T15:22:50Z

@blueorangutan package

blueorangutan · 2026-05-27T15:24:06Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-05-27T16:14:12Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18063

…apacity during template management

Damans227 · 2026-05-28T12:42:56Z

@blueorangutan package

blueorangutan · 2026-05-28T12:44:05Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-05-28T13:37:03Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18078

kiranchavala · 2026-06-08T13:01:16Z

@blueorangutan test

blueorangutan · 2026-06-08T13:02:03Z

@kiranchavala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-06-08T13:57:38Z

[SF] Trillian Build Failed (tid-16260)

kiranchavala · 2026-06-11T07:00:49Z

@blueorangutan test

blueorangutan · 2026-06-11T07:02:03Z

@kiranchavala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-06-11T21:44:38Z

[SF] Trillian test result (tid-16286)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49957 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12877-t16286-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

kiranchavala

LGTM

Tested the feature manually.

The Public and private Templates honor the global setting value of secstorage.public.template.copy.max and secstorage.private.template.copy.max and copies the template to the respective secondary storage.

validate secstorage.public.template.copy.max at global and zone level	Pass
validate secstorage.private.template.copy.max at global and zone level	Pass
Validate publlic and private template creation operation from volume	Pass
Validate publlic and private template creation operation from a volume snapshot	Pass
Validate Upload Template operation from Local	Pass
Validate Copy Template operation from one zone to another	Pass
Validate Update Template operation from public to private and vice versa	Pass

weizhouapache · 2026-06-12T07:40:10Z

LGTM
Tested the feature manually.
The Public and private Templates honor the global setting value of secstorage.public.template.copy.max and secstorage.private.template.copy.max and copies the template to the respective secondary storage.
validate secstorage.public.template.copy.max at global and zone level Pass
validate secstorage.private.template.copy.max at global and zone level Pass
Validate publlic and private template creation operation from volume Pass
Validate publlic and private template creation operation from a volume snapshot Pass
Validate Upload Template operation from Local Pass
Validate Copy Template operation from one zone to another Pass
Validate Update Template operation from public to private and vice versa Pass

great, thanks @kiranchavala

@harikrishna-patnala
will you review the feature before merge ?

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-06-24T11:15:34Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

Damans227 · 2026-06-24T11:57:13Z

@blueorangutan package

blueorangutan · 2026-06-24T11:58:08Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-06-24T13:05:31Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18364

rajujith

LGTM
Tested.

weizhouapache · 2026-06-29T06:12:34Z

LGTM Tested.

thanks @rajujith @kiranchavala @Damans227

merging

Add config keys for controlling public/private template secondary sto…
40b9e4d
…rage replica counts

Daman Arora added 2 commits March 23, 2026 12:36

Allow configurable secondary storage replica count for public and pri…
f997444
…vate templates

Add unit tests
0708236

DaanHoogland requested a review from Copilot March 25, 2026 16:12

Copilotstarted reviewing on behalf of DaanHoogland March 25, 2026 16:13 View session

CopilotAI reviewed Mar 25, 2026
View reviewed changes

winterhazel added this to the 4.23.0 milestone Mar 28, 2026

Damans227 mentioned this pull request Apr 15, 2026
Add documentation for Template replication on secondary storage apache/cloudstack-documentation#643
Open

Refactor secondary storage replica limit handling to use zone-specifi…
e5bfeab
…c values

Damans227 requested a review from sureshanaparti April 15, 2026 09:06

sureshanaparti reviewed Apr 15, 2026
View reviewed changes

Comment threadengine/components-api/src/main/java/com/cloud/template/TemplateManager.java Outdated

sureshanaparti reviewed Apr 15, 2026
View reviewed changes

Comment threadengine/components-api/src/main/java/com/cloud/template/TemplateManager.java Outdated

sureshanaparti reviewed Apr 15, 2026
View reviewed changes

Comment threadserver/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java Outdated

sureshanaparti reviewed Apr 15, 2026
View reviewed changes

Comment threadserver/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java Outdated

sureshanaparti requested a review from Copilot April 15, 2026 09:24

Copilotstarted reviewing on behalf of sureshanaparti April 15, 2026 09:25 View session

CopilotAI reviewed Apr 15, 2026
View reviewed changes

Comment threadserver/src/main/java/com/cloud/template/TemplateAdapterBase.java Outdated

Refactor as per pr comments
6dbeb52

Damans227 requested a review from sureshanaparti April 16, 2026 06:33

sureshanaparti approved these changes Apr 27, 2026
View reviewed changes

sureshanaparti added the status:needs-testing label Apr 27, 2026

Enforce secondary storage copy limits and replicate templates up to c…
f1e5884
…apacity during template management

kiranchavala approved these changes Jun 12, 2026
View reviewed changes

weizhouapache added status:ready-for-merge and removed status:needs-testing labels Jun 12, 2026

weizhouapache marked this pull request as ready for review June 24, 2026 09:16

CopilotAI review requested due to automatic review settings June 24, 2026 09:16

CopilotAI reviewed Jun 24, 2026

github-actionsBot added the status:has-conflicts label Jun 24, 2026

Merge branch 'main' into feature/FR284-template-secstorage-copy-control
655ce0c

github-actionsBot removed the status:has-conflicts label Jun 24, 2026

rajujith approved these changes Jun 29, 2026
View reviewed changes

weizhouapache merged commit 5ed4894 into apache:mainJun 29, 2026
24 of 27 checks passed

Uh oh!

Conversation

Damans227 commented Mar 23, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

How Has This Been Tested?

Uh oh!

codecovBot commented Mar 23, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Damans227 commented Mar 23, 2026

Uh oh!

blueorangutan commented Mar 23, 2026

Uh oh!

blueorangutan commented Mar 23, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

CopilotAIMar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

CopilotAIMar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Damans227 commented Apr 15, 2026

Uh oh!

blueorangutan commented Apr 15, 2026

Uh oh!

blueorangutan commented Apr 15, 2026

Uh oh!

sureshanaparti commented Apr 27, 2026

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

Damans227 commented Apr 27, 2026

Uh oh!

Damans227 commented May 27, 2026

Uh oh!

blueorangutan commented May 27, 2026

Uh oh!

blueorangutan commented May 27, 2026

Uh oh!

Damans227 commented May 28, 2026

Uh oh!

blueorangutan commented May 28, 2026

Uh oh!

blueorangutan commented May 28, 2026

Uh oh!

kiranchavala commented Jun 8, 2026

Uh oh!

blueorangutan commented Jun 8, 2026

Uh oh!

blueorangutan commented Jun 8, 2026

Uh oh!

kiranchavala commented Jun 11, 2026

Uh oh!

blueorangutan commented Jun 11, 2026

Uh oh!

blueorangutan commented Jun 11, 2026

Uh oh!

Damans227 commented Mar 23, 2026•
edited
Loading

codecovBot commented Mar 23, 2026•
edited
Loading