Skip to content

Added configuration and Integration test to restrict public template … - #4774

Merged
nvazquez merged 16 commits into
apache:mainfrom
soreana:template-restriction
Apr 22, 2022
Merged

Added configuration and Integration test to restrict public template …#4774
nvazquez merged 16 commits into
apache:mainfrom
soreana:template-restriction

Conversation

@soreana

@soreanasoreana commented Mar 9, 2021

Copy link
Copy Markdown
Member

Description

As a cloud provider, we don't want our customers to see other templates. This pr limits template access to the domain.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

How Has This Been Tested?

To test this feature, I created two domains named Test0, Test1 and Test2, each with their respective domain admins (test0, test1 and test2).
I used cloudmonkey command to list different combination of templateFilters ( all, featured, self, selfexecutable, sharedexecutable, executable and community ) and accounts ( admin, test0, test1 and test2 ).

Pre configuration:

  1. Create three domain with their respective domain admin accounts.
  2. Register new template for each account like the following table.
  3. Follow the test cases.
OwnerTemplates
AdminU20, SystemVM
Test0U0
Test1U1
Test2U2

Test case one.

  1. Set share.public.templates to false for every domain
  2. List templates in every account. You can't see any templates except the owneres templates.

Test case two

  1. Set share.public.templates like the following table.
  2. List templates in test0 account. You should see combination of the U20, U0, U2 with different templatefilter but not U1
Ownershare.public.templates
Admintrue
Test0Doesn't matter
Test1false
Test2true

Test case three

  1. Set share.public.templates like the test case two
  2. List templates in test0 with template id. You should be ablet to see the U20, U0, U2 but empty result if you use U1 id.

I wrote this script to test this pr, you can find it in the following link. You need the cmk command and you should put admin, test1, and test2 users info in the cmk configuration file. How to run this?

  • Put account names in accounts array defined at top of the script
  • ./listTemplates.sh will list all filter for all accounts
  • ./listTemplates.sh list all templates using all possible filters for
  • ./listTemplates.sh list all templates for accounts in ('admin' 'test' 'test2') using
  • ./listTemplates.sh list all templates for using

Comment threadengine/components-api/src/main/java/com/cloud/template/TemplateManager.java Outdated
@harikrishna-patnala

Copy link
Copy Markdown
Member

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@harikrishna-patnala a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2890

@harikrishna-patnala

harikrishna-patnala commented Mar 9, 2021

Copy link
Copy Markdown
Member

@soreana, does this have any impact on upgrades, existing templates in the environment might have already shared and used across domains. Does a domain lose viewing or accessing the templates which are already used when the configuration parameter is set to true.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✖️ centos7 ✖️ centos8 ✔️ debian. SL-JID 59

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ centos7 ✖️ centos8 ✔️ debian. SL-JID 63

@soreana

Copy link
Copy Markdown
MemberAuthor

@harikrishna-patnala I didn't noticed any issue. Here is what I tested today.

  1. Set the global setting to false (templates shared between domains).
  2. Created the instance (named INSTANCE) in test2 domain using Debian template (which belongs to test1 domain)
  3. Set the global setting to false (templates shared between domains).
  4. Debian Template was removed from the template list.
  5. I tested everything related to INSTANCE including: VNC console, Migration, Stop, Start, Reboot, Snapshot
  6. I found nothing.

Let me know if you have other concerns.

Btw, while I tried to reinstall the INSTANCE I lost access to Debian template which is legitimate behaviour. See attached.

Without restriction (global setting sets to false)

Screenshot 2021-03-09 at 19 33 44

With restriction (global setting sets to true)

Screenshot 2021-03-09 at 19 33 16

@harikrishna-patnala

Copy link
Copy Markdown
Member

@soreana I have raised the question exactly like Reinstall VM you mentioned above, loosing access to the template which are already in use. May be keeping the configuration scope to Domain level will give flexibility in allowing access in these cases.

Let's hear from others too. Thanks.

@DaanHooglandDaanHoogland added this to the 4.16.0.0 milestone Mar 11, 2021
@DaanHoogland

Copy link
Copy Markdown
Contributor

Not that I want to intrude on your discussion @soreana@harikrishna-patnala , but would it make sense to add a scope to the filter:

all, featured, self, selfexecutable, sharedexecutable, executable, community, domain

@soreana

Copy link
Copy Markdown
MemberAuthor

@DaanHoogland It is nice feature. But it will not address access right issue fixed in this pr.

@soreana

soreana commented Mar 11, 2021

Copy link
Copy Markdown
MemberAuthor

@harikrishna-patnala In regards to your Reinstall VM comment. If test1 user (who owned the template) makes it private, test2 user (who used the template) will lose access to the template and he/she can't see that template in the Reinstall VM list.

As a result, with or without these changes, the test2 user might lose access to the template. It is an inherent risk in using community templates. :D

Reinstall VM before making Debian template private

Screenshot 2021-03-11 at 10 47 24

Reinstall VM after making Debian template private

Screenshot 2021-03-11 at 10 43 05

@harikrishna-patnala

Copy link
Copy Markdown
Member

Thanks @soreana for testing further. If you can add scope to domain that will be great.

@soreana

soreana commented Mar 16, 2021

Copy link
Copy Markdown
MemberAuthor

@harikrishna-patnala I updated the PR. I migrated the setting to the domain level. It is a limited code change required to do so :D. I tested that with subdomains as well. More details:

Root (Ubuntu)
|-- domain0 (Debian)
| |-- sub0 (Fedora)
| |-- sub1
|-- domain1

If you set restrict.public.template.access.to.domain to true for domain1, domain1 access to Debian and Fedora will be limited.

If you set restrict.public.template.access.to.domain to true for sub1, sub1 access to Fedora will be limited.

Let me know if you have other concerns.

P.S: I'm updating the test cases.

@nvazquez

Copy link
Copy Markdown
Contributor

Hi @soreana looks like the PR has a conflict, can you please resolve it?

@harikrishna-patnala

Copy link
Copy Markdown
Member

domain scope change looks good @soreana, please resolve the conflicts on the PR.

@soreana

Copy link
Copy Markdown
MemberAuthor

I fixed the conflict.
@nvazquez@harikrishna-patnala Sorry for the late response. As I was away for couple of weeks, I completely forgot about this.

@nvazquez

Copy link
Copy Markdown
Contributor

No problem, thanks @soreana
@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 528

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@DaanHooglandDaanHoogland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code looks generally good, needs testing and some remarks on the integration test

Comment threadtest/integration/component/test_template_access_across_domains.py Outdated
Comment threadtest/integration/component/test_template_access_across_domains.py Outdated
Comment threadtest/integration/component/test_template_access_across_domains.py Outdated
Comment threadtest/integration/component/test_template_access_across_domains.py Outdated
Comment threadtest/integration/component/test_template_access_across_domains.py Outdated
@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-1246)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46884 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4774-t1246-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 84 look OK, 4 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
ContextSuite context=TestKubernetesCluster>:teardownError72.00test_kubernetes_clusters.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_trueFailure396.74test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_falseFailure341.05test_routers_network_ops.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACLFailure539.81test_vpc_redundant.py
test_02_redundant_VPC_default_routesFailure373.90test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routersFailure549.32test_vpc_redundant.py
test_05_rvpc_multi_tiersFailure553.14test_vpc_redundant.py
test_05_rvpc_multi_tiersError553.16test_vpc_redundant.py
test_01_vpc_site2site_vpn_multiple_optionsFailure305.87test_vpc_vpn.py

@nvazquez

Copy link
Copy Markdown
Contributor

@soreana can you address the open comments on the marvin tests?

@yadvr

Copy link
Copy Markdown
Member

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with

SystemVM template(s). I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✖️ el8 ✖️ debian ✖️ suse15. SL-JID 3251

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3255

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@sureshanaparti

Copy link
Copy Markdown
Contributor

@sureshanaparti please advise if this is ready after your testing, thanks

@nvazquez@soreana test LGTM

Manually covered with the following tests, and verified the public templates access restricted to other domains when the global or the respective domain's config "share.public.templates.with.other.domains" is set to false.

  • Created domains: Domain1 and Domain2.
  • Created accounts/users: Domain1Admin and Domain2Admin.
  • Registered public templates from macchinina-admin from Admin account.
  • Registered public templates from macchinina-domain1, centos65-domain1 from Domain1Admin account.
  • Registered public templates from macchinina-domain2, centos65-domain2 from Domain2Admin account.

Tests with global level setting:
(i) The global config "share.public.templates.with.other.domains" is set to true by default, and all the public templates of domain1/domain2 are listed for admin, domain1 admin, and domain2 admin.
(ii) Set the global config "restrict.public.template.access.to.domain" to false. Verified the public templates of domain1 are not listed for domain2 admin, and the public templates of domain2 are not listed for domain1 admin. All the public templates are listed for root admin.

Tests with domain level setting:
(i) Set Domain1 config "share.public.templates.with.other.domains": true and Domain2 config "share.public.templates.with.other.domains": false
=> All the public templates are listed for root admin.

=> The public templates of domain1 are listed for domain2 admin.
=> The public templates of domain2 are not listed for domain1 admin.
(ii) Set Domain1 config "share.public.templates.with.other.domains": false and Domain2 config "share.public.templates.with.other.domains": true

=> All the public templates are listed for root admin.

=> The public templates of domain1 are not listed for domain2 admin.

=> The public templates of domain2 are listed for domain1 admin.
(iii) Set Domain1 config "share.public.templates.with.other.domains": true and Domain2 config "share.public.templates.with.other.domains": true

=> All the public templates are listed for root admin.

=> The public templates of domain1 are listed for domain2 admin

=> The public templates of domain2 are listed for domain1 admin
(iv) Set Domain1 config "share.public.templates.with.other.domains": false and Domain2 config "share.public.templates.with.other.domains": false

=> All the public templates are listed for root admin.

=> The public templates of domain1 are not listed for domain2 admin
=> The public templates of domain2 are not listed for domain1 admin

@nvazquez

Copy link
Copy Markdown
Contributor

Hi @soreana all looking good except travis that is still failing in all these cases:

==== Marvin Init Successful ====
=== TestName: test_01_check_cross_domain_template_access | Status : EXCEPTION ===
=== TestName: test_02_create_template | Status : EXCEPTION ===
=== TestName: test_03_check_subdomain_template_access | Status : EXCEPTION ===
=== TestName: test_04_check_non_public_template_access | Status : EXCEPTION ===
=== TestName: test_05_check_non_public_template_subdomain_access | Status : EXCEPTION ===
=== TestName: test_06_check_sub_public_template_sub_domain_access | Status : EXCEPTION ===
=== TestName: test_07_check_default_public_template_sub_domain_access | Status : EXCEPTION ===
=== TestName: test_08_check_non_public_template_sub_domain_access | Status : EXCEPTION ===

Function to update the global setting "restrict.public.access.to.templates" for domain
"""
update_configuration_cmd = updateConfiguration.updateConfigurationCmd()
update_configuration_cmd.name = "restrict.public.template.access.to.domain"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@soreana
since you have changed the name of global configuration to share.public.templates.with.other.domains, can you change the name in component test ? (maybe the value as well)

Comment thread.travis.yml Outdated
The test needs to be updated to use the new configuration name
@acs-robot

Copy link
Copy Markdown

Found UI changes, kicking a new UI QA build
@blueorangutan ui

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-3958)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37433 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4774-t3958-kvm-centos7.zip
Smoke tests completed. 95 look OK, 1 have errors
Only failed tests results shown below:

TestResultTime (s)Test File
test_01_nicError141.55test_nic.py

@acs-robot

Copy link
Copy Markdown

PR Coverage Report

CLASSINSTRUCTION MISSEDINSTRUCTION COVEREDBRANCH MISSEDBRANCH COVEREDLINE MISSEDLINE COVERED
Network55404201070
Volume109020440
NetworkOrchestrationService010100010
StorageManager021100018
AgentAttache1042012402190
AgentManagerImpl3463038007140
ClusteredAgentManagerImpl2361024205360
NetworkOrchestrator972201198019140
VolumeOrchestrator5170056009770
DataCenterVnetVO58000240
VlanVO186020720
AccountGuestVlanMapVO46000190
NetworkDaoImpl3307011604670
NetworkOfferingVO414310012612
SnapshotVO172581204820
VolumeVO5291334017939
SnapshotDaoImpl7680801280
Upgrade41610to4170082740232
DomainRouterDaoImpl160402202280
DirectDownloadCertificateHostMapDaoImpl121000180
DirectDownloadCertificateHostMapVO54040190
AncientDataMotionStrategy145852170433810
DefaultSnapshotStrategy7582201001016437
SnapshotDataFactoryImpl152581553713
DefaultVMSnapshotStrategy4866775230100140
ScaleIOVMSnapshotStrategy106348002241
StorageStrategyFactoryImpl377715818
SnapshotDataStoreDaoImpl16836044229213
DefaultHostListener53704001010
VolumeObject7736657739178117
LibvirtComputingResource8927196110961401992444
LibvirtVMDef37114641136
LibvirtMigrateCommandWrapper9773831583622898
LibvirtReadyCommandWrapper83722110
LibvirtRevertSnapshotCommandWrapper3871302006916
LibvirtUtilitiesHelper1155842278
IscsiAdmStorageAdaptor98105201780
IscsiAdmStoragePool129000320
KVMStoragePoolManager92647601921
KVMStorageProcessor599249844822123971
LibvirtStorageAdaptor33761925707972
LibvirtStoragePool239891596429
LinstorStorageAdaptor107605002460
ManagedNfsStorageAdaptor44602001250
ScaleIOStorageAdaptor684121751315231
KVMHostInfo1501461043038
QemuImg67406401600
MockVmManagerImpl149509003380
VmwareServerDiscoverer1586018203780
VmwareManagerImpl261352829638615117
VmwareResource2006002242043300
VmwareStorageProcessor98929940021222
XcpServerDiscoverer1554101176634521
CitrixResourceBase145665571452343160122
CitrixReadyCommandWrapper5837311511
KubernetesClusterManagerImpl4724048007600
KubernetesClusterVO2980601090
KubernetesClusterActionWorker1540011402810
KubernetesClusterResourceModifierActionWorker1843014003250
KubernetesClusterStartWorker2678016804260
ListVMsMetricsCmd39000100
MetricsServiceImpl167309803370
NetScalerControlCenterResource1943014404680
NetscalerResource68820806016230
ElastistorHostListener1500140300
DateraPrimaryDataStoreDriver3195028307480
DateraHostListener63507401360
CloudStackPrimaryDataStoreDriverImpl903011402290
LinstorPrimaryDataStoreDriverImpl144209103480
ScaleIOPrimaryDataStoreDriver2537024605370
ScaleIOHostListener1960140430
SolidFirePrimaryDataStoreDriver3347028406970
SolidFireHostListener54506001120
SolidFireSharedHostListener4070300820
SAMLUtils202465411153108
DomainChecker1206030002380
ApiDBUtils2367021005900
ApiResponseHelper1214401274027790
ParamProcessWorker1050015502410
QueryManagerImpl1423301248024030
ViewResponseHelper1662015003050
UserVmJoinDaoImpl1531018403280
VolumeJoinDaoImpl77009401710
VolumeJoinVO267000930
Config152516230642342
ConfigurationManagerImpl1810403032035700
LibvirtServerDiscoverer988011602180
IpAddressManagerImpl4045046108060
NetworkModelImpl61820838013000
NetworkServiceImpl1337201862025480
AutoScaleManagerImpl3297032006910
ConfigDriveNetworkElement1466017303060
FirewallManagerImpl2522039104550
GuestNetworkGuru622298983412464
PrivateNetworkGuru3940460880
LoadBalancingRulesManagerImpl60240666012540
NetworkHelperImpl2019026404280
RulesManagerImpl4074049207900
SecurityGroupManagerImpl2330026004970
NetworkACLServiceImpl2698030205200
VpcManagerImpl68700758013110
ResourceManagerImpl85120982016580
ConfigurationServerImpl2075017804990
ManagementServerImpl1181901052023610
StatsCollector1875010403080
StorageManagerImpl85470974017040
VolumeApiServiceImpl1085101500020350
StoragePoolMonitor42707201070
SnapshotManager10700090
SnapshotManagerImpl4186041007510
TaggedResourceManagerImpl4730580960
TemplateManagerImpl49960696010420
AccountManagerImpl64460906013510
UserVmManagerImpl2079802566038690
VMSnapshotManagerImpl3135029206230
BackupManagerImpl2776022404880
DirectDownloadManagerImpl1679018603610
UnmanagedVMsManagerImpl4881054208000
MockNetworkManagerImpl4850220830
PremiumSecondaryStorageManagerImpl77506401160
SecondaryStorageManagerImpl34941493431162332
VirtualMachineMO8953135106317198231

@nvazqueznvazquez reopened this Apr 21, 2022
@nvazquez
nvazquez merged commit debfb45 into apache:mainApr 22, 2022
nlgordon added a commit to ippathways/cloudstack that referenced this pull request Jun 29, 2022
…PR: apache#4774 in commit e94c1e2. Had to make a few changes to support 4.11 and cleanup some of the quirks.
Slair1 pushed a commit to ippathways/cloudstack that referenced this pull request Nov 27, 2022
* Pulling in the relevant parts of the global settings version of this PR: apache#4774 in commit e94c1e2. Had to make a few changes to support 4.11 and cleanup some of the quirks.
* Making the 'allow.public.user.templates' global setting allow domain and resource admins to still upload public templates. Previously it was limited to just root admins.
* One additional template filter type that includes public templates
* Fixing permissions bug where there weren't permissions to templates in child domains
* Updating comments based on code review
* Was previously allowing public, but not featured for domain admins. Added featured for domain admins. Also limited public to only when the new setting is enabled.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

10 participants

@soreana@harikrishna-patnala@blueorangutan@DaanHoogland@nvazquez@shwstppr@sureshanaparti@weizhouapache@yadvr@acs-robot