Fix exceeding of resource limits with powerflex by vishesh92 · Pull Request #9008 · apache/cloudstack

vishesh92 · 2024-04-30T10:22:47Z

Description

Because we have the condition that a powerflex/scaleio volume can be a size /8 the user can exploit this and request a smaller number without exceeding the limit, but eventually PF will deploy a volume of size [req-size ]< and /8, thus allowing the user to break the limit

STEPS TO REPRODUCE

Set a tag limit of 6GB
Deploy a volume with the offering of the tag limit and override the size to '5'
Cloudstack will consider correctly that request is within the tag limit 5<6, but will eventually deploy a volume with 8 GB exceeding the limit
After syncing resource counts this the error becomes evident.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

vishesh92 · 2024-04-30T10:34:37Z

@blueorangutan package

sureshanaparti

clgtm

codecov-commenter · 2024-04-30T10:45:28Z

Codecov Report

Attention: Patch coverage is 11.11111% with 96 lines in your changes are missing coverage. Please review.

Project coverage is 15.28%. Comparing base (e9ff270) to head (09f188e).

❗ Current head 09f188e differs from pull request most recent head f9e96af. Consider uploading reports for the commit f9e96af to get more accurate results

Files	Patch %	Lines
.../src/main/java/com/cloud/vm/UserVmManagerImpl.java	12.00%	35 Missing and 9 partials ⚠️
...stack/engine/orchestration/VolumeOrchestrator.java	0.00%	26 Missing ⚠️
...n/java/com/cloud/storage/VolumeApiServiceImpl.java	0.00%	11 Missing ⚠️
.../cloud/resourcelimit/ResourceLimitManagerImpl.java	0.00%	10 Missing ⚠️
.../subsystem/api/storage/PrimaryDataStoreDriver.java	0.00%	3 Missing ⚠️
...n/java/com/cloud/vm/VirtualMachineManagerImpl.java	0.00%	1 Missing ⚠️
...atastore/driver/ScaleIOPrimaryDataStoreDriver.java	85.71%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #9008 +/- ##
============================================
+ Coverage 4.26% 15.28% +11.01% - Complexity 0 11528 +11528 
============================================
Files 363 5425 +5062 Lines 29565 474074 +444509 Branches 5190 61618 +56428 ============================================
+ Hits 1262 72462 +71200 - Misses 28160 393553 +365393 - Partials 143 8059 +7916

Flag	Coverage Δ
uitests	`4.26% <ø> (ø)`
unittests	`16.01% <11.11%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sureshanaparti · 2024-04-30T13:15:18Z

@blueorangutan package

blueorangutan · 2024-04-30T13:16:03Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-04-30T14:33:57Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9487

vishesh92 · 2024-04-30T16:12:02Z

@blueorangutan test

blueorangutan · 2024-04-30T16:14:03Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

vishesh92 · 2024-05-01T06:07:36Z

@blueorangutan package

blueorangutan · 2024-05-01T06:08:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-05-01T06:55:08Z

[SF] Trillian test result (tid-10100)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 51007 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9008-t10100-kvm-centos7.zip
Smoke tests completed. 126 look OK, 5 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	435.40	test_events_resource.py
test_create_pvlan_network	`Error`	0.07	test_pvlan.py
test_01_restore_vm	`Error`	0.20	test_restore_vm.py
test_02_restore_vm_with_disk_offering	`Error`	0.14	test_restore_vm.py
test_03_restore_vm_with_disk_offering_custom_size	`Error`	0.14	test_restore_vm.py
test_04_restore_vm_allocated_root	`Error`	0.14	test_restore_vm.py
ContextSuite context=TestRestoreVM>:teardown	`Error`	1.24	test_restore_vm.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	390.50	test_vpc_redundant.py
test_01_redundant_vpc_site2site_vpn	`Failure`	442.61	test_vpc_vpn.py

blueorangutan · 2024-05-01T07:31:39Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9495

vishesh92 · 2024-05-02T13:21:15Z

@blueorangutan test

blueorangutan · 2024-05-02T13:22:03Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-05-02T13:57:09Z

[SF] Trillian Build Failed (tid-10140)

vishesh92 · 2024-05-02T19:54:38Z

@blueorangutan test

blueorangutan · 2024-05-02T19:56:04Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-05-03T11:16:47Z

[SF] Trillian test result (tid-10148)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 53386 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9008-t10148-kvm-centos7.zip
Smoke tests completed. 127 look OK, 4 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	432.41	test_events_resource.py
ContextSuite context=TestRestoreVM>:setup	`Error`	0.00	test_restore_vm.py
test_02_trigger_shutdown	`Failure`	346.72	test_safe_shutdown.py
test_02_list_cpvm_vm	`Failure`	0.05	test_ssvm.py
test_04_cpvm_internals	`Failure`	0.05	test_ssvm.py
test_12_destroy_cpvm	`Error`	3.19	test_ssvm.py

vishesh92 · 2024-05-07T05:08:06Z

@blueorangutan package

blueorangutan · 2024-05-07T05:10:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-05-07T06:15:31Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 9555

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

vishesh92 · 2024-05-07T07:52:15Z

@blueorangutan package

blueorangutan · 2024-05-07T07:54:05Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-05-07T09:13:59Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9559

borisstoyanov · 2024-05-07T09:45:49Z

@blueorangutan test keepEnv

blueorangutan · 2024-05-07T09:48:03Z

@borisstoyanov a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-05-08T00:34:41Z

[SF] Trillian test result (tid-10180)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 51401 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9008-t10180-kvm-centos7.zip
Smoke tests completed. 130 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	418.47	test_events_resource.py

borisstoyanov

LGTM, manually checked that the limit is honoured when producing larger volume with Powerflex

* Fix exceeding of resource limits with powerflex * Add e2e tests * Update server/src/main/java/com/cloud/vm/UserVmManagerImpl.java Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com> * fixup --------- Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

boring-cyborgBot added component:api component:compute component:integration-test component:marvin component:orchestration component:primary-storage component:storage component:vpc Python Warning... Python code Ahead! labels Apr 30, 2024

vishesh92 changed the title ~~Fixup limits exceed scaleio~~Fixup resource limits exceeding with scaleioApr 30, 2024

vishesh92 changed the title ~~Fixup resource limits exceeding with scaleio~~Fix exceeding of resource limits with powerflexApr 30, 2024

vishesh92 force-pushed the fixup-limits-exceed-scaleio branch 2 times, most recently from 09183d3 to b3b53a7Compare April 30, 2024 10:29

sureshanaparti reviewed Apr 30, 2024
View reviewed changes

Comment threadserver/src/main/java/com/cloud/vm/UserVmManagerImpl.java Outdated

sureshanaparti approved these changes Apr 30, 2024
View reviewed changes

vishesh92 force-pushed the fixup-limits-exceed-scaleio branch from dc000a9 to 09f188eCompare May 7, 2024 05:05

apache deleted a comment from blueorangutan May 7, 2024

vishesh92and others added 4 commits May 7, 2024 13:18

Fix exceeding of resource limits with powerflex
33ed96c

Add e2e tests
a73654b

Update server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
a9c6a8d
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

fixup
f9e96af

vishesh92 force-pushed the fixup-limits-exceed-scaleio branch from 09f188e to f9e96afCompare May 7, 2024 07:51

borisstoyanov approved these changes May 8, 2024
View reviewed changes

yadvr marked this pull request as ready for review May 8, 2024 15:24

yadvr merged commit 21af134 into apache:mainMay 8, 2024

yadvr deleted the fixup-limits-exceed-scaleio branch May 8, 2024 15:25

yadvr added this to the 4.20.0.0 milestone May 8, 2024

weizhouapache mentioned this pull request Aug 5, 2024
Build: drop EL7 support, support JRE17 for packages and sonar check #8609
Merged
13 tasks

This was referenced Jul 16, 2025
Fix to create instances with smaller templates (< 1 GB) on PowerFlex/ScaleIO storage #11211
Merged
Unable to create instance on PowerFlex storage (with smaller templates) #10981
Closed

Uh oh!

Conversation

vishesh92 commented Apr 30, 2024

Description

STEPS TO REPRODUCE

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

vishesh92 commented Apr 30, 2024

Uh oh!

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Apr 30, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sureshanaparti commented Apr 30, 2024

Uh oh!

blueorangutan commented Apr 30, 2024

Uh oh!

blueorangutan commented Apr 30, 2024

Uh oh!

vishesh92 commented Apr 30, 2024

Uh oh!

blueorangutan commented Apr 30, 2024

Uh oh!

vishesh92 commented May 1, 2024

Uh oh!

blueorangutan commented May 1, 2024

Uh oh!

blueorangutan commented May 1, 2024

Uh oh!

blueorangutan commented May 1, 2024

Uh oh!

vishesh92 commented May 2, 2024

Uh oh!

blueorangutan commented May 2, 2024

Uh oh!

blueorangutan commented May 2, 2024

Uh oh!

vishesh92 commented May 2, 2024

Uh oh!

blueorangutan commented May 2, 2024

Uh oh!

blueorangutan commented May 3, 2024

Uh oh!

vishesh92 commented May 7, 2024

Uh oh!

blueorangutan commented May 7, 2024

Uh oh!

blueorangutan commented May 7, 2024

Uh oh!

vishesh92 commented May 7, 2024

Uh oh!

blueorangutan commented May 7, 2024

Uh oh!

blueorangutan commented May 7, 2024

Uh oh!

borisstoyanov commented May 7, 2024

Uh oh!

blueorangutan commented May 7, 2024

Uh oh!

blueorangutan commented May 8, 2024

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

codecov-commenter commented Apr 30, 2024•
edited
Loading