Refactor of Allocator classes by BryanMLima · Pull Request #9074 · apache/cloudstack

BryanMLima · 2024-05-10T20:49:48Z

Description

This PR refactors some *Allocator classes, improving modularity and code legibility. This PR also made some changes to logs across these classes.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

How Has This Been Tested?

I tested the allocation process in my personal lab, using both the RandomAllocator and FirstFitAllocator allocators. I tried some variation of tags and offerings, and everything looks good. Furthermore, I also added a lot of unit tests for the methods that I refactored.

How did you try to break this feature and the system with this change?

codecov · 2024-05-10T20:58:21Z

Codecov Report

❌ Patch coverage is 80.75314% with 46 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.05%. Comparing base (856d83a) to head (803c3fa).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...gent/manager/allocator/impl/FirstFitAllocator.java	79.38%	23 Missing and 4 partials ⚠️
.../agent/manager/allocator/impl/RandomAllocator.java	89.09%	6 Missing ⚠️
...agent/manager/allocator/impl/TestingAllocator.java	0.00%	3 Missing ⚠️
...om/cloud/deploy/DeploymentPlanningManagerImpl.java	62.50%	2 Missing and 1 partial ⚠️
...in/java/com/cloud/server/ManagementServerImpl.java	0.00%	3 Missing ⚠️
.../src/main/java/com/cloud/host/dao/HostDaoImpl.java	0.00%	2 Missing ⚠️
...i/command/admin/host/FindHostsForMigrationCmd.java	0.00%	1 Missing ⚠️
...loudstack/api/command/admin/host/ListHostsCmd.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #9074 +/- ##
============================================
+ Coverage 18.01% 18.05% +0.03% - Complexity 16607 16653 +46 
============================================
Files 6029 6030 +1 Lines 542154 542077 -77 Branches 66451 66398 -53 ============================================
+ Hits 97681 97847 +166 + Misses 433457 433222 -235 + Partials 11016 11008 -8

Flag	Coverage Δ
uitests	`3.52% <ø> (ø)`
unittests	`19.21% <80.75%> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2024-06-19T04:16:42Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions · 2024-07-12T08:04:09Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

JoaoJandre

LGTM, I did some basic testing, changed the allocator to FirstFit and created some VMs, the VM allocation worked fine with and without tags. However, my tests were limited, further testing would be good.

DaanHoogland · 2024-08-26T08:11:09Z

@blueorangutan package

blueorangutan · 2024-08-26T08:12:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-08-26T09:30:59Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10801

DaanHoogland · 2024-08-26T13:06:42Z

@blueorangutan LLtest

winterhazel · 2026-04-16T19:58:11Z

@blueorangutan package

blueorangutan · 2026-04-16T20:00:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-16T20:10:46Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 17525

winterhazel · 2026-04-17T11:41:34Z

@blueorangutan package

blueorangutan · 2026-04-17T11:42:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-17T12:27:22Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17537

winterhazel

I went through the code again, and confirmed that it still looks good over 4.23. There was a mistake in the GPU-enabled host prioritization logic of the FirstFitAllocator which I fixed in the latest commits.

Also, I've done some testing with the FirstFit and Random allocators using distinct (i) host, template, and service offering tags (including tag rules) combinations, (ii) guest OS rules, and (iii) CPU and memory available on the hosts. I did not test the HVM and GPU filters/priorization, but the refactored code seems to have the same logic, and there's a good amount of unit tests for it.

@DaanHoogland I think that we can merge this one with regression tests. Could you run the CI one last time?

DaanHoogland · 2026-04-21T10:51:47Z

@blueorangutan test

blueorangutan · 2026-04-21T10:54:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-04-22T01:00:01Z

[SF] Trillian test result (tid-15927)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 47940 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9074-t15927-kvm-ol8.zip
Smoke tests completed. 145 look OK, 3 have errors, 3 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestVPCRedundancy>:setup	`Error`	0.00	test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setup	`Error`	0.00	test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup	`Error`	0.00	test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup	`Error`	0.00	test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup	`Error`	0.00	test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup	`Error`	0.00	test_vpc_vpn.py
all_test_webhook_delivery	`Skipped`	---	test_webhook_delivery.py
all_test_webhook_lifecycle	`Skipped`	---	test_webhook_lifecycle.py
all_test_host_maintenance	`Skipped`	---	test_host_maintenance.py

…o-method

winterhazel · 2026-04-24T11:10:18Z

@blueorangutan package

blueorangutan · 2026-04-24T11:12:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-24T12:06:13Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17599

winterhazel · 2026-04-24T12:51:55Z

@DaanHoogland @sureshanaparti could we attempt another CI run here?

sureshanaparti · 2026-04-24T12:53:40Z

@blueorangutan test

blueorangutan · 2026-04-24T12:56:04Z

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-04-24T12:58:06Z

[SF] Trillian Build Failed (tid-15945)

winterhazel · 2026-04-27T11:19:16Z

@sureshanaparti looks like something went wrong with the CI. Could you have a look?

sureshanaparti · 2026-04-27T12:48:44Z

@sureshanaparti looks like something went wrong with the CI. Could you have a look?

seems to be capacity issue, re-triggering...

@blueorangutan test

blueorangutan · 2026-04-27T12:50:04Z

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-04-28T04:15:55Z

[SF] Trillian test result (tid-15954)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 52555 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9074-t15954-kvm-ol8.zip
Smoke tests completed. 149 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestClusterDRS>:setup	`Error`	0.00	test_cluster_drs.py
test_list_system_vms_metrics_history	`Failure`	0.60	test_metrics_api.py

winterhazel · 2026-04-30T13:29:38Z

Latest test failures seem unrelated and passed on #9074 (comment), so I am proceeding with the merge.

github-actionsBot added the status:has-conflicts label Jun 19, 2024

BryanMLima force-pushed the refactor-allocate-to-method branch from 1ff8f4a to 42d16d9Compare July 11, 2024 15:47

boring-cyborgBot added component:api component:build component:compute component:management-server component:orchestration labels Jul 11, 2024

github-actionsBot added status:has-conflicts and removed status:has-conflicts labels Jul 11, 2024

BryanMLima force-pushed the refactor-allocate-to-method branch from 537c10f to 952c273Compare July 15, 2024 13:26

github-actionsBot removed the status:has-conflicts label Jul 15, 2024

BryanMLima changed the title ~~Refactoring FirstFitAllocator class~~Refactoring Allocator classesJul 15, 2024

BryanMLima marked this pull request as ready for review July 15, 2024 13:44

BryanMLima changed the title ~~Refactoring Allocator classes~~Refactor of Allocator classesJul 15, 2024

BryanMLima requested review from DaanHoogland and JoaoJandre and removed request for JoaoJandre July 15, 2024 13:45

DaanHoogland reviewed Jul 15, 2024
View reviewed changes

Comment threadserver/src/main/java/com/cloud/agent/manager/allocator/impl/TestingAllocator.java

JoaoJandre approved these changes Jul 18, 2024
View reviewed changes

BryanMLima force-pushed the refactor-allocate-to-method branch 2 times, most recently from dd1eb14 to 952c273Compare July 24, 2024 14:28

BryanMLima force-pushed the refactor-allocate-to-method branch from 952c273 to b7cc66fCompare August 23, 2024 19:11

Fix GPU allocation mistake
d2fa21a

winterhazel added status:testing and removed status:needs-testing labels Apr 16, 2026

Fix failing tests
ca53b51

winterhazel approved these changes Apr 17, 2026
View reviewed changes

winterhazel added status:waiting-for-smoketest-results and removed status:testing labels Apr 17, 2026

Merge remote-tracking branch 'upstream/main' into refactor-allocate-t…
803c3fa
…o-method

winterhazel mentioned this pull request May 8, 2026
Fix NPE on findHostsForMigration when no suitable hosts are found #13138
Merged
12 tasks

Uh oh!

Conversation

BryanMLima commented May 10, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecovBot commented May 10, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actionsBot commented Jun 19, 2024

Uh oh!

github-actionsBot commented Jul 12, 2024

Uh oh!

Uh oh!

JoaoJandre left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Aug 26, 2024

Uh oh!

blueorangutan commented Aug 26, 2024

Uh oh!

blueorangutan commented Aug 26, 2024

Uh oh!

DaanHoogland commented Aug 26, 2024

Uh oh!

winterhazel commented Apr 16, 2026

Uh oh!

blueorangutan commented Apr 16, 2026

Uh oh!

blueorangutan commented Apr 16, 2026

Uh oh!

winterhazel commented Apr 17, 2026

Uh oh!

blueorangutan commented Apr 17, 2026

Uh oh!

blueorangutan commented Apr 17, 2026

Uh oh!

winterhazel left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Apr 21, 2026

Uh oh!

blueorangutan commented Apr 21, 2026

Uh oh!

blueorangutan commented Apr 22, 2026

Uh oh!

winterhazel commented Apr 24, 2026

Uh oh!

blueorangutan commented Apr 24, 2026

Uh oh!

blueorangutan commented Apr 24, 2026

Uh oh!

winterhazel commented Apr 24, 2026

Uh oh!

sureshanaparti commented Apr 24, 2026

Uh oh!

blueorangutan commented Apr 24, 2026

Uh oh!

blueorangutan commented Apr 24, 2026

Uh oh!

winterhazel commented Apr 27, 2026

Uh oh!

sureshanaparti commented Apr 27, 2026

Uh oh!

blueorangutan commented Apr 27, 2026

Uh oh!

blueorangutan commented Apr 28, 2026

Uh oh!

winterhazel commented Apr 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

BryanMLima commented May 10, 2024•
edited
Loading

codecovBot commented May 10, 2024•
edited
Loading