Fix bug in cloud client missing the project id and refactor the package

[elivlo]

Description of changes:
The client only uses the project ID in the function GetVMByID and not in the function getVMByName. This causes the CSI where the instance ID is not found in the metadata and the VM is in a project to fail. Eventually they fall back into a crash loop back off and the CSI is not usable.

It refactors all functions of the cloud package to use the default project_id option. This reduces code duplication by using cloudstack client methods that support the default options. This refactoring is based on the following fix: It fixes the function getVMByName by using the method GetVirtualMachineByName instead of ListVirtualMachines.

These changes ensure that the project id is automatically included and makes the code more mature.

Testing performed:
I installed k8s into Cloudstack inside of a project and without. The setup of the CSI worked when running in the root project but not in sub projects with the current version 3.0.0. I am using my PR version currently in a cluster on a Cloudstack instance within a project and with that it is working just fine.

Please create a bugfix release when the PR is approved and merged.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[artem-sidorenko]

@Pearl1594 any chance to get a review on this one?

[tromob]

This PR would also fix volume resize for project-owned volumes — reported in apache/cloudstack#13634.

On v3.0.0 / 3.0.1 / main, ExpandVolume in pkg/cloud/volumes.go uses the raw SDK call c.Volume.GetVolumeByID(volumeID) without projectid, so a project-scoped key gets Count:0 and ControllerExpandVolume fails forever (PVC stuck in Resizing). The global csClient.DefaultOptions(cloudstack.WithProject(config.ProjectID)) added here would cover that path too. Could the scope note be extended to mention volume resize, and ExpandVolume additionally skip the call when currentSize >= newSize? Tested on CloudStack 4.22 + Kubernetes v1.33.

[Pearl1594]

code lgtm

[elivlo]

@Pearl1594 I have fixed the comment and rebased my branch on the main branch. It is now ready to merge. Should I also update the helm appVersion or will you do that?

[elivlo]

@Pearl1594 Could you please have a look into this PR again and create a release if the changes are fine?

[elivlo]

@tromob I guess that we have to rely on our own private forks.