cloud-provider-otc

Warning

Experimental — for demo purposes only. DO NOT USE IN PRODUCTION. This project is an experiment and is provided as-is, without any guarantees of stability, support, or maintenance.

A Kubernetes Cloud Controller Manager (CCM) for Open Telekom Cloud (OTC).

Built on the official k8s.io/cloud-provider framework using the OTC SDK gophertelekomcloud (AK/SK auth). Registers the cloud provider otc and manages the otc:// providerID identity. Intended as a workload-cluster addon for clusters provisioned by cluster-api-provider-otc.

Status: v1 feature-complete (node controller + LoadBalancer), live-verified against OTC eu-de. See ROADMAP.md.

Note

This project was developed with substantial AI assistance (Claude Code). All changes are human-reviewed, covered by unit tests, and the delivered controllers have been live-verified against real OTC infrastructure.

Compatibility

The CCM version tracks Kubernetes: v0.33.x is built against k8s.io/cloud-provider v0.33 and targets Kubernetes 1.33 workload clusters. Pin the CCM minor to your cluster's Kubernetes minor.

Kubernetes	CCM
v1.33	v0.33.x

What it does

• Node controller (InstancesV2): sets providerID, node addresses, and topology.kubernetes.io/{zone,region} labels; removes the node.cloudprovider.kubernetes.io/uninitialized taint. Backed by OTC ECS.
• LoadBalancer: Service type=LoadBalancer → OTC ELB, both shared (v2) and dedicated (v3) products, with EIP lifecycle management.

Routes, Zones, Instances, and Clusters are intentionally not implemented (Zones/Instances are superseded by InstancesV2; route management is left to the CNI).

LoadBalancer Services

LoadBalancer support (service controller, Service type=LoadBalancer → OTC ELB) is implemented for both ELB products: shared ELB v2 (default) and dedicated ELB v3 (via the otc.cloud/elb-class: dedicated annotation). It is enabled by setting cloudConfig.subnetId (the Neutron subnet ID, not the VPC subnet ID) to render a [LoadBalancer] section in cloud.conf; if subnetId is empty the CCM runs without LoadBalancer support even if the service controller is listed.

Annotations

All annotations are optional; a plain Service type=LoadBalancer gets a shared, internet-facing ELB with an auto-allocated EIP.

Annotation	Values	Default	Notes
otc.cloud/elb-class	shared, dedicated	shared	dedicated selects an ELB v3 (hourly-billed) instance; requires network-id in the [LoadBalancer] config section.
otc.cloud/elb-internal	true, false	false	No EIP is allocated or bound; only the VIP is used. Contradicts elb-eip-id.
otc.cloud/elb-eip-id	existing EIP ID	—	Bind-your-own EIP. Contradicts elb-internal.
otc.cloud/elb-bandwidth-mbit	positive integer	cloudConfig.bandwidthMbit (chart default 100)	Bandwidth for an auto-created EIP.

Ownership

Each Service gets a deterministically named ELB: kube_<clusterName>_<namespace>_<service> (clusterName is the cloudConfig.clusterName chart value, or the --cluster-name flag if not set). The CCM also tags the ELB with a description of kubernetes.io/service-uid=<Service UID> and uses that as the source of truth for ownership — if an ELB with the expected name exists but its description doesn't match the Service's UID, the CCM refuses to touch it (never adopts, never mutates, never deletes a foreign LB).

EIP policy

• Auto-created EIP (default, non-internal, no elb-eip-id): allocated and bound by the CCM, name-marked as owned, and released only when the Service is deleted. Switching an existing Service to otc.cloud/elb-internal: "true" does not release an already-created EIP (known v1 limitation—delete and recreate the Service instead).
• BYO EIP (elb-eip-id or spec.loadBalancerIP pointing at an existing EIP): bound but never released by the CCM — it is the operator's EIP to manage.

v1 limitations

• No SCTP — only TCP and UDP ports are supported.
• Changing externalTrafficPolicy on a live Service is not reconciled in place; delete and recreate the Service to pick up the change.
• Changing otc.cloud/elb-class on a live Service is rejected with an error event; the old class's LB is never mutated in place — delete and recreate the Service to switch class.
• UDP listeners are implemented but pending live verification against real OTC ELB.
• The effective cluster name must never change for a cluster with live LoadBalancer Services: LB lookup is name-based (kube_<cluster>_...), so a rename orphans every existing ELB and its EIP (a new set is created under the new name). Set cloudConfig.clusterName once, before the first Service.

cloud.conf [LoadBalancer] section

The [LoadBalancer] section (rendered from cloudConfig.* values) configures LoadBalancer support. All keys are optional and are rendered only when their values are set.

Key	Purpose
subnet-id	Required to enable LoadBalancer support. The Neutron subnet ID where ELBs are created (not the VPC subnet ID).
network-id	Optional. The Neutron network ID; required only for elb-class: dedicated (ELB v3).
bandwidth-mbit	Optional; default 100 (Mbps). Bandwidth for auto-created EIPs.
cluster-name	Optional; overrides --cluster-name for LB naming (kube_<cluster>_<ns>_<svc>). Rendered automatically by the CAPI provider for ccm: otc clusters; the DaemonSet args are now identical for every cluster.

Why not the upstream OpenStack CCM?

OTC authenticates with AK/SK (not Keystone) and exposes no native Octavia API, so the upstream openstack-cloud-controller-manager fits poorly. See docs/context/investigation-ccm-csi-loadbalancer.md.

Development

Gates (run before submitting changes): make build, go vet ./..., gofmt -l ., make test, make lint. See CONTRIBUTING.md.

The runtime otc-credentials principal (used by both this CCM and the sibling csi-driver-otc) needs specific IAM grants beyond a generic ECS/ELB policy — see docs/context/otc-iam-permissions.md.

Container image

Release images are published to GHCR by CI on version tags:

ghcr.io/d-velop/otc-cloud-controller-manager:<version>

Mirroring to OTC SWR (optional)

If workload-cluster nodes should pull in-region (or have no public egress), mirror the image to OTC SWR:

1. Create an SWR organization in the OTC console (e.g. my-org), region eu-de.
2. Log in to SWR (login credential is generated from your AK/SK in the SWR console under Login Command, or via swr tooling):

   docker login -u <region>@<AK> -p <login-key> swr.eu-de.otc.t-systems.com
   

3. Build and push (set your org and version):

   make docker-build docker-push \
   IMG=swr.eu-de.otc.t-systems.com/my-org/otc-cloud-controller-manager:<version>
   

   (or retag/push the GHCR release image with docker tag — no local build needed).
4. Set the SWR repo public so nodes pull without a secret, or keep it private and deliver an image-pull Secret via the CRS (see below).

Delivery to a workload cluster (ClusterResourceSet)

The CCM is delivered as a CAPI addon. See the cluster-api-provider-otc samples (config/samples/otc-ccm-crs.yaml). In short, on the management cluster:

1. Render the CCM manifests and create the CRS ConfigMap (cluster-agnostic — no per-cluster values here):

   helm template otc-ccm charts/otc-cloud-controller-manager --namespace kube-system \
   --set image.repository=ghcr.io/d-velop/otc-cloud-controller-manager \
   --set image.tag=<version> > /tmp/otc-ccm.yaml
   kubectl create configmap otc-ccm -n default --from-file=otc-ccm.yaml=/tmp/otc-ccm.yaml
   

2. Create the cloud.conf Secret CRS resource from your otc-credentials values (all cluster-specific values go here). It MUST be typed addons.cluster.x-k8s.io/resource-set — a plain Opaque Secret makes CAPI reject the whole ResourceSet (the CCM ConfigMap would then never apply either):

   helm template otc-ccm charts/otc-cloud-controller-manager --namespace kube-system \
   --set cloudConfig.create=true \
   --set cloudConfig.accessKey=<from otc-credentials> \
   --set cloudConfig.secretKey=<from otc-credentials> \
   --set cloudConfig.domainName=<from otc-credentials> \
   --set cloudConfig.projectName=<project> \
   --set cloudConfig.subnetId=<neutron-subnet-id> \
   --set cloudConfig.networkId=<neutron-network-id> \
   --set cloudConfig.clusterName=<capi-cluster-name> \
   --show-only templates/secret.yaml > /tmp/otc-cloud-config.yaml
   # projectName is optional; networkId is required only for elb-class: dedicated
   kubectl create secret generic otc-ccm-cloudconfig -n default \
   --type=addons.cluster.x-k8s.io/resource-set \
   --from-file=cloud-config.yaml=/tmp/otc-cloud-config.yaml
   

3. Apply the otc-ccm ClusterResourceSet; label the target Cluster with ccm=otc.

Upgrading from chart <0.4.0: if updating the management-cluster static otc-ccm ConfigMap from a previous chart render, the cloud.conf Secret must gain the cluster-name key in the same change — the refreshed DaemonSet no longer carries the --cluster-name flag, and re-applying manifests without the Secret key would cause LoadBalancers to be renamed to kube_kubernetes_..., orphaning all existing ELBs.

Related projects

• cluster-api-provider-otc — Cluster API infrastructure provider for OTC (provisions the clusters this CCM runs in).
• csi-driver-otc — OTC-native CSI driver for EVS block storage.

License

Licensed under the Apache License, Version 2.0. See LICENSE.