From 2a301ff8716b29b6909491bd7ddc9c5c2cd7c316 Mon Sep 17 00:00:00 2001 From: Bryce Wilkinson Date: Tue, 4 Aug 2026 14:21:33 +0300 Subject: [PATCH] feat(helm): expose gateway scheduling fields and data volume config The gateway pod template exposes nodeSelector, affinity and tolerations but not priorityClassName or topologySpreadConstraints, and the StatefulSet hardcodes the data volume at 1Gi with no StorageClass control. Operators who need any of these have to fork the chart or mutate the rendered output out-of-band. Add four optional passthroughs: - priorityClassName: the gateway is a control-plane component. When it shares nodes with the workloads it serves, the default priority gives it no advantage under node pressure, so a local capacity crunch can evict it and widen into a fleet-wide outage. - topologySpreadConstraints: spread replicas across zones or nodes. Only affinity was available, which is a blunter tool for even spreading. - persistence.size / persistence.storageClassName: volumeClaimTemplates is immutable, so a claim created at 1Gi cannot be grown through the chart later. Omitting storageClassName also leaves the claim Pending on clusters with no default StorageClass. Both scheduling fields go in the shared gatewayPodTemplate, so they apply identically to the statefulset and deployment workload shapes. persistence applies to the statefulset only, which is where the volume exists. All four render only when set and default to current behaviour: 1Gi, no storageClassName, and neither scheduling field emitted. A default render is byte-identical to before this change. Signed-off-by: Bryce Wilkinson --- deploy/helm/openshell/README.md | 4 + .../openshell/templates/_gateway-workload.tpl | 7 + .../helm/openshell/templates/statefulset.yaml | 5 +- .../gateway_scheduling_persistence_test.yaml | 122 ++++++++++++++++++ deploy/helm/openshell/values.yaml | 21 +++ 5 files changed, 158 insertions(+), 1 deletion(-) create mode 100644 deploy/helm/openshell/tests/gateway_scheduling_persistence_test.yaml diff --git a/deploy/helm/openshell/README.md b/deploy/helm/openshell/README.md index d4310cb9a7..ddccbf10cd 100644 --- a/deploy/helm/openshell/README.md +++ b/deploy/helm/openshell/README.md @@ -166,6 +166,8 @@ add `ci/values-spire.yaml` to the OpenShell release values files. | nameOverride | string | `"openshell"` | Override the chart name used in generated resource names. | | networkPolicy.enabled | bool | `true` | Create a NetworkPolicy restricting SSH ingress on sandbox pods to the gateway. | | nodeSelector | object | `{}` | Node selector for the gateway pod. | +| persistence.size | string | `"1Gi"` | Size of the gateway data volume. Note that volumeClaimTemplates is immutable: changing this only affects claims created from now on, and does not resize the volume of an existing StatefulSet. | +| persistence.storageClassName | string | `""` | StorageClass for the gateway data volume. Empty = omit the field, using the cluster's default StorageClass. Set this on clusters that have no default StorageClass, otherwise the claim stays Pending. | | pkiInitJob.enabled | bool | `true` | Run a pre-install/pre-upgrade Job that creates gateway and client mTLS Secrets. When certManager.enabled=true, cert-manager owns TLS and this same hook runs in JWT-only mode even if pkiInitJob.enabled remains true. | | pkiInitJob.serverDnsNames | list | `[]` | Extra DNS SANs to append to the server certificate. | | pkiInitJob.serverIpAddresses | list | `[]` | Extra IP SANs to append to the server certificate. | @@ -173,6 +175,7 @@ add `ci/values-spire.yaml` to the OpenShell release values files. | podLabels | object | `{}` | Extra labels to add to the gateway pod. | | podLifecycle.terminationGracePeriodSeconds | int | `5` | Grace period, in seconds, before Kubernetes terminates the gateway pod. | | podSecurityContext.fsGroup | int | `1000` | fsGroup assigned to the gateway pod. | +| priorityClassName | string | `""` | PriorityClass for the gateway pod. The gateway is a control-plane component; when it shares nodes with the workloads it serves, the default priority gives it no advantage under node pressure. Empty = omit the field. | | probes.liveness.failureThreshold | int | `3` | Liveness probe failure threshold before the container is restarted. | | probes.liveness.initialDelaySeconds | int | `2` | Liveness probe initial delay, in seconds. | | probes.liveness.periodSeconds | int | `5` | Liveness probe period, in seconds. | @@ -246,6 +249,7 @@ add `ci/values-spire.yaml` to the OpenShell release values files. | supervisor.sideloadMethod | string | `""` | How the supervisor binary is delivered into sandbox pods. Empty (default) = auto-detect from cluster version: K8s >= v1.35 -> "image-volume" (ImageVolume enabled by default; GA in v1.36) K8s < v1.35 -> "init-container" (copies via init container + emptyDir) On K8s v1.33-v1.34 with the ImageVolume feature gate manually enabled, set this to "image-volume" explicitly. | | supervisor.topology | string | `"combined"` | Supervisor pod topology for Kubernetes sandboxes. "combined" runs the current single supervisor container in the agent pod. "sidecar" runs network enforcement in a dedicated sidecar and the process supervisor as a low-capability wrapper in the agent container. | | tolerations | list | `[]` | Tolerations for the gateway pod. | +| topologySpreadConstraints | list | `[]` | Topology spread constraints for the gateway pod, used to spread replicas across zones or nodes. Empty = omit the field. | | workload.allowMultiReplicaStatefulSet | bool | `false` | Allow replicaCount > 1 while rendering a StatefulSet. Prefer workload.kind=deployment for external database-backed multi-replica gateways; this override exists for operators who explicitly require StatefulSet identity or storage semantics. | | workload.kind | string | `"statefulset"` | Gateway workload controller kind. Use `statefulset` for the default SQLite database, or `deployment` when server.externalDbSecret points at an external database. | diff --git a/deploy/helm/openshell/templates/_gateway-workload.tpl b/deploy/helm/openshell/templates/_gateway-workload.tpl index 5931047e5f..6cdb800c83 100644 --- a/deploy/helm/openshell/templates/_gateway-workload.tpl +++ b/deploy/helm/openshell/templates/_gateway-workload.tpl @@ -162,6 +162,9 @@ spec: configMap: name: {{ .Values.server.oidc.caConfigMapName }} {{- end }} + {{- with .Values.priorityClassName }} + priorityClassName: {{ . }} + {{- end }} {{- with .Values.nodeSelector }} nodeSelector: {{- toYaml . | nindent 4 }} @@ -174,4 +177,8 @@ spec: tolerations: {{- toYaml . | nindent 4 }} {{- end }} + {{- with .Values.topologySpreadConstraints }} + topologySpreadConstraints: + {{- toYaml . | nindent 4 }} + {{- end }} {{- end }} diff --git a/deploy/helm/openshell/templates/statefulset.yaml b/deploy/helm/openshell/templates/statefulset.yaml index 30571f80ba..6a7b6013d4 100644 --- a/deploy/helm/openshell/templates/statefulset.yaml +++ b/deploy/helm/openshell/templates/statefulset.yaml @@ -21,7 +21,10 @@ spec: name: openshell-data spec: accessModes: ["ReadWriteOnce"] + {{- with .Values.persistence.storageClassName }} + storageClassName: {{ . | quote }} + {{- end }} resources: requests: - storage: 1Gi + storage: {{ .Values.persistence.size }} {{- end }} diff --git a/deploy/helm/openshell/tests/gateway_scheduling_persistence_test.yaml b/deploy/helm/openshell/tests/gateway_scheduling_persistence_test.yaml new file mode 100644 index 0000000000..1bca060fb9 --- /dev/null +++ b/deploy/helm/openshell/tests/gateway_scheduling_persistence_test.yaml @@ -0,0 +1,122 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: gateway scheduling fields and data volume +templates: + - templates/gateway-config.yaml + - templates/statefulset.yaml + - templates/deployment.yaml +release: + name: openshell + namespace: my-namespace + +tests: + # priorityClassName — shared pod template, so both workload shapes + - it: omits priorityClassName by default on the StatefulSet + template: templates/statefulset.yaml + asserts: + - notExists: + path: spec.template.spec.priorityClassName + + - it: sets priorityClassName on the StatefulSet when configured + template: templates/statefulset.yaml + set: + priorityClassName: system-cluster-critical + asserts: + - equal: + path: spec.template.spec.priorityClassName + value: system-cluster-critical + + - it: sets priorityClassName on the Deployment when configured + template: templates/deployment.yaml + set: + workload.kind: deployment + server.externalDbSecret: openshell-db + priorityClassName: system-cluster-critical + asserts: + - equal: + path: spec.template.spec.priorityClassName + value: system-cluster-critical + + # topologySpreadConstraints — shared pod template, so both workload shapes + - it: omits topologySpreadConstraints by default on the StatefulSet + template: templates/statefulset.yaml + asserts: + - notExists: + path: spec.template.spec.topologySpreadConstraints + + - it: sets topologySpreadConstraints on the StatefulSet when configured + template: templates/statefulset.yaml + set: + topologySpreadConstraints: + - maxSkew: 1 + topologyKey: topology.kubernetes.io/zone + whenUnsatisfiable: DoNotSchedule + labelSelector: + matchLabels: + app.kubernetes.io/name: openshell + asserts: + - equal: + path: spec.template.spec.topologySpreadConstraints[0].maxSkew + value: 1 + - equal: + path: spec.template.spec.topologySpreadConstraints[0].topologyKey + value: topology.kubernetes.io/zone + - equal: + path: spec.template.spec.topologySpreadConstraints[0].whenUnsatisfiable + value: DoNotSchedule + + - it: sets topologySpreadConstraints on the Deployment when configured + template: templates/deployment.yaml + set: + workload.kind: deployment + server.externalDbSecret: openshell-db + topologySpreadConstraints: + - maxSkew: 1 + topologyKey: kubernetes.io/hostname + whenUnsatisfiable: ScheduleAnyway + asserts: + - equal: + path: spec.template.spec.topologySpreadConstraints[0].topologyKey + value: kubernetes.io/hostname + + # persistence — StatefulSet only + - it: defaults the data volume to 1Gi with no storageClassName + template: templates/statefulset.yaml + asserts: + - equal: + path: spec.volumeClaimTemplates[0].spec.resources.requests.storage + value: 1Gi + - notExists: + path: spec.volumeClaimTemplates[0].spec.storageClassName + + - it: sets the data volume size when configured + template: templates/statefulset.yaml + set: + persistence.size: 10Gi + asserts: + - equal: + path: spec.volumeClaimTemplates[0].spec.resources.requests.storage + value: 10Gi + + - it: sets the data volume storageClassName when configured + template: templates/statefulset.yaml + set: + persistence.storageClassName: fast-ssd + asserts: + - equal: + path: spec.volumeClaimTemplates[0].spec.storageClassName + value: fast-ssd + + - it: keeps the claim name and access mode unchanged + template: templates/statefulset.yaml + set: + persistence.size: 5Gi + persistence.storageClassName: fast-ssd + asserts: + - equal: + path: spec.volumeClaimTemplates[0].metadata.name + value: openshell-data + - equal: + path: spec.volumeClaimTemplates[0].spec.accessModes[0] + value: ReadWriteOnce diff --git a/deploy/helm/openshell/values.yaml b/deploy/helm/openshell/values.yaml index 0525ed475d..32d66563c0 100644 --- a/deploy/helm/openshell/values.yaml +++ b/deploy/helm/openshell/values.yaml @@ -160,6 +160,27 @@ tolerations: [] # -- Affinity rules for the gateway pod. affinity: {} +# -- PriorityClass for the gateway pod. The gateway is a control-plane +# component; when it shares nodes with the workloads it serves, the default +# priority gives it no advantage under node pressure. Empty = omit the field. +priorityClassName: "" + +# -- Topology spread constraints for the gateway pod, used to spread replicas +# across zones or nodes. Empty = omit the field. +topologySpreadConstraints: [] + +# Gateway data volume (workload.kind=statefulset only). The StatefulSet's +# openshell-data claim holds the default SQLite database. +persistence: + # -- Size of the gateway data volume. Note that volumeClaimTemplates is + # immutable: changing this only affects claims created from now on, and does + # not resize the volume of an existing StatefulSet. + size: 1Gi + # -- StorageClass for the gateway data volume. Empty = omit the field, using + # the cluster's default StorageClass. Set this on clusters that have no + # default StorageClass, otherwise the claim stays Pending. + storageClassName: "" + # Server configuration server: # -- Gateway log level.