fix(chart): every pod spec now states automountServiceAccountToken, and a guard keeps it that way (backend#2345)

[LukasWodka]

Closes backend#2345. Follow-up to backend#2344 / #793, which fixed the telemetry Collector instance; this closes the class.

Why this is not just six more lines

#2344 described the Collector as the only silent pod spec. It wasn't — 7 of 13 were silent. The defect was never that someone chose wrong; it was that nobody chose, and nothing asked. Fixing the seventh instance leaves the eighth free to happen, so the guard is the point of this PR and the six edits are what let it be armed green.

The six decisions, each read off the chart

template	value	why
resource-monitor-daemonset	true	resource-monitor-rbac.yaml binds its SA to a ClusterRole over pods, nodes, nodes/status, metrics.k8s.io — polling those is the job
jobs-manager-deployment	true	rbac.yaml: creates/deletes training Jobs, reads pod logs, issues TokenReviews
auto-upgrade-cronjob	true	auto-upgrade-rbac.yaml: */*/* in-namespace plus cluster-scoped storage and RBAC — it runs helm upgrade
image-refresh-cronjob	true	image-refresh-rbac.yaml: patches Deployments and DaemonSets
mysql-deployment	false	names no ServiceAccount
gpu-device-plugin (×2)	false	names no ServiceAccount

The four trues were already the effective value, so nothing changes at runtime — they become decisions instead of inheritances.

The one behavioural change is the three falses: mysql-client and both device-plugin DaemonSets stop mounting the namespace's default SA token. Checked before deciding — a database plus two busybox init containers make no API calls, and the upstream NVIDIA/AMD plugins talk to the kubelet over the device-plugin socket mounted in the same pod spec. Worth a reviewer's eye since the device plugins are third-party.

The guard — scripts/tests/automount-token-explicit.sh, added to DRIFT_GUARDS

Renders the charts and reads the pod specs back out of the manifests, attributing each to its template via helm's # Source: line. It holds no list of workloads and no list of expected values — a template added tomorrow is checked tomorrow.

Three fail-closed paths, because the obvious way to write this guard is the way that lies:

1. UNREACHED. A pod spec can only be inspected if some value combination renders it. The combination list is checked for coverage against the templates that declare a pod-bearing kind — a template no combination reaches fails, naming itself, rather than passing as "nothing to check". This is the failure mode a matrix-driven guard has, and it is the one that would make the file lie.
2. Zero pod specs found is a broken guard, not agreement.
3. Contradiction: a pod spec setting false while the chart binds its SA to a Role/ClusterRole — the workload cannot use permissions it was granted. This is what stops a true from being flipped back and staying green.

The mirror rule is deliberately not enforced

bound to a Role → must be true holds with no exceptions here. The reverse — true → must be bound — does not: ingestor/templates/post-install-job.yaml sets true with no binding, correctly, because it presents its token to jobs-manager to be TokenReviewed, never to the API server. "Bound" and "needs its token" are different predicates, and a guard that assumed otherwise would need an exception list — the restating this guard exists to avoid. Written down in the header rather than left for the next person to rediscover.

Values in the false direction that RBAC can't speak to are pinned by unit tests instead (mysql_test.yaml, gpu_device_plugin_test.yaml), asserted per vendor because the GPU file holds two separate pod specs.

Verification

helm unittest client -> 542 passed, 34 suites, 0 failures (539 before; +3)
make drift -> all 14 guards green (13 before; +1)
shellcheck -S warning -x scripts/tests/automount-token-explicit.sh -> clean
bash scripts/check-style.sh -> clean

Guard output, so the decisions are reviewable at a glance:

 true CronJob t-auto-upgrade client/templates/auto-upgrade-cronjob.yaml
false Job t-egress-enforcement-check client/templates/egress-enforcement-check.yaml
false Deployment t-egress-proxy client/templates/egress-proxy-deployment.yaml
false Job t-egress-reachability-check client/templates/egress-reachability-check.yaml
false DaemonSet amdgpu-device-plugin-daemonset client/templates/gpu-device-plugin.yaml
false DaemonSet nvidia-device-plugin-daemonset client/templates/gpu-device-plugin.yaml
true CronJob t-image-refresh client/templates/image-refresh-cronjob.yaml
true Deployment t-jobs-manager client/templates/jobs-manager-deployment.yaml
false Deployment mysql-client client/templates/mysql-deployment.yaml
false Deployment t-requests-proxy client/templates/requests-proxy-deployment.yaml
true DaemonSet t-resource-monitor client/templates/resource-monitor-daemonset.yaml
true Job t-storage-assertions-check client/templates/storage-assertions-check.yaml
false DaemonSet t-telemetry-collector client/templates/telemetry-collector-daemonset.yaml
true Job t-submit ingestor/templates/post-install-job.yaml
ok: all 14 distinct pod spec(s) state it, across all 13 pod-bearing template(s);
no RBAC-bound pod spec refuses its token (5 bound ServiceAccount(s))

Mutations — anchor asserted applied each time, then restored

mutation	result
strip the key from mysql-deployment	offender named with kind, name and source
delete the two GPU render combinations	gpu-device-plugin.yaml reported UNREACHED
make render_all() emit nothing	"no pod specs found — a broken guard, not agreement"
flip jobs-managertrue -> false	contradiction: bound SA refusing its token
flip mysqlfalse -> true	unit test reddens (Expected: false / Actual: true)
flip only the AMD DaemonSet	1 failed, 13 passed — the per-vendor split earns its keep

Note on the base

The first commit here is #793's, cherry-picked, so the guard could be armed green rather than landed red (CLAUDE.md rule 4) — the Collector is one of the seven and its fix is still in review. Merge #793 first; this then rebases to the single commit that matters. If you'd rather not carry the duplicate, say so and I'll rebase once #793 lands.

---

Note

Medium Risk
Security-hardening of ServiceAccount token mounts, including a real behavior change for MySQL and third-party GPU device plugins. The four true workloads are documentation-only at runtime; a wrong false on an API-using pod would break upgrades, image refresh, or job submission, which the new guard is meant to catch.

Overview
Makes automountServiceAccountToken an explicit decision on every remaining silent pod spec, and adds a drift guard so a new template cannot inherit Kubernetes’ default true again.

Runtime change: MySQL and both GPU device-plugin DaemonSets now set false (they name no SA and never call the API). jobs-manager, resource-monitor, auto-upgrade, and image-refresh set true — already the effective default, now stated.

The new automount-token-explicit.sh guard (wired into DRIFT_GUARDS) renders the charts, requires the key on every pod spec, fails on unreached templates or empty renders, and rejects a bound SA that sets false. Unit tests pin the false direction for MySQL and each GPU vendor. Chart version 1.9.62 → 1.9.63.

^{Reviewed by Cursor Bugbot for commit f755652. Bugbot is set up for automated code reviews on this repo. Configure here.}

[saqlainsyed007]

Follow-up to #793 that closes the class (backend#2345), and the guard is genuinely the point. Reviewed all six template edits and the new drift check against the code.

The six decisions are each correct and justified from the chart's own RBAC.true where the ServiceAccount is bound to real permissions the workload exists to use — jobs-manager (creates/deletes Jobs, reads logs, TokenReviews), resource-monitor (polls pods/nodes/metrics), auto-upgrade (helm upgrade over *), image-refresh (patches Deployments/DaemonSets); false where no ServiceAccount is named and no API calls happen — both GPU device-plugins (talk to the kubelet over the device-plugin socket) and mysql (a DB + busybox init containers). The false ones were previously mounting the namespace default token by omission.

The guard clears the house bar on every axis:

• Presence, not a value. It requires every rendered pod spec to state the key in one direction or the other — because the defect was that nobody chose, not that anyone chose wrong.
• Derived, not restated. Pod specs are read out of rendered manifests, and the template denominator is grepped from the files, so a template added tomorrow is checked tomorrow — no hand-list to go stale.
• One value check, in the only direction with no exceptions. A pod spec whose SA is the subject of a rendered RoleBinding/ClusterRoleBinding may not set false (the chart would grant permissions the pod can't reach); the bound-SA set is derived from the rendered bindings. The mirror is deliberately not enforced, correctly — post-install-job sets true with no binding because it presents its token for TokenReview, so "bound" and "needs its token" aren't the same predicate, and a guard assuming they were would need the exception list this file exists to avoid.
• Fails closed twice. Zero pod specs found is an error, not agreement; and any pod-bearing template that no value combination in render_all() reaches is reported UNREACHED and fails — the matrix-guard blind spot handled — plus a template rendering conflicting values across combinations is itself a finding.

The helm-unittest additions pin direction per GPU vendor and for mysql (equal on the key + notExists serviceAccountName), mutation-proof against a flip back. Chart bumped 1.9.62→1.9.63, guard registered in DRIFT_GUARDS. Green, no open threads. Approving.

[LukasWodka]

/fr-pass