fix(chart): resolve serviceDbAccounts per environment — unbreak dev + staging ingestion (backend#1752)

[LukasWodka]

Fixesbackend#1752. dev and staging ingestion are broken right now — this unbreaks them.

Verified, not inferred

I ran the published images rather than reasoning from PR descriptions:

image	version	DB_USER with the flag off
ghcr.io/tracebloc/ingestor:dev	0.8.5	ValueError
ghcr.io/tracebloc/ingestor:stg	0.8.5	ValueError
prodDigest sha256:9098b3c9…	0.7.7	'edgeuser' — works

data-ingestors#468 removed the edgeuser fallback on the stated precondition that this flag was "on fleet-wide". It was on nowhere. dev and staging edges float on the :dev/:stg channels and took the change the same day, so every ingestion Job fails at Config() before reading a byte.

Prod escaped only because its ingestor is digest-pinned to the 0.7 line — a caution engineered for D16 (#1151), not for this, and one that client#490 is about to spend.

The actual gap

backend#1151 records the rollout as "flip per environment, dev first". There was no mechanism for that. One global boolean meant "per environment" was hand-editing every edge and remembering which fleet was where. The fleet's posture couldn't be read in one place and nothing tested it.

So the flag now resolves the way channelTags already does — explicit override wins, otherwise keyed on the resolvedCLIENT_ENV:

serviceDbAccounts: # operator override, normally unsetserviceDbAccountsByEnv:
dev: true # already running the ingestor that needs the credsstg: true # ← this is what unbreaks stagingprod: false # explicitly unchanged

prod stays off, deliberately. It runs the 0.7-line ingestor which still has the fallback. Flipping prod is a separate windowed decision (#1528 S0 drill) — and it must happen before prod's ingestor is repinned past 0.8.2, which is the constraint I flagged on client#490.

Resolution goes through tracebloc.clientEnv, so the documented alias staging maps to stg instead of silently missing its entry. That's backend#1723 — one env-name spelling away — and it has its own test.

Safety of the flip

The Secret and the jobs-manager env are gated on the same resolver, so a flip is atomic in one upgrade: the Secret is created, jobs-manager restarts, validates the wiring (_require_service_db_env fail-fasts if half-wired), mints tb_meta/tb_ingest, and spawned Jobs carry the credentials. A test asserts the Secret and the env move together.

Evidence

• 384 chart tests (was 375), helm lint clean.
• Rendered across every env name and both override directions.
• Mutation-checked: reverting the templates turns the dev/stg cases red; swapping the resolver for a raw CLIENT_ENV lookup turns the staging alias case red on its own.

One of my own checks was vacuous mid-way and I want to flag it: my first render sweep reported "not wired" for every environment, and I nearly read that as a logic bug. The renders were erroring on values.schema.json (got null, want boolean) and my grep counted zero either way. The corrected sweep asserts the render succeeded before interpreting the result.

Note for review — pre-existing, not from this change

With the flag on, secrets.yaml renders two Secrets both named <release>-secrets. Identical on develop with serviceDbAccounts=true, so out of scope here, but worth someone's eyes.

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes MySQL credential wiring and Secret emission for dev/stg fleets on upgrade; prod stays off but mistakes in env resolution or prod override could break ingestion or widen DB identity exposure.

Overview
Restores dev/staging ingestion by turning on dedicated MySQL identities (tb_meta / tb_ingest) where the floating ingestor now requires per-job DB_USER/DB_PASSWORD, while leaving prod off until a deliberate flip.

The chart no longer treats serviceDbAccounts as a single global boolean. A new tracebloc.serviceDbAccounts helper resolves explicit serviceDbAccounts override first, else serviceDbAccountsByEnv keyed on normalized CLIENT_ENV (so staging → stg). Defaults in values.yaml are dev/stg: true, prod: false. jobs-manager, requests-proxy, and secrets now gate SERVICE_DB_ACCOUNTS / TB_* env and Secret keys on that helper instead of .Values.serviceDbAccounts alone.

Adds serviceDbAccountsByEnv to the values schema (nullable serviceDbAccounts), a Helm unittest suite for env/alias/override/Secret lockstep, bumps chart to 1.9.33, and updates SECURITY.md for per-env rollout and ingestor ordering.

^{Reviewed by Cursor Bugbot for commit d7285f1. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

Added docs/SECURITY.md — the org rule is that a statement this change falsifies travels with it, and §4.1.1 / §8.10 had four:

• tb_meta / tb_ingest listed as "not yet consumed" — S2 shipped (client-runtime#305/fix(installer): make the training size configurable at install (TRACEBLOC_TRAINING_RESOURCES) #308, client#664); they are consumed wherever the flag is on
• edgeuser listed as the default identity for jobs-manager, requests-proxy and ingestion pods "today" — true only on prod now
• "gated on serviceDbAccounts (default off) … purely additive … a default install authenticates exactly as it did before" — wrong on both counts
• the S1/S2 rollout steps still read as pending

And the document didn't mention the hazard at all, which is the part that actually cost a day. §4.1.1 now carries it as an explicit ordering constraint:

Turn this flag on for a fleet before its ingestor moves to 0.8.x, never after. For prod that means before images.ingestor.prodDigest moves past 0.8.2 (client#490).

with a pointer to the CI check that enforces it (backend#1754).

A security doc describing a retired fallback as present is worse than one that says nothing — it's the document someone reads before flipping a prod fleet.

384 chart tests, style guard, fact-check and drift all clean.