From 0887cff20189cc18c847380fb5298f6945302393 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Mon, 24 Aug 2026 15:09:02 +0100 Subject: [PATCH 1/5] docs(config): add ADR 0019 for raw API-response passthrough guardrails --- .../0019-config-api-response-passthrough.md | 121 ++++++++++++++++++ docs/adr/README.md | 1 + 2 files changed, 122 insertions(+) create mode 100644 docs/adr/0019-config-api-response-passthrough.md diff --git a/docs/adr/0019-config-api-response-passthrough.md b/docs/adr/0019-config-api-response-passthrough.md new file mode 100644 index 0000000000..8c319d5ec8 --- /dev/null +++ b/docs/adr/0019-config-api-response-passthrough.md @@ -0,0 +1,121 @@ +# 0019. Raw API-response passthrough on API-sourced config (`_apiResponse`) + +**Status**: accepted +**Date**: 2026-08-24 + +## Problem Statement + +`@supabase/config` is headed for npm as the shared config contract for the +CLI, Studio, and other consumers, and it will grow a mapping from the +Management API's `/v2/projects/{ref}/config` response into the config shape +(the translation layer `config diff`/`config pull` and Studio's drift +detection all need). The Management API evolves continuously; the package will +publish deliberately. Without a designed escape hatch, every new API field is +unreachable to package consumers until someone maps it and cuts a release — +the package's release cadence becomes a bottleneck on every service team's +API velocity. + +Two properties of the v2 response make a naive passthrough dangerous: + +- Secrets are returned as HMAC digests of their values (e.g. inside the + `auth` attribute record). Anything that carries the raw response must never + be written into a config file. +- Config values flow through structural walks — the sparse subtraction core + from ADR 0018, default omission, encode-to-file. A non-config key visible + to those walks would register as drift or get persisted. + +## Decision + +API-sourced config values carry the raw response, governed by five rules: + +1. **`_apiResponse?: Record`** — an optional field on the + config value produced from an API response, holding the raw v2 + `data.attributes` object verbatim. It is present only when the value was + built from an API response; file-sourced config never has it. Its absence + therefore does not mean "no unmapped fields exist". +2. **Lenient input decode.** The schema that decodes the v2 attributes must + pass unknown keys through without failing. This is the primary protection + against API-ahead-of-package skew; `_apiResponse` is only the access + mechanism for what lenient decoding let through. +3. **One metadata-key rule.** Keys starting with `$` or `_` are metadata, not + config. They are excluded from every structural walk (sparse subtraction, + default omission, value-origin tracking) and from every encode/persist + path. The `$schema` key preserved by `io.ts` is the existing precedent; + `_apiResponse` is the second member of the same rule, implemented once in + the shared walk core rather than per call site. +4. **Never persisted.** Because encode strips metadata keys, `_apiResponse` + (and its HMAC'd secret digests) can never land in `config.toml` / + `config.json`, including via `config pull`-style flows. +5. **Fallback, not contract.** When a raw key later graduates into the typed + mapping, the typed field wins; the raw key remains readable but its naming, + units, or polarity may differ from the typed field (the API↔config mapping + includes renames, boolean inversions, and unit conversions). Consumers + needing "which API fields does this package version not understand" use a + registry-derived `unmappedApiFields()` helper (raw attributes minus the + keys the mapping consumed) rather than a second stored field. + +## Rationale + +- The alternative to rule 2 — a closed decode — converts every additive API + change into a hard decode failure for all published consumers. That is + strictly worse than an unmapped-but-reachable field. +- Rule 3 exists because the ADR 0018 subtraction core compares config values + structurally: a remote config carrying `_apiResponse` would otherwise diff + as drifted against every baseline, producing exactly the phantom-drift + false positives the drift feature is trying to eliminate. +- Storing the whole raw response (rule 1) rather than only the unmapped + leftovers keeps the field's contents stable across package releases; + the leftovers are derivable and shrink release-over-release, which is + correct semantics for a helper but confusing semantics for stored data. + +## Consequences + +### Positive + +- Consumers are decoupled from the package's publish cadence for read access + to new API fields. +- One metadata-key rule covers `$schema` and `_apiResponse` together, in one + shared walk implementation. +- `unmappedApiFields()` doubles as a mapping-completeness check: a + nonempty result on a known API version is a to-do list for the registry. + +### Negative + +- Field presence leaks provenance (API-sourced vs file-sourced). Accepted — + it is arguably a feature — but code must not treat absence as "fully + mapped". +- Two representations of graduated fields coexist (raw and typed) and can + disagree in naming/units/polarity; the documented typed-wins precedence + mitigates but cannot remove the confusion risk. +- Every future structural walk must honor the metadata-key rule; keeping the + walks funneled through the shared core is what makes this tractable. + +## Alternatives Considered + +1. **Store only unmapped leftovers (`_unmapped`)**: contents change with + every package release even when the API response is identical; unstable + stored data. Derive leftovers via helper instead. +2. **Return a `{ config, raw }` pair instead of embedding**: keeps config + values walk-clean by construction, but the raw half does not travel with + the value through application code and state stores; each consumer would + rebuild the pairing. Embedding plus the strip rule is more ergonomic for + the same safety. +3. **Strict decode of the v2 attributes**: rejected outright; see Rationale. +4. **Inline passthrough of unknown keys per section** (zod + `.passthrough()`-style): unknown API keys arrive in API naming/units, + which for this mapping differ from config naming (renames, inversions, + unit conversions). Inlining them beside typed config fields misrepresents + them as config values and puts them back in the path of structural walks. + +## Related Decisions + +- ADR 0018: sparse config subtraction — defines the structural walks that the + metadata-key rule scopes out. +- ADR 0009: configuration schema & validation. + +## See Also + +- supabase/supabase#48906 — Studio config-drift work; vendors a temporary + schema mirror explicitly awaiting the published `@supabase/config` package. +- Linear: CLI-2155/CLI-2156 (sparse + diff), CLI-2231 (entrypoint split), + CLI-2234 (public-surface audit), CLI-2169 (publish umbrella). diff --git a/docs/adr/README.md b/docs/adr/README.md index c0bc1cf7b5..1946bcd20b 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -58,6 +58,7 @@ When an ADR becomes outdated, mark it as `deprecated` or reference the supersedi | 0015 | [Managed Stack Contract Fixtures](0015-managed-stack-contract-fixtures.md) | superseded | | 0016 | [Legacy Port Completion and Go CLI Authority Scope](0016-legacy-port-completion-and-go-cli-authority-scope.md) | proposed | | 0017 | [Simplified Managed Stack Architecture](0017-simplified-managed-stack-architecture.md) | accepted | +| 0019 | [Raw API-Response Passthrough on API-Sourced Config](0019-config-api-response-passthrough.md) | accepted | ## Template From bf7a489fb02877cc4c461b657813e6a81f16ab00 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Mon, 24 Aug 2026 16:00:07 +0100 Subject: [PATCH 2/5] docs(config): address codex review on ADR 0019 Scope the metadata-key rule to reserved struct positions instead of a $/_ prefix match, since dynamic Schema.Record keys (function slugs, edge runtime secret names) legitimately start with either character. Require executeRaw for the v2 config fetch so the generated client's strict schema doesn't strip unknown fields before the lenient decode runs. Require non-enumerable/out-of-band storage for _apiResponse so generic serialization can't observe it. Carve secrets out of typed graduation so an HMAC digest can't reach the encoder through the typed field. --- .../0019-config-api-response-passthrough.md | 112 +++++++++++++----- 1 file changed, 85 insertions(+), 27 deletions(-) diff --git a/docs/adr/0019-config-api-response-passthrough.md b/docs/adr/0019-config-api-response-passthrough.md index 8c319d5ec8..10daae4c75 100644 --- a/docs/adr/0019-config-api-response-passthrough.md +++ b/docs/adr/0019-config-api-response-passthrough.md @@ -32,41 +32,91 @@ API-sourced config values carry the raw response, governed by five rules: config value produced from an API response, holding the raw v2 `data.attributes` object verbatim. It is present only when the value was built from an API response; file-sourced config never has it. Its absence - therefore does not mean "no unmapped fields exist". -2. **Lenient input decode.** The schema that decodes the v2 attributes must - pass unknown keys through without failing. This is the primary protection - against API-ahead-of-package skew; `_apiResponse` is only the access - mechanism for what lenient decoding let through. -3. **One metadata-key rule.** Keys starting with `$` or `_` are metadata, not - config. They are excluded from every structural walk (sparse subtraction, - default omission, value-origin tracking) and from every encode/persist - path. The `$schema` key preserved by `io.ts` is the existing precedent; - `_apiResponse` is the second member of the same rule, implemented once in - the shared walk core rather than per call site. + therefore does not mean "no unmapped fields exist". It is defined as a + non-enumerable property (or held out-of-band, e.g. in a module-private + `WeakMap`) rather than an ordinary object + field, so `JSON.stringify`, object spread, and any encoder outside this + package — not just the package's own — cannot observe it by construction. + The metadata-key strip rule below is defense-in-depth for this package's + own structural walks, not the sole safeguard. +2. **Lenient decode, applied before the strict generated schema sees the + body.** The CLI fetches the v2 config response via `@supabase/api`'s + `executeRaw` (`packages/api/src/internal/client.ts`), not `execute`. + `execute` decodes through the generated `V2GetProjectConfigOutput` + `Schema.Struct`, which discards excess properties, so by the time + `@supabase/config`'s lenient schema ran on that output any API-ahead field + would already be gone. `executeRaw` returns the response undecoded; + `@supabase/config`'s schema then decodes `data.attributes` and must pass + unknown keys through without failing. This lenient decode is the primary + protection against API-ahead-of-package skew; `_apiResponse` is only the + access mechanism for what it let through. +3. **One metadata-key rule, scoped to fixed struct shapes.** `_apiResponse` + and `$schema` are metadata, not config, wherever they occur as a declared + field of a fixed `Schema.Struct` (the config document root, or the value + object an API-sourced section attaches `_apiResponse` to). The rule is a + position match on those two reserved names, not a `$`/`_` prefix match, + and it never reaches into a dynamic `Schema.Record` key space: function + slugs (`functions.`, e.g. `functions._health`), edge runtime secret + names (`edge_runtime.secrets.`, e.g. `secrets._TOKEN`), env var + names, and other user-named keys keep whatever leading `_`/`$` character + they're given, because those keys are user data, not metadata, regardless + of spelling. Reserved-name matches are excluded from every structural walk + (sparse subtraction, default omission, value-origin tracking) and from + every encode/persist path. The `$schema` key preserved by `io.ts` is the + existing precedent; `_apiResponse` is the second member of the same rule, + implemented once in the shared walk core rather than per call site. 4. **Never persisted.** Because encode strips metadata keys, `_apiResponse` (and its HMAC'd secret digests) can never land in `config.toml` / `config.json`, including via `config pull`-style flows. -5. **Fallback, not contract.** When a raw key later graduates into the typed - mapping, the typed field wins; the raw key remains readable but its naming, - units, or polarity may differ from the typed field (the API↔config mapping - includes renames, boolean inversions, and unit conversions). Consumers - needing "which API fields does this package version not understand" use a - registry-derived `unmappedApiFields()` helper (raw attributes minus the - keys the mapping consumed) rather than a second stored field. +5. **Fallback, not contract — with a secret carve-out.** When a raw key later + graduates into the typed mapping, the typed field wins; the raw key + remains readable but its naming, units, or polarity may differ from the + typed field (the API↔config mapping includes renames, boolean inversions, + and unit conversions). This precedence does not apply to fields the schema + annotates `x-secret` (`packages/config/src/lib/env.ts`'s `secret()` / + `env({ secret: true })`, e.g. `edge_runtime.secrets.*`): the API only ever + returns an HMAC digest for these, never the underlying value, so letting + the typed field win would hand the digest to the normal encoder and + persist it despite rule 4. For `x-secret` fields the mapping omits the + API-sourced value entirely, leaving any locally configured value + untouched. Consumers needing "which API fields does this package version + not understand" use a registry-derived `unmappedApiFields()` helper (raw + attributes minus the keys the mapping consumed) rather than a second + stored field. ## Rationale - The alternative to rule 2 — a closed decode — converts every additive API change into a hard decode failure for all published consumers. That is - strictly worse than an unmapped-but-reachable field. -- Rule 3 exists because the ADR 0018 subtraction core compares config values - structurally: a remote config carrying `_apiResponse` would otherwise diff - as drifted against every baseline, producing exactly the phantom-drift - false positives the drift feature is trying to eliminate. + strictly worse than an unmapped-but-reachable field. Routing through + `executeRaw` rather than `execute` is required, not incidental: the + generated client's strict `Schema.Struct` decode already drops excess + properties, so a lenient decode layered on top of it would have nothing + left to be lenient about. +- Rule 3 is scoped to fixed struct positions, not a `$`/`_` prefix match, + because the config schema uses dynamic `Schema.Record` keys (function + slugs, edge runtime secret names, env var names) that legitimately start + with either character. A prefix match would make the metadata rule silently + drop user config through the same structural walks it's meant to protect. + Scoped to reserved names, it still exists because the ADR 0018 subtraction + core compares config values structurally: a remote config carrying + `_apiResponse` would otherwise diff as drifted against every baseline, + producing exactly the phantom-drift false positives the drift feature is + trying to eliminate. - Storing the whole raw response (rule 1) rather than only the unmapped leftovers keeps the field's contents stable across package releases; the leftovers are derivable and shrink release-over-release, which is correct semantics for a helper but confusing semantics for stored data. + Non-enumerable/out-of-band storage is required rather than a plain field + because the metadata-key rule only protects this package's own walks — + a shared npm contract also gets serialized by `JSON.stringify`, object + spread, and consumer-written encoders this package doesn't control. +- The rule 5 secret carve-out exists because typed graduation and the + metadata-key rule solve different problems: metadata-key strips the raw + copy, but a graduated secret's *typed* value is not metadata-keyed and + remains eligible for the normal encoder. Without the carve-out, an HMAC + digest could reach `config.toml` through the typed field even though rule + 4 successfully blocked the raw one. ## Consequences @@ -85,10 +135,18 @@ API-sourced config values carry the raw response, governed by five rules: it is arguably a feature — but code must not treat absence as "fully mapped". - Two representations of graduated fields coexist (raw and typed) and can - disagree in naming/units/polarity; the documented typed-wins precedence - mitigates but cannot remove the confusion risk. -- Every future structural walk must honor the metadata-key rule; keeping the - walks funneled through the shared core is what makes this tractable. + disagree in naming/units/polarity; the documented typed-wins precedence, + minus the secret carve-out, mitigates but cannot remove the confusion risk. +- Every future structural walk must honor the metadata-key rule, and that + rule must stay position-scoped (reserved struct field, not string prefix) + as new dynamic `Schema.Record` sections are added; keeping the walks + funneled through the shared core is what makes this tractable. +- The mapping layer must special-case every `x-secret`-annotated field to + skip typed graduation, rather than treating all API-sourced fields + uniformly; missing one lets a digest reach the encoder. +- CLI callers must use `executeRaw` for this endpoint instead of the + generated `execute`, forgoing the generated client's typed output for the + v2 config response specifically. ## Alternatives Considered From 4c98523be33af9aa36afa593e483215f0f1e54b8 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Mon, 24 Aug 2026 16:26:26 +0100 Subject: [PATCH 3/5] docs(config): address codex round-2 review on ADR 0019 Reframe rule 3 around exclusion-by-construction: sparse.ts's subtractValue iterates Object.entries, so a non-enumerable _apiResponse is invisible to every walk with zero key checks and no schema/path plumbing, and $schema never exists on decoded values at all (io.ts extracts it pre-decode and re-attaches on write). Split the persistence policies accordingly: $schema must be written, _apiResponse never can be. State that _apiResponse is not a decode schema field (file input can never produce it), that generic copies drop it by design with an explicit accessor/attach path for consumers that need it, that the executeRaw caller status-checks before decoding, and that pull-style rewrites must source x-secret fields from the local document with drift skipping them. --- .../0019-config-api-response-passthrough.md | 160 +++++++++++------- 1 file changed, 99 insertions(+), 61 deletions(-) diff --git a/docs/adr/0019-config-api-response-passthrough.md b/docs/adr/0019-config-api-response-passthrough.md index 10daae4c75..d50737b8b2 100644 --- a/docs/adr/0019-config-api-response-passthrough.md +++ b/docs/adr/0019-config-api-response-passthrough.md @@ -37,37 +37,55 @@ API-sourced config values carry the raw response, governed by five rules: `WeakMap`) rather than an ordinary object field, so `JSON.stringify`, object spread, and any encoder outside this package — not just the package's own — cannot observe it by construction. - The metadata-key strip rule below is defense-in-depth for this package's - own structural walks, not the sole safeguard. + It is not a declared field of any decode schema: file decoding can never + produce it, and the API mapping attaches it only after decode. A + `_apiResponse` key found inside a config *file* is therefore ordinary + unknown input at struct positions and ordinary user data at dynamic + record positions — presence remains a reliable API-provenance signal. + Invisibility to serialization implies invisibility to generic copying: + `{ ...config }`, `structuredClone`, and serializing state stores yield a + valid config value *without* the raw response, by design. Safety outranks + propagation here, and absence is already a defined state under this rule; + the package exposes a read accessor and the mapping's attach step for + consumers that must carry the raw across a copy boundary explicitly. 2. **Lenient decode, applied before the strict generated schema sees the body.** The CLI fetches the v2 config response via `@supabase/api`'s `executeRaw` (`packages/api/src/internal/client.ts`), not `execute`. `execute` decodes through the generated `V2GetProjectConfigOutput` `Schema.Struct`, which discards excess properties, so by the time `@supabase/config`'s lenient schema ran on that output any API-ahead field - would already be gone. `executeRaw` returns the response undecoded; - `@supabase/config`'s schema then decodes `data.attributes` and must pass - unknown keys through without failing. This lenient decode is the primary - protection against API-ahead-of-package skew; `_apiResponse` is only the - access mechanism for what it let through. -3. **One metadata-key rule, scoped to fixed struct shapes.** `_apiResponse` - and `$schema` are metadata, not config, wherever they occur as a declared - field of a fixed `Schema.Struct` (the config document root, or the value - object an API-sourced section attaches `_apiResponse` to). The rule is a - position match on those two reserved names, not a `$`/`_` prefix match, - and it never reaches into a dynamic `Schema.Record` key space: function - slugs (`functions.`, e.g. `functions._health`), edge runtime secret - names (`edge_runtime.secrets.`, e.g. `secrets._TOKEN`), env var - names, and other user-named keys keep whatever leading `_`/`$` character - they're given, because those keys are user data, not metadata, regardless - of spelling. Reserved-name matches are excluded from every structural walk - (sparse subtraction, default omission, value-origin tracking) and from - every encode/persist path. The `$schema` key preserved by `io.ts` is the - existing precedent; `_apiResponse` is the second member of the same rule, - implemented once in the shared walk core rather than per call site. -4. **Never persisted.** Because encode strips metadata keys, `_apiResponse` - (and its HMAC'd secret digests) can never land in `config.toml` / - `config.json`, including via `config pull`-style flows. + would already be gone. `executeRaw` returns the response undecoded and + deliberately does not filter on HTTP status, so the caller checks the + status and maps non-2xx responses to API errors before any decode — as + existing `executeRaw` callers do — rather than feeding an error body into + the config schema. `@supabase/config`'s schema then decodes + `data.attributes` and must pass unknown keys through without failing. + This lenient decode is the primary protection against + API-ahead-of-package skew; `_apiResponse` is only the access mechanism + for what it let through. +3. **Metadata is invisible to walks by construction, not filtered by key.** + Neither reserved name is excluded from the structural walks (sparse + subtraction, default omission, value-origin tracking) by a key check in + the walk core — the walks never see them at all. `$schema` lives entirely + at the io boundary: `io.ts` reads it off the raw parsed document before + schema decode and re-attaches it on write, so it never exists on a + decoded config value. `_apiResponse` is non-enumerable (rule 1), so + `Object.entries`-based walks — `sparse.ts`'s `subtractValue` included — + skip it without knowing it exists. No walk needs schema or path context, + and dynamic `Schema.Record` key spaces are safe automatically: an + enumerable, file-supplied key that merely spells a reserved name + (`functions._apiResponse` as a function slug, + `edge_runtime.secrets._TOKEN` as a secret name) is user data and flows + through every walk normally, because metadata status comes from *how the + property is attached*, never from how the key is spelled. +4. **Divergent persistence policies.** The reserved names share walk + invisibility but not a persistence rule. `$schema` is document metadata + that must be written: `io.ts`'s `toConfigDocument` re-attaches it on + every persist so editor and schema tooling keep working. `_apiResponse` + must never be written — and cannot be: non-enumerable means no encoder, + this package's or a consumer's, can see it, so it (and its HMAC'd secret + digests) cannot land in `config.toml`/`config.json`, including via + `config pull`-style flows. 5. **Fallback, not contract — with a secret carve-out.** When a raw key later graduates into the typed mapping, the typed field wins; the raw key remains readable but its naming, units, or polarity may differ from the @@ -78,11 +96,16 @@ API-sourced config values carry the raw response, governed by five rules: returns an HMAC digest for these, never the underlying value, so letting the typed field win would hand the digest to the normal encoder and persist it despite rule 4. For `x-secret` fields the mapping omits the - API-sourced value entirely, leaving any locally configured value - untouched. Consumers needing "which API fields does this package version - not understand" use a registry-derived `unmappedApiFields()` helper (raw - attributes minus the keys the mapping consumed) rather than a second - stored field. + API-sourced value entirely. Omission alone only governs the mapping: + flows that rewrite a config file (a `config pull` that persists the + merged result) must source `x-secret` fields from the existing local + document, or the rewrite would drop the user's secret along with the + digest; and drift comparison must treat `x-secret` fields as not + comparable, since a local plaintext or `env(...)` reference can never + meaningfully equal a remote digest. Consumers needing "which API fields + does this package version not understand" use a registry-derived + `unmappedApiFields()` helper (raw attributes minus the keys the mapping + consumed) rather than a second stored field. ## Rationale @@ -93,14 +116,16 @@ API-sourced config values carry the raw response, governed by five rules: generated client's strict `Schema.Struct` decode already drops excess properties, so a lenient decode layered on top of it would have nothing left to be lenient about. -- Rule 3 is scoped to fixed struct positions, not a `$`/`_` prefix match, - because the config schema uses dynamic `Schema.Record` keys (function - slugs, edge runtime secret names, env var names) that legitimately start - with either character. A prefix match would make the metadata rule silently - drop user config through the same structural walks it's meant to protect. - Scoped to reserved names, it still exists because the ADR 0018 subtraction - core compares config values structurally: a remote config carrying - `_apiResponse` would otherwise diff as drifted against every baseline, +- Rule 3 rejects both a `$`/`_` prefix match and schema-aware key filtering + in the walk core. A prefix match would silently drop user config: dynamic + `Schema.Record` keys (function slugs, edge runtime secret names, env var + names) legitimately start with either character. And threading schema/path + context through `subtractValue` just to recognize two reserved names is + complexity with no payoff once those names never occupy enumerable + positions in the first place — attachment mode, not key inspection, is + what makes a property metadata. Walk invisibility still matters because + the ADR 0018 subtraction core compares config values structurally: an + enumerable `_apiResponse` would diff as drifted against every baseline, producing exactly the phantom-drift false positives the drift feature is trying to eliminate. - Storing the whole raw response (rule 1) rather than only the unmapped @@ -108,15 +133,20 @@ API-sourced config values carry the raw response, governed by five rules: the leftovers are derivable and shrink release-over-release, which is correct semantics for a helper but confusing semantics for stored data. Non-enumerable/out-of-band storage is required rather than a plain field - because the metadata-key rule only protects this package's own walks — - a shared npm contract also gets serialized by `JSON.stringify`, object - spread, and consumer-written encoders this package doesn't control. -- The rule 5 secret carve-out exists because typed graduation and the - metadata-key rule solve different problems: metadata-key strips the raw - copy, but a graduated secret's *typed* value is not metadata-keyed and - remains eligible for the normal encoder. Without the carve-out, an HMAC - digest could reach `config.toml` through the typed field even though rule - 4 successfully blocked the raw one. + because a shared npm contract gets serialized by `JSON.stringify`, object + spread, and consumer-written encoders this package doesn't control. The + same mechanism that hides the raw from serializers necessarily hides it + from generic copies — a property `JSON.stringify` skips is a property + `{ ...spread }` skips. The ADR resolves that tension in favor of safety: + a copy that silently kept secret digests would be a leak, while a copy + that drops the escape hatch is just a config value in the absence state + rule 1 already defines. +- The rule 5 secret carve-out exists because typed graduation and rule 3 + solve different problems: walk invisibility hides the raw copy, but a + graduated secret's *typed* value is an ordinary enumerable config field + and remains eligible for the normal encoder. Without the carve-out, an + HMAC digest could reach `config.toml` through the typed field even though + rule 4 successfully blocked the raw one. ## Consequences @@ -124,8 +154,8 @@ API-sourced config values carry the raw response, governed by five rules: - Consumers are decoupled from the package's publish cadence for read access to new API fields. -- One metadata-key rule covers `$schema` and `_apiResponse` together, in one - shared walk implementation. +- Both reserved names stay out of the structural walks by construction — + zero key-checks in the walk core to maintain or get wrong. - `unmappedApiFields()` doubles as a mapping-completeness check: a nonempty result on a known API version is a to-do list for the registry. @@ -137,13 +167,20 @@ API-sourced config values carry the raw response, governed by five rules: - Two representations of graduated fields coexist (raw and typed) and can disagree in naming/units/polarity; the documented typed-wins precedence, minus the secret carve-out, mitigates but cannot remove the confusion risk. -- Every future structural walk must honor the metadata-key rule, and that - rule must stay position-scoped (reserved struct field, not string prefix) - as new dynamic `Schema.Record` sections are added; keeping the walks - funneled through the shared core is what makes this tractable. +- The guardrail rests on attachment invariants: `_apiResponse` is safe only + while attached non-enumerably (or out-of-band), `$schema` only while it + stays at the io boundary. A refactor that reifies either as an ordinary + enumerable field silently reintroduces both phantom drift and the + persistence leak. +- Generic copies (`{ ...spread }`, `structuredClone`, serializing state + stores) drop the raw metadata by design; a consumer that must carry it + across such a boundary re-attaches it explicitly via the package's + accessor/attach helpers. - The mapping layer must special-case every `x-secret`-annotated field to skip typed graduation, rather than treating all API-sourced fields - uniformly; missing one lets a digest reach the encoder. + uniformly; missing one lets a digest reach the encoder. Pull-style write + flows carry the matching obligation to source `x-secret` fields from the + local document, and drift comparison must skip them. - CLI callers must use `executeRaw` for this endpoint instead of the generated `execute`, forgoing the generated client's typed output for the v2 config response specifically. @@ -153,11 +190,12 @@ API-sourced config values carry the raw response, governed by five rules: 1. **Store only unmapped leftovers (`_unmapped`)**: contents change with every package release even when the API response is identical; unstable stored data. Derive leftovers via helper instead. -2. **Return a `{ config, raw }` pair instead of embedding**: keeps config - values walk-clean by construction, but the raw half does not travel with - the value through application code and state stores; each consumer would - rebuild the pairing. Embedding plus the strip rule is more ergonomic for - the same safety. +2. **Return a `{ config, raw }` pair instead of embedding**: equally + walk-clean, but the pair must be threaded through every signature and + store even for the dominant in-process case, where a non-enumerable + embedded property travels free with the object reference. Both designs + lose the raw across serializing boundaries (see Consequences), so the + pair's only advantage disappears while its ergonomic cost remains. 3. **Strict decode of the v2 attributes**: rejected outright; see Rationale. 4. **Inline passthrough of unknown keys per section** (zod `.passthrough()`-style): unknown API keys arrive in API naming/units, @@ -167,8 +205,8 @@ API-sourced config values carry the raw response, governed by five rules: ## Related Decisions -- ADR 0018: sparse config subtraction — defines the structural walks that the - metadata-key rule scopes out. +- ADR 0018: sparse config subtraction — defines the structural walks that + rule 3 keeps the metadata invisible to. - ADR 0009: configuration schema & validation. ## See Also From 57df21cd2c44e8e6f239e8cf3953c918ba86fd58 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 25 Aug 2026 09:55:06 +0100 Subject: [PATCH 4/5] docs(config): address avallete review on ADR 0019 Reconcile mapping ownership with ADR 0018: the API-to-ProjectConfig translation originally assigned to the diff core (CLI-2156) now lives in @supabase/config so the CLI and Studio share one mapper; 0018's two ownership sentences are amended and both ADRs cross-reference the supersession. Clarify rule 2 that lenient decode tolerates unknown keys without inlining them onto ProjectConfig (Alternative 4 stays rejected). Swap the rule 5 carve-out example from the local-only edge_runtime.secrets.* to Auth x-secret fields that actually map from the v2 auth attribute record. Specify unmappedApiFields() as a deep path subtraction against the registry of mapped API paths, not a top-level key subtract. --- docs/adr/0018-sparse-config-subtraction.md | 5 +-- .../0019-config-api-response-passthrough.md | 34 +++++++++++++------ 2 files changed, 27 insertions(+), 12 deletions(-) diff --git a/docs/adr/0018-sparse-config-subtraction.md b/docs/adr/0018-sparse-config-subtraction.md index 0bd3e91062..56aa6826fc 100644 --- a/docs/adr/0018-sparse-config-subtraction.md +++ b/docs/adr/0018-sparse-config-subtraction.md @@ -25,7 +25,7 @@ All functions are pure and synchronous, operating on decoded config values, with ## Rationale -- **One core instead of a cascade.** The merge-and-prune cascade sketched in CLI-2155's planning comment reduces algebraically to `subtract(merge(local, remote), defaults)`. Exporting the subtraction gives `config diff` and `config pull` the shared comparison core without binding this package to the Management API's response shape (translating that shape to `ProjectConfig` is CLI-2156's concern). +- **One core instead of a cascade.** The merge-and-prune cascade sketched in CLI-2155's planning comment reduces algebraically to `subtract(merge(local, remote), defaults)`. Exporting the subtraction gives `config diff` and `config pull` the shared comparison core while keeping the subtraction itself independent of the Management API's response shape. (This ADR originally assigned the translation of that shape to `ProjectConfig` to the diff core, CLI-2156; ADR 0019 supersedes that assignment — the mapping lives in `@supabase/config` as a separate layer so the CLI and Studio share one mapper. The subtraction core's independence from the API shape is unchanged.) - **Defaults derived, not duplicated.** Every field's default already lives in the schema (`default` annotations plus `withDecodingDefaultKey`). Decoding `{}` materializes them; a parallel hand-written defaults object would drift. - **Strict deep equality, array order matters.** Order is semantically load-bearing for values like `api.extra_search_path` (Postgres `search_path` resolution order). Treating reordered arrays as equal would prune values that behave differently from the default. The false-positive cost (a semantically-default-but-reordered array survives as harmless noise) is far cheaper than wrongly deleting meaningful config. - **Dropping empty sections is lossless.** Every section carries a section-level decoding default, so an absent section and an empty section decode identically; empty carcass headers are pure diff noise. @@ -49,7 +49,7 @@ All functions are pure and synchronous, operating on decoded config values, with ## Alternatives Considered -1. **Merge-and-prune cascade as a single function** (`(apiResponse, configToml) → massaged config`): binds `@supabase/config` to the Management API response shape and entangles this package with remote-to-local translation, which belongs to the diff core (CLI-2156). +1. **Merge-and-prune cascade as a single function** (`(apiResponse, configToml) → massaged config`): entangles the subtraction with remote-to-local translation in one function. (ADR 0019 later moved the translation itself into this package as a separate mapping layer; the rejection stands — mapping and subtraction remain separate functions.) 2. **Recursing into `remotes` against global defaults**: looks obviously correct, is subtly wrong — pruning a remote's `api.max_rows = 1000` (global default) under a base that sets `500` changes the branch's effective value from 1000 to 500. 3. **Hand-written defaults reference object**: duplicates ~100 defaults already declared in the schema and drifts silently. 4. **Order-insensitive array comparison**: prunes reordered arrays whose order is semantically meaningful (`extra_search_path`). @@ -58,6 +58,7 @@ All functions are pure and synchronous, operating on decoded config values, with - [ADR 0009](0009-configuration-schema-and-validation.md): Configuration Schema & Validation — the umbrella charter this decision answers a slice of (default config generation, `@supabase/config` package architecture) - [ADR 0006](0006-environment-management.md): Environment Management — remote blocks and branch mapping semantics +- [ADR 0019](0019-config-api-response-passthrough.md): Raw API-Response Passthrough — supersedes this ADR's assignment of API→`ProjectConfig` translation to the diff core (CLI-2156): the mapping lives in `@supabase/config` so the CLI and Studio share one implementation, while the subtraction core stays independent of the API shape ## See Also diff --git a/docs/adr/0019-config-api-response-passthrough.md b/docs/adr/0019-config-api-response-passthrough.md index d50737b8b2..085b0bd9fa 100644 --- a/docs/adr/0019-config-api-response-passthrough.md +++ b/docs/adr/0019-config-api-response-passthrough.md @@ -59,10 +59,12 @@ API-sourced config values carry the raw response, governed by five rules: status and maps non-2xx responses to API errors before any decode — as existing `executeRaw` callers do — rather than feeding an error body into the config schema. `@supabase/config`'s schema then decodes - `data.attributes` and must pass unknown keys through without failing. - This lenient decode is the primary protection against - API-ahead-of-package skew; `_apiResponse` is only the access mechanism - for what it let through. + `data.attributes` and must tolerate unknown keys without failing. + Tolerate, not inline: an unknown key never lands on the decoded + `ProjectConfig` (that is Alternative 4, rejected) — it stays reachable + only through the raw object rule 1 attaches. This lenient decode is the + primary protection against API-ahead-of-package skew; `_apiResponse` is + the only access mechanism for what it tolerated. 3. **Metadata is invisible to walks by construction, not filtered by key.** Neither reserved name is excluded from the structural walks (sparse subtraction, default omission, value-origin tracking) by a key check in @@ -92,7 +94,9 @@ API-sourced config values carry the raw response, governed by five rules: typed field (the API↔config mapping includes renames, boolean inversions, and unit conversions). This precedence does not apply to fields the schema annotates `x-secret` (`packages/config/src/lib/env.ts`'s `secret()` / - `env({ secret: true })`, e.g. `edge_runtime.secrets.*`): the API only ever + `env({ secret: true })` — e.g. `auth.external..secret` or + `auth.captcha.secret`, which map from the v2 response's `auth` attribute + record): the API only ever returns an HMAC digest for these, never the underlying value, so letting the typed field win would hand the digest to the normal encoder and persist it despite rule 4. For `x-secret` fields the mapping omits the @@ -102,10 +106,16 @@ API-sourced config values carry the raw response, governed by five rules: document, or the rewrite would drop the user's secret along with the digest; and drift comparison must treat `x-secret` fields as not comparable, since a local plaintext or `env(...)` reference can never - meaningfully equal a remote digest. Consumers needing "which API fields - does this package version not understand" use a registry-derived - `unmappedApiFields()` helper (raw attributes minus the keys the mapping - consumed) rather than a second stored field. + meaningfully equal a remote digest. Local-only secrets such as + `edge_runtime.secrets.*` never appear in the API response at all; the + graduation carve-out is moot for them, and the source-from-local + obligation is what keeps them in the file. Consumers needing "which API + fields does this package version not understand" use a registry-derived + `unmappedApiFields()` helper rather than a second stored field. The + subtraction is by path, not by top-level key: the registry records every + mapped API path (`auth.`, `database.major_version`, …) and the + helper walks the raw attributes deeply, so a new field nested inside an + existing section — the common API-ahead case — still surfaces. ## Rationale @@ -206,7 +216,11 @@ API-sourced config values carry the raw response, governed by five rules: ## Related Decisions - ADR 0018: sparse config subtraction — defines the structural walks that - rule 3 keeps the metadata invisible to. + rule 3 keeps the metadata invisible to. This ADR supersedes 0018's + assignment of API→`ProjectConfig` translation to the diff core (CLI-2156): + the mapping lives in `@supabase/config` so the CLI and Studio share one + implementation. 0018 is amended accordingly; its subtraction core stays + independent of the API shape. - ADR 0009: configuration schema & validation. ## See Also From f03dd0cec4c8520320ec09eac570bb7f04b14a23 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 25 Aug 2026 10:09:56 +0100 Subject: [PATCH 5/5] docs(config): fix oxfmt formatting in ADR 0019 --- docs/adr/0019-config-api-response-passthrough.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/adr/0019-config-api-response-passthrough.md b/docs/adr/0019-config-api-response-passthrough.md index 085b0bd9fa..b6e4c2ccb2 100644 --- a/docs/adr/0019-config-api-response-passthrough.md +++ b/docs/adr/0019-config-api-response-passthrough.md @@ -39,12 +39,12 @@ API-sourced config values carry the raw response, governed by five rules: package — not just the package's own — cannot observe it by construction. It is not a declared field of any decode schema: file decoding can never produce it, and the API mapping attaches it only after decode. A - `_apiResponse` key found inside a config *file* is therefore ordinary + `_apiResponse` key found inside a config _file_ is therefore ordinary unknown input at struct positions and ordinary user data at dynamic record positions — presence remains a reliable API-provenance signal. Invisibility to serialization implies invisibility to generic copying: `{ ...config }`, `structuredClone`, and serializing state stores yield a - valid config value *without* the raw response, by design. Safety outranks + valid config value _without_ the raw response, by design. Safety outranks propagation here, and absence is already a defined state under this rule; the package exposes a read accessor and the mapping's attach step for consumers that must carry the raw across a copy boundary explicitly. @@ -78,8 +78,8 @@ API-sourced config values carry the raw response, governed by five rules: enumerable, file-supplied key that merely spells a reserved name (`functions._apiResponse` as a function slug, `edge_runtime.secrets._TOKEN` as a secret name) is user data and flows - through every walk normally, because metadata status comes from *how the - property is attached*, never from how the key is spelled. + through every walk normally, because metadata status comes from _how the + property is attached_, never from how the key is spelled. 4. **Divergent persistence policies.** The reserved names share walk invisibility but not a persistence rule. `$schema` is document metadata that must be written: `io.ts`'s `toConfigDocument` re-attaches it on @@ -153,7 +153,7 @@ API-sourced config values carry the raw response, governed by five rules: rule 1 already defines. - The rule 5 secret carve-out exists because typed graduation and rule 3 solve different problems: walk invisibility hides the raw copy, but a - graduated secret's *typed* value is an ordinary enumerable config field + graduated secret's _typed_ value is an ordinary enumerable config field and remains eligible for the normal encoder. Without the carve-out, an HMAC digest could reach `config.toml` through the typed field even though rule 4 successfully blocked the raw one.