diff --git a/.changeset/metadata-redaction-seam.md b/.changeset/metadata-redaction-seam.md new file mode 100644 index 0000000000..9e53620909 --- /dev/null +++ b/.changeset/metadata-redaction-seam.md @@ -0,0 +1,34 @@ +--- +"@objectstack/spec": minor +"@objectstack/service-datasource": patch +--- + +feat(spec): per-type metadata read-path redaction seam in `kernel`, and ONE definition of "what is a credential key" in `data` (#8300) + +Two additions to `@objectstack/spec`, both enabling #8154's security invariant +(stored credentials must never serve cleartext on the metadata read path): + +- **`kernel/metadata-type-redaction.ts`** — `registerMetadataTypeRedactor` / + `getMetadataTypeRedactor` / `listMetadataTypeRedactorTypes`, the same + built-in-map + runtime-overlay registry pattern as its siblings + `registerMetadataTypeSchema` and `registerMetadataTypeActions`. The + `datasource` redactor is wired as a **built-in** (present the moment the + module loads), because registering it from the opt-in datasource-admin + plugin is measured fail-open: `sys_metadata` rows and the `/meta` read exits + exist without that plugin. +- **`data/datasource-credential-redaction.ts`** — the credential-key + derivation and read-path redaction previously in + `@objectstack/service-datasource` (`refusedCredentialKeys`, + `redactableConfigKeys`, `redactUrlPassword`, `redactDatasourceConfig`, + `RedactedDatasourceConfig`), moved here so the datasource-admin read path + and the metadata read path share one security list. The key set is derived + from each driver's own `z.never()` contract plus the pre-#8078 alias list + and turso's still-writable `encryptionKey` — byte-equal to what the + service-datasource original derived, pinned by test. + +`@objectstack/service-datasource` re-exports the moved names from +`@objectstack/spec/data` (existing imports keep compiling; behaviour +unchanged) and keeps `restoreRedactedConfig`, the admin service's write-path +inverse. + + diff --git a/packages/services/service-datasource/src/datasource-config-redaction.ts b/packages/services/service-datasource/src/datasource-config-redaction.ts index be3e3c7f52..91b3ecbcb5 100644 --- a/packages/services/service-datasource/src/datasource-config-redaction.ts +++ b/packages/services/service-datasource/src/datasource-config-redaction.ts @@ -2,242 +2,47 @@ /** * Read-path credential redaction for a datasource's driver `config` (#8081, - * the services half of #7990). + * the services half of #7990) — and the write-path inverse that keeps the + * redaction from turning "Save" into credential deletion. * - * #8078 closed the WRITE door: `config.password` / `config.authToken` are - * declared-unwritable (`z.never()`) on every driver that has them, so no new - * row can carry an inline credential. It could not close the READ door, and it - * did not try: rows written before it still carry cleartext, and - * `DatasourceAdminService.getDatasource()` handed `config` back verbatim while - * its own doc comment claimed the credential had been stripped. This module is - * the strip that comment described. + * ## Where the definition lives now (#8300) * - * ## What counts as a credential here + * The derivation half — what counts as a credential key, and the read-path + * redaction built on it — MOVED to `@objectstack/spec/data` + * (`datasource-credential-redaction.ts`), so that this package and the + * metadata read path (#8154, via the `kernel/metadata-type-redaction.ts` + * seam) share ONE security list instead of two derived copies that must + * agree. The re-exports below keep every existing consumer of this module + * compiling unchanged; the moved module's header carries the full rationale + * (three key sources, the unknown-driver posture, URL-userinfo boundaries). * - * Three sources, in descending order of authority: - * - * 1. **Derived from the driver's own contract** — a config key whose schema is - * `z.never()` IS the shape #8078 gave a refused inline credential - * (`refusedInlineCredentialKey`), so reading the schema is reading the - * refusal list rather than re-typing it. A driver that refuses a new - * credential key tomorrow is covered here the day it lands, which a - * hand-maintained list in this package would not be. - * 2. **Former alias spellings** ({@link FORMER_CREDENTIAL_ALIASES}) — `passwd` - * / `pwd` / `token` / `jwt` / `auth_token` / `authtoken` used to be - * `aliases` that the parse RENAMED onto the canonical key; #8078 moved them - * to `guidance`, which refuses them. Neither spelling appears in the schema - * shape, and a stored row never went through the parse that would have - * renamed it — the wizard persists through `metadata.register`, whose - * validation is a structural name/label check. So the only place these can - * still be found is exactly the place this module reads: a stored row. - * 3. **Credential-shaped keys that are still WRITABLE** ({@link - * STILL_WRITABLE_CREDENTIAL_KEYS}) — today just turso's `encryptionKey`, an - * AES-256 key the binder has no slot for (#8081 scope item 4, which owns - * the decision about giving it one). Redacting it on READ neither grants - * nor removes that slot: the key stays writable, stays stored, and stays - * injected at connect. It simply stops being served back in cleartext, - * which is the one question this module answers. - * - * For a driver the platform ships no contract for, source 1 is empty — the - * registry is saying "nothing to check against", not "nothing to protect". The - * canonical spellings are therefore redacted by NAME for unknown drivers too. - * That asymmetry with the write gate (which deliberately lets an unknown - * driver's config through untouched) is intentional: declining to REFUSE an - * unrecognised key is a boundary choice about authoring, while serving a key - * literally named `password` back in cleartext is a leak under any boundary. - * - * ## URL-embedded credentials - * - * A `postgresql://user:pass@host/db` in `config.url` carries the same secret as - * `config.password`. When this module landed, refusing it was explicitly - * UNRULED (#7990/#8078 pinned the acceptance as a fact); #8082 has since ruled - * it (maintainer 2026-08-12, Option A), and the WRITE door now refuses a URL - * userinfo password via the spec's shared value-level parse - * (`urlUserinfoPassword`, `@objectstack/spec` `data/driver/common.zod.ts`). - * This module is still the READ half: a scrub that dropped `config.password` - * and then served the identical credential one key over would be a scrub in - * name only — the same "claims a protection it does not perform" shape #8081 - * exists to end. So the read path redacts the PASSWORD COMPONENT of a URL's - * userinfo and leaves everything else, including the username, byte-for-byte — - * and the redacted shape it serves (`user@host`) is exactly what the write - * door still accepts, which is what keeps an untouched "Save" on a legacy row - * working. - * - * ## Why redaction must be reversible + * ## What stays here: {@link restoreRedactedConfig} * * `getDatasource()` feeds the Studio edit form, and `updateDatasource()` takes * that form's `config` back as a whole-object patch. A scrub with no inverse - * would therefore turn every "Save" on an unmodified form into silent credential - * DELETION — trading a disclosure bug for a data-loss bug. {@link - * restoreRedactedConfig} is that inverse, and it is the same rule the secret - * path next to it has always used ("preserve the existing `credentialsRef` - * unless a new secret rewraps it"), applied to the material this module hides. - */ - -import { getDriverConfigSchema } from '@objectstack/spec/data'; - -/** - * Canonical inline-credential spellings, used for a driver whose contract this - * platform does not ship. Kept in sync with the schemas by - * `datasource-config-redaction.test.ts`, which asserts every `z.never()` key - * across every builtin driver appears here — so a new refused key cannot land - * without this fallback learning it. - */ -const CANONICAL_CREDENTIAL_KEYS = ['password', 'authToken'] as const; - -/** - * Pre-#8078 alias spellings of the keys above. A row written through the wizard - * (which does not parse) can hold these verbatim; a row written through an - * authoring door had them renamed onto the canonical key before storage. + * would therefore turn every "Save" on an unmodified form into silent + * credential DELETION — trading a disclosure bug for a data-loss bug. + * {@link restoreRedactedConfig} is that inverse, and it is the same rule the + * secret path next to it has always used ("preserve the existing + * `credentialsRef` unless a new secret rewraps it"), applied to the material + * the redaction hides. It stays in this package because restoration is a + * write policy of the admin service's own edit round-trip, not a spec-derived + * fact — and the generic metadata write door's equivalent carry-forward is + * #8154's, deliberately not built here. */ -const FORMER_CREDENTIAL_ALIASES = [ - 'passwd', - 'pwd', - 'token', - 'jwt', - 'auth_token', - 'authtoken', -] as const; -/** - * Credential-shaped config keys that remain WRITABLE by deliberate spec choice, - * and so are never found by the `z.never()` derivation. - * - * `encryptionKey` (turso) is an AES-256 key for the local database file. #8078 - * left it writable because the datasource secret binder injects exactly one - * secret slot and `external.credentialsRef` resolution cannot target a second - * one; giving it a slot is #8081 scope item 4 and is NOT decided here. - */ -const STILL_WRITABLE_CREDENTIAL_KEYS: Record = { - turso: ['encryptionKey'], -}; - -/** Unwrap `.optional()` / `.default()` / `.nullable()` down to the base type. */ -function baseTypeOf(schema: unknown): string | undefined { - let node: any = schema; - for (let depth = 0; node && depth < 10; depth += 1) { - const def = node.def ?? node._def; - const type: string | undefined = def?.type; - if (!type) return undefined; - if (type === 'optional' || type === 'default' || type === 'nullable' || type === 'readonly') { - node = def.innerType; - continue; - } - return type; - } - return undefined; -} - -/** - * The inline-credential keys a driver's own contract declares unwritable. - * - * Empty for a driver with no shipped contract — see the module note on why the - * canonical spellings are still redacted in that case. - */ -export function refusedCredentialKeys(driver: unknown): string[] { - let shape: Record | undefined; - try { - const schema: any = getDriverConfigSchema(driver as never); - const raw = schema?.shape; - shape = typeof raw === 'function' ? raw() : raw; - } catch { - return []; - } - if (!shape) return []; - return Object.entries(shape) - .filter(([, member]) => baseTypeOf(member) === 'never') - .map(([key]) => key); -} +import { redactableConfigKeys, redactUrlPassword } from '@objectstack/spec/data'; -/** Every config key this module hides for `driver`, canonical + alias + writable-but-secret. */ -export function redactableConfigKeys(driver: unknown): string[] { - const derived = refusedCredentialKeys(driver); - const canonical = derived.length > 0 ? derived : [...CANONICAL_CREDENTIAL_KEYS]; - const stillWritable = typeof driver === 'string' ? (STILL_WRITABLE_CREDENTIAL_KEYS[driver] ?? []) : []; - return [...new Set([...canonical, ...FORMER_CREDENTIAL_ALIASES, ...stillWritable])]; -} - -/** - * `scheme://[user[:password]@]rest`. Anchored, and every class excludes `/?#` - * so a password-looking substring in a path or query cannot be mistaken for one - * — `https://host/a:b@c` has no userinfo and must come back untouched. - * - * The password group deliberately ALLOWS `@` and is greedy, which (with - * backtracking) makes the match end at the LAST `@` before the path — the - * userinfo boundary RFC 3986 actually defines. A lazier class stopping at the - * first `@` would split `postgres://u:p@ss@host/db` after `p`, leave `ss@host` - * in place, and publish a fragment of the password while looking redacted. - * Such a URL is malformed (a literal `@` in userinfo must be `%40`), which is - * precisely why it must not be the case that decides how much leaks. - */ -const URL_USERINFO_RE = /^([a-z][a-z0-9+.\-]*:\/\/)([^/?#@:]*)(:[^/?#]*)@/i; - -/** - * Strip the password component from a URL's userinfo, preserving the scheme, - * the username, and everything from the host onward. - * - * Returns the input unchanged when there is nothing to strip, which is what - * makes "did this value change?" a usable test for whether a credential was - * present. - */ -export function redactUrlPassword(value: string): string { - return value.replace(URL_USERINFO_RE, (_m, scheme: string, user: string) => `${scheme}${user}@`); -} - -/** A driver `config` with its credential material removed, and what was removed. */ -export interface RedactedDatasourceConfig { - config: Record; - /** - * Config keys whose value was removed or rewritten, sorted. Serving this - * alongside the redacted config is the difference between a caller that knows - * a credential is being withheld and one that infers it from an absence. - */ - redactedKeys: string[]; -} - -/** - * Remove every stored credential from a driver `config` for serving on a read - * path. - * - * Pure: the input object is never mutated, so a caller holding the stored - * record (the connect path does) is unaffected. - */ -export function redactDatasourceConfig( - driver: unknown, - config: Record | undefined, -): RedactedDatasourceConfig { - if (!config || typeof config !== 'object') return { config: {}, redactedKeys: [] }; - - const hidden = new Set(redactableConfigKeys(driver)); - const out: Record = {}; - const redactedKeys: string[] = []; - - for (const [key, value] of Object.entries(config)) { - if (hidden.has(key)) { - // Dropped, not masked. A mask would round-trip back through the wizard as - // a literal new password, and post-#8078 the canonical spellings would - // then be REFUSED at the write door — turning an untouched "Save" into an - // error the author cannot act on. An absent key is the shape the form - // already understands from `hasSecret`. - if (value !== undefined) redactedKeys.push(key); - continue; - } - if (typeof value === 'string') { - const redacted = redactUrlPassword(value); - if (redacted !== value) { - out[key] = redacted; - redactedKeys.push(key); - continue; - } - } - out[key] = value; - } - - return { config: out, redactedKeys: redactedKeys.sort() }; -} +export { + refusedCredentialKeys, + redactableConfigKeys, + redactUrlPassword, + redactDatasourceConfig, + type RedactedDatasourceConfig, +} from '@objectstack/spec/data'; /** - * Re-apply the credential material {@link redactDatasourceConfig} hid, for a + * Re-apply the credential material `redactDatasourceConfig` hid, for a * patch that is round-tripping a previously-read config back to the store. * * The rule is deliberately narrow: stored material is carried forward ONLY diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index 9e3a4e7fc8..4d23024db8 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -472,6 +472,7 @@ "RecordSurface (type)", "RecordSurfaceOptions (interface)", "RecordSurfaceViewport (type)", + "RedactedDatasourceConfig (interface)", "ReferenceIdValue (type)", "ReferenceIdValueSchema (const)", "ReferenceResolution (type)", @@ -702,10 +703,14 @@ "platformProvisionsStorage (function)", "provisionPrimary (function)", "readAutonumberCounter (function)", + "redactDatasourceConfig (function)", + "redactUrlPassword (function)", + "redactableConfigKeys (function)", "reduceFilterKeyVerdict (function)", "reduceFilterVerdict (function)", "referenceTargetOf (function)", "referencedFields (function)", + "refusedCredentialKeys (function)", "refusedInlineCredentialKey (function)", "renderAutonumber (function)", "resolveAutonumberFormat (function)", diff --git a/packages/spec/api-surface/kernel.json b/packages/spec/api-surface/kernel.json index 7326367def..d4c34e869a 100644 --- a/packages/spec/api-surface/kernel.json +++ b/packages/spec/api-surface/kernel.json @@ -226,7 +226,9 @@ "MetadataQueryResultSchema (const)", "MetadataQuerySchema (const)", "MetadataReadDecoration (type)", + "MetadataRedactionResult (interface)", "MetadataType (type)", + "MetadataTypeRedactor (type)", "MetadataTypeRegistryEntry (type)", "MetadataTypeRegistryEntryParsed (type)", "MetadataTypeRegistryEntrySchema (const)", @@ -492,6 +494,7 @@ "featureGatePredicate (function)", "getMetadataCreateSeed (function)", "getMetadataTypeActions (function)", + "getMetadataTypeRedactor (function)", "getMetadataTypeSchema (function)", "isConsumerInstallable (function)", "isKnownPlatformCapability (function)", @@ -500,10 +503,12 @@ "lintUnknownStackKeys (function)", "listLintableAuthoringCollections (function)", "listMetadataCreateSeedTypes (function)", + "listMetadataTypeRedactorTypes (function)", "listMetadataTypeSchemaTypes (function)", "listUnregisteredKindSchemaTypes (function)", "lowerRequiresFeature (function)", "registerMetadataTypeActions (function)", + "registerMetadataTypeRedactor (function)", "registerMetadataTypeSchema (function)", "resolveLockState (function)", "stripReadDecorations (function)", diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index ed47d25107..b4a14c2e1a 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -472,6 +472,7 @@ "RecordSurface": "src/data/record-surface.ts#RecordSurface (type)", "RecordSurfaceOptions": "src/data/record-surface.ts#RecordSurfaceOptions (interface)", "RecordSurfaceViewport": "src/data/record-surface.ts#RecordSurfaceViewport (type)", + "RedactedDatasourceConfig": "src/data/datasource-credential-redaction.ts#RedactedDatasourceConfig (interface)", "ReferenceIdValue": "src/data/field-value.zod.ts#ReferenceIdValue (type)", "ReferenceIdValueSchema": "src/data/field-value.zod.ts#ReferenceIdValueSchema (const)", "ReferenceResolution": "src/data/seed-loader.zod.ts#ReferenceResolution (type)", @@ -702,10 +703,14 @@ "platformProvisionsStorage": "src/data/injected-system-column-provenance.ts#platformProvisionsStorage (function)", "provisionPrimary": "src/data/display-name.ts#provisionPrimary (function)", "readAutonumberCounter": "src/data/autonumber-format.ts#readAutonumberCounter (function)", + "redactDatasourceConfig": "src/data/datasource-credential-redaction.ts#redactDatasourceConfig (function)", + "redactUrlPassword": "src/data/datasource-credential-redaction.ts#redactUrlPassword (function)", + "redactableConfigKeys": "src/data/datasource-credential-redaction.ts#redactableConfigKeys (function)", "reduceFilterKeyVerdict": "src/data/filter-verdict.ts#reduceFilterKeyVerdict (function)", "reduceFilterVerdict": "src/data/filter-verdict.ts#reduceFilterVerdict (function)", "referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)", "referencedFields": "src/data/autonumber-format.ts#referencedFields (function)", + "refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)", "refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)", "renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)", "resolveAutonumberFormat": "src/data/autonumber-format.ts#resolveAutonumberFormat (function)", diff --git a/packages/spec/export-origins/kernel.json b/packages/spec/export-origins/kernel.json index 689307e3f0..0a694296bd 100644 --- a/packages/spec/export-origins/kernel.json +++ b/packages/spec/export-origins/kernel.json @@ -226,7 +226,9 @@ "MetadataQueryResultSchema": "src/kernel/metadata-plugin.zod.ts#MetadataQueryResultSchema (const)", "MetadataQuerySchema": "src/kernel/metadata-plugin.zod.ts#MetadataQuerySchema (const)", "MetadataReadDecoration": "src/kernel/metadata-read-decorations.ts#MetadataReadDecoration (type)", + "MetadataRedactionResult": "src/kernel/metadata-type-redaction.ts#MetadataRedactionResult (interface)", "MetadataType": "src/kernel/metadata-plugin.zod.ts#MetadataType (type)", + "MetadataTypeRedactor": "src/kernel/metadata-type-redaction.ts#MetadataTypeRedactor (type)", "MetadataTypeRegistryEntry": "src/kernel/metadata-plugin.zod.ts#MetadataTypeRegistryEntry (type)", "MetadataTypeRegistryEntryParsed": "src/kernel/metadata-plugin.zod.ts#MetadataTypeRegistryEntryParsed (type)", "MetadataTypeRegistryEntrySchema": "src/kernel/metadata-plugin.zod.ts#MetadataTypeRegistryEntrySchema (const)", @@ -492,6 +494,7 @@ "featureGatePredicate": "src/kernel/public-auth-features.ts#featureGatePredicate (function)", "getMetadataCreateSeed": "src/kernel/metadata-create-seeds.ts#getMetadataCreateSeed (function)", "getMetadataTypeActions": "src/kernel/metadata-type-schemas.ts#getMetadataTypeActions (function)", + "getMetadataTypeRedactor": "src/kernel/metadata-type-redaction.ts#getMetadataTypeRedactor (function)", "getMetadataTypeSchema": "src/kernel/metadata-type-schemas.ts#getMetadataTypeSchema (function)", "isConsumerInstallable": "src/kernel/plugin.zod.ts#isConsumerInstallable (function)", "isKnownPlatformCapability": "src/kernel/platform-capabilities.ts#isKnownPlatformCapability (function)", @@ -500,10 +503,12 @@ "lintUnknownStackKeys": "src/kernel/metadata-authoring-lint.ts#lintUnknownStackKeys (function)", "listLintableAuthoringCollections": "src/kernel/metadata-authoring-lint.ts#listLintableAuthoringCollections (function)", "listMetadataCreateSeedTypes": "src/kernel/metadata-create-seeds.ts#listMetadataCreateSeedTypes (function)", + "listMetadataTypeRedactorTypes": "src/kernel/metadata-type-redaction.ts#listMetadataTypeRedactorTypes (function)", "listMetadataTypeSchemaTypes": "src/kernel/metadata-type-schemas.ts#listMetadataTypeSchemaTypes (function)", "listUnregisteredKindSchemaTypes": "src/kernel/metadata-type-schemas.ts#listUnregisteredKindSchemaTypes (function)", "lowerRequiresFeature": "src/kernel/public-auth-features.ts#lowerRequiresFeature (function)", "registerMetadataTypeActions": "src/kernel/metadata-type-schemas.ts#registerMetadataTypeActions (function)", + "registerMetadataTypeRedactor": "src/kernel/metadata-type-redaction.ts#registerMetadataTypeRedactor (function)", "registerMetadataTypeSchema": "src/kernel/metadata-type-schemas.ts#registerMetadataTypeSchema (function)", "resolveLockState": "src/kernel/metadata-protection.zod.ts#resolveLockState (function)", "stripReadDecorations": "src/kernel/metadata-read-decorations.ts#stripReadDecorations (function)", diff --git a/packages/spec/src/data/datasource-credential-redaction.test.ts b/packages/spec/src/data/datasource-credential-redaction.test.ts new file mode 100644 index 0000000000..e81e2a62d9 --- /dev/null +++ b/packages/spec/src/data/datasource-credential-redaction.test.ts @@ -0,0 +1,180 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * #8300 — the ONE definition of "what is a credential key", moved here from + * `service-datasource`'s `datasource-config-redaction.ts` (PR #8126). + * + * ## The derivation pin — the security-drift guard + * + * The first describe block pins, per driver, the EXACT key set the + * `service-datasource` original derived on the day of the move (byte-equal + * arrays, insertion order included). #8300's whole reason to exist is that a + * credential list duplicated across two doors drifts; the move is only safe if + * the moved module derives what the original derived. Any change to this set — + * a driver refusing a new key, an alias added or dropped — must be a + * deliberate edit to this pin, made in the same PR that changes the + * derivation's inputs, never an accident this test lets through. + * + * ## The write-door alignment pin + * + * `redactUrlPassword` (read half) and `urlUserinfoPassword` (write half, + * `driver/common.zod.ts`, #8082) draw the same RFC 3986 userinfo boundaries by + * design — the write door must refuse precisely the material the read door + * redacts. While they lived in different packages that claim was comment-only; + * now both are here, the property block below holds them aligned. + */ + +import { describe, expect, it } from 'vitest'; + +import { + BUILTIN_DRIVER_IDS, + getDriverConfigSchema, + urlUserinfoPassword, +} from './driver/index'; +import { + redactDatasourceConfig, + redactUrlPassword, + redactableConfigKeys, + refusedCredentialKeys, +} from './datasource-credential-redaction'; + +/** The pre-#8078 alias spellings — the hand-written half of the definition. */ +const ALIASES = ['passwd', 'pwd', 'token', 'jwt', 'auth_token', 'authtoken']; + +describe('derivation pin: the moved module derives EXACTLY what the service-datasource original derived', () => { + it('z.never() derivation, per driver with a shipped contract', () => { + expect(refusedCredentialKeys('postgres')).toEqual(['password']); + expect(refusedCredentialKeys('mysql')).toEqual(['password']); + expect(refusedCredentialKeys('mongodb')).toEqual(['password']); + expect(refusedCredentialKeys('turso')).toEqual(['authToken']); + // Credential-less drivers declare none, and must not acquire one by accident. + expect(refusedCredentialKeys('sqlite')).toEqual([]); + expect(refusedCredentialKeys('sqlite-wasm')).toEqual([]); + expect(refusedCredentialKeys('memory')).toEqual([]); + // No contract ⇒ no derived verdict (the fallback below covers the names). + expect(refusedCredentialKeys('not-a-real-driver')).toEqual([]); + }); + + it('full redactable set, byte-equal per driver (the #8300 drift guard)', () => { + // These literals ARE the pin: they reproduce, key for key and in order, + // what `service-datasource`'s `redactableConfigKeys` answered on + // origin/main at the move. Changing them is changing the platform's + // credential-redaction surface — do it deliberately, with the driver + // contract change that motivates it, never as clean-up. + expect(redactableConfigKeys('postgres')).toEqual(['password', ...ALIASES]); + expect(redactableConfigKeys('mysql')).toEqual(['password', ...ALIASES]); + expect(redactableConfigKeys('mongodb')).toEqual(['password', ...ALIASES]); + expect(redactableConfigKeys('turso')).toEqual(['authToken', ...ALIASES, 'encryptionKey']); + + // A driver whose contract refuses nothing — or that ships no contract at + // all — falls back to BOTH canonical spellings by name: the registry is + // saying "nothing to check against", not "nothing to protect". + const fallback = ['password', 'authToken', ...ALIASES]; + expect(redactableConfigKeys('sqlite')).toEqual(fallback); + expect(redactableConfigKeys('sqlite-wasm')).toEqual(fallback); + expect(redactableConfigKeys('memory')).toEqual(fallback); + expect(redactableConfigKeys('not-a-real-driver')).toEqual(fallback); + // A non-string driver value cannot index the still-writable table, and + // must not throw — a stored row's `driver` is whatever was stored. + expect(redactableConfigKeys(undefined)).toEqual(fallback); + }); + + it('every z.never() key across every builtin driver is covered by the unknown-driver fallback', () => { + // Guards the one hand-written canonical list: if a driver refuses a NEW + // credential key, the fallback used for contract-less drivers must learn + // it too, or an unknown driver's config would serve that spelling in + // cleartext. Same invariant the service-datasource suite pins through the + // re-export — held here at the source as well, so it cannot be lost to a + // consumer-side test reshuffle. + const declared = new Set(); + for (const id of BUILTIN_DRIVER_IDS as readonly string[]) { + for (const key of refusedCredentialKeys(id)) declared.add(key); + } + expect(declared.size).toBeGreaterThan(0); + const fallback = new Set(redactableConfigKeys('a-driver-with-no-contract')); + for (const key of declared) expect(fallback.has(key)).toBe(true); + }); +}); + +describe('redactDatasourceConfig — the read-path scrub, at its new home', () => { + it('drops stored credential keys and rewrites URL-embedded passwords, naming both', () => { + const { config, redactedKeys } = redactDatasourceConfig('postgres', { + host: 'db.internal', + database: 'app', + username: 'admin', + password: 'hunter2', + url: 'postgresql://admin:hunter2@db.internal:5432/app', + }); + expect(config).toEqual({ + host: 'db.internal', + database: 'app', + username: 'admin', + url: 'postgresql://admin@db.internal:5432/app', + }); + expect(redactedKeys).toEqual(['password', 'url']); + }); + + it('is pure — the stored record object is never mutated', () => { + const stored = { host: 'h', password: 'hunter2' }; + const { config } = redactDatasourceConfig('postgres', stored); + expect(stored).toEqual({ host: 'h', password: 'hunter2' }); + expect(config).not.toBe(stored); + }); + + it('a clean config answers redactedKeys: [] — "ran, nothing to hide", not an absence', () => { + const { config, redactedKeys } = redactDatasourceConfig('postgres', { + host: 'h', + database: 'd', + }); + expect(config).toEqual({ host: 'h', database: 'd' }); + expect(redactedKeys).toEqual([]); + }); + + it('a driver with no shipped contract still has its canonical credentials hidden', () => { + expect(getDriverConfigSchema('not-a-real-driver')).toBeUndefined(); + const { config, redactedKeys } = redactDatasourceConfig('not-a-real-driver', { + host: 'h', + password: 'hunter2', + authToken: 'jwt', + }); + expect(config).toEqual({ host: 'h' }); + expect(redactedKeys).toEqual(['authToken', 'password']); + }); +}); + +describe('write-door alignment: redactUrlPassword removes exactly what urlUserinfoPassword refuses', () => { + const CARRYING = [ + 'postgresql://admin:hunter2@db:5432/app', + 'mongodb://u:p@a.example.com:27017/db?replicaSet=rs0', + 'postgres://u:p@ss@host/db', // malformed literal `@` — judged whole on both sides + 'mysql://u:p@h1:3306,h2:3306/db', // multi-host DSN WHATWG parsing mangles + ]; + const CREDENTIAL_FREE = [ + 'postgresql://admin@db:5432/app', + 'postgresql://db:5432/app', + 'libsql://my-db.turso.io', + 'file:./data/objectstack.db', + ':memory:', + 'https://host/a:b@c', + 'a:b@c', + ]; + + it('the redacted form of a credential-carrying URL is exactly what the write door accepts', () => { + for (const url of CARRYING) { + expect(urlUserinfoPassword(url)).toBeDefined(); + const redacted = redactUrlPassword(url); + expect(redacted).not.toBe(url); + // The property #8126 depends on for the legacy-row round trip: the READ + // door's output must pass the WRITE door, or every untouched "Save" + // on a legacy row 400s. + expect(urlUserinfoPassword(redacted)).toBeUndefined(); + } + }); + + it('a URL the write door accepts comes back from the read door byte-for-byte', () => { + for (const url of CREDENTIAL_FREE) { + expect(urlUserinfoPassword(url)).toBeUndefined(); + expect(redactUrlPassword(url)).toBe(url); + } + }); +}); diff --git a/packages/spec/src/data/datasource-credential-redaction.ts b/packages/spec/src/data/datasource-credential-redaction.ts new file mode 100644 index 0000000000..1ff94874cc --- /dev/null +++ b/packages/spec/src/data/datasource-credential-redaction.ts @@ -0,0 +1,240 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Read-path credential redaction for a datasource's driver `config` — the ONE + * definition of "what is a credential key" (#8300, moved here from + * `@objectstack/service-datasource`'s `datasource-config-redaction.ts`, which + * PR #8126 built for #8081, the services half of #7990). + * + * ## Why this lives in `@objectstack/spec/data` + * + * Two packages must agree on the credential-key set: `service-datasource` + * (the datasource-admin read path, PR #8126) and the metadata read path + * (#8154's per-type redaction hook, registered through + * `kernel/metadata-type-redaction.ts`). They share **no** dependency except + * `@objectstack/spec`, so spec is the only place one definition can live — + * a second derived copy would duplicate a *security* list across two doors + * that must agree (#8300 rejected that on measured drift grounds). The + * derivation is contract-reading, not business logic: it reads the driver + * schemas that already live beside it. + * + * ## What counts as a credential here + * + * Three sources, in descending order of authority: + * + * 1. **Derived from the driver's own contract** — a config key whose schema is + * `z.never()` IS the shape #8078 gave a refused inline credential + * (`refusedInlineCredentialKey`), so reading the schema is reading the + * refusal list rather than re-typing it. A driver that refuses a new + * credential key tomorrow is covered here the day it lands, which a + * hand-maintained list in a consumer package would not be. + * 2. **Former alias spellings** ({@link FORMER_CREDENTIAL_ALIASES}) — `passwd` + * / `pwd` / `token` / `jwt` / `auth_token` / `authtoken` used to be + * `aliases` that the parse RENAMED onto the canonical key; #8078 moved them + * to `guidance`, which refuses them. Neither spelling appears in the schema + * shape, and a stored row never went through the parse that would have + * renamed it — the wizard persists through `metadata.register`, whose + * validation is a structural name/label check. So the only place these can + * still be found is exactly the place this module reads: a stored row. + * 3. **Credential-shaped keys that are still WRITABLE** ({@link + * STILL_WRITABLE_CREDENTIAL_KEYS}) — today just turso's `encryptionKey`, an + * AES-256 key the binder has no slot for (#8081 scope item 4, which owns + * the decision about giving it one). Redacting it on READ neither grants + * nor removes that slot: the key stays writable, stays stored, and stays + * injected at connect. It simply stops being served back in cleartext, + * which is the one question this module answers. + * + * For a driver the platform ships no contract for, source 1 is empty — the + * registry is saying "nothing to check against", not "nothing to protect". The + * canonical spellings are therefore redacted by NAME for unknown drivers too. + * That asymmetry with the write gate (which deliberately lets an unknown + * driver's config through untouched) is intentional: declining to REFUSE an + * unrecognised key is a boundary choice about authoring, while serving a key + * literally named `password` back in cleartext is a leak under any boundary. + * + * ## URL-embedded credentials + * + * A `postgresql://user:pass@host/db` in `config.url` carries the same secret as + * `config.password`. The WRITE door refuses a URL userinfo password via the + * shared value-level parse in `driver/common.zod.ts` (`urlUserinfoPassword`, + * #8082 maintainer-ruled Option A 2026-08-12); this module is the READ half: + * a scrub that dropped `config.password` and then served the identical + * credential one key over would be a scrub in name only. So the read path + * redacts the PASSWORD COMPONENT of a URL's userinfo and leaves everything + * else, including the username, byte-for-byte — and the redacted shape it + * serves (`user@host`) is exactly what the write door still accepts, which is + * what keeps an untouched "Save" on a legacy row working. + * + * ## What stays in `service-datasource` + * + * `restoreRedactedConfig` — the write-path inverse that stops a redacted + * round-trip deleting the stored credential — stays with the admin service + * whose edit form needs it, importing the key set from here. Redaction is a + * spec-derived fact; restoration is a service-side write policy. + */ + +import { getDriverConfigSchema } from './driver/config-registry.zod'; + +/** + * Canonical inline-credential spellings, used for a driver whose contract this + * platform does not ship. Kept in sync with the schemas by + * `datasource-credential-redaction.test.ts`, which asserts every `z.never()` + * key across every builtin driver appears here — so a new refused key cannot + * land without this fallback learning it. + */ +const CANONICAL_CREDENTIAL_KEYS = ['password', 'authToken'] as const; + +/** + * Pre-#8078 alias spellings of the keys above. A row written through the wizard + * (which does not parse) can hold these verbatim; a row written through an + * authoring door had them renamed onto the canonical key before storage. + */ +const FORMER_CREDENTIAL_ALIASES = [ + 'passwd', + 'pwd', + 'token', + 'jwt', + 'auth_token', + 'authtoken', +] as const; + +/** + * Credential-shaped config keys that remain WRITABLE by deliberate spec choice, + * and so are never found by the `z.never()` derivation. + * + * `encryptionKey` (turso) is an AES-256 key for the local database file. #8078 + * left it writable because the datasource secret binder injects exactly one + * secret slot and `external.credentialsRef` resolution cannot target a second + * one; giving it a slot is #8081 scope item 4 and is NOT decided here. + */ +const STILL_WRITABLE_CREDENTIAL_KEYS: Record = { + turso: ['encryptionKey'], +}; + +/** Unwrap `.optional()` / `.default()` / `.nullable()` down to the base type. */ +function baseTypeOf(schema: unknown): string | undefined { + let node: any = schema; + for (let depth = 0; node && depth < 10; depth += 1) { + const def = node.def ?? node._def; + const type: string | undefined = def?.type; + if (!type) return undefined; + if (type === 'optional' || type === 'default' || type === 'nullable' || type === 'readonly') { + node = def.innerType; + continue; + } + return type; + } + return undefined; +} + +/** + * The inline-credential keys a driver's own contract declares unwritable. + * + * Empty for a driver with no shipped contract — see the module note on why the + * canonical spellings are still redacted in that case. + */ +export function refusedCredentialKeys(driver: unknown): string[] { + let shape: Record | undefined; + try { + const schema: any = getDriverConfigSchema(driver); + const raw = schema?.shape; + shape = typeof raw === 'function' ? raw() : raw; + } catch { + return []; + } + if (!shape) return []; + return Object.entries(shape) + .filter(([, member]) => baseTypeOf(member) === 'never') + .map(([key]) => key); +} + +/** Every config key this module hides for `driver`, canonical + alias + writable-but-secret. */ +export function redactableConfigKeys(driver: unknown): string[] { + const derived = refusedCredentialKeys(driver); + const canonical = derived.length > 0 ? derived : [...CANONICAL_CREDENTIAL_KEYS]; + const stillWritable = typeof driver === 'string' ? (STILL_WRITABLE_CREDENTIAL_KEYS[driver] ?? []) : []; + return [...new Set([...canonical, ...FORMER_CREDENTIAL_ALIASES, ...stillWritable])]; +} + +/** + * `scheme://[user[:password]@]rest`. Anchored, and every class excludes `/?#` + * so a password-looking substring in a path or query cannot be mistaken for one + * — `https://host/a:b@c` has no userinfo and must come back untouched. + * + * The password group deliberately ALLOWS `@` and is greedy, which (with + * backtracking) makes the match end at the LAST `@` before the path — the + * userinfo boundary RFC 3986 actually defines. A lazier class stopping at the + * first `@` would split `postgres://u:p@ss@host/db` after `p`, leave `ss@host` + * in place, and publish a fragment of the password while looking redacted. + * Such a URL is malformed (a literal `@` in userinfo must be `%40`), which is + * precisely why it must not be the case that decides how much leaks. + * + * The write door's detector (`urlUserinfoPassword` in `driver/common.zod.ts`) + * draws the same boundaries; now that both live in this package, the pin + * keeping them aligned is `datasource-credential-redaction.test.ts`. + */ +const URL_USERINFO_RE = /^([a-z][a-z0-9+.\-]*:\/\/)([^/?#@:]*)(:[^/?#]*)@/i; + +/** + * Strip the password component from a URL's userinfo, preserving the scheme, + * the username, and everything from the host onward. + * + * Returns the input unchanged when there is nothing to strip, which is what + * makes "did this value change?" a usable test for whether a credential was + * present. + */ +export function redactUrlPassword(value: string): string { + return value.replace(URL_USERINFO_RE, (_m, scheme: string, user: string) => `${scheme}${user}@`); +} + +/** A driver `config` with its credential material removed, and what was removed. */ +export interface RedactedDatasourceConfig { + config: Record; + /** + * Config keys whose value was removed or rewritten, sorted. Serving this + * alongside the redacted config is the difference between a caller that knows + * a credential is being withheld and one that infers it from an absence. + */ + redactedKeys: string[]; +} + +/** + * Remove every stored credential from a driver `config` for serving on a read + * path. + * + * Pure: the input object is never mutated, so a caller holding the stored + * record (the connect path does) is unaffected. + */ +export function redactDatasourceConfig( + driver: unknown, + config: Record | undefined, +): RedactedDatasourceConfig { + if (!config || typeof config !== 'object') return { config: {}, redactedKeys: [] }; + + const hidden = new Set(redactableConfigKeys(driver)); + const out: Record = {}; + const redactedKeys: string[] = []; + + for (const [key, value] of Object.entries(config)) { + if (hidden.has(key)) { + // Dropped, not masked. A mask would round-trip back through the wizard as + // a literal new password, and post-#8078 the canonical spellings would + // then be REFUSED at the write door — turning an untouched "Save" into an + // error the author cannot act on. An absent key is the shape the form + // already understands from `hasSecret`. + if (value !== undefined) redactedKeys.push(key); + continue; + } + if (typeof value === 'string') { + const redacted = redactUrlPassword(value); + if (redacted !== value) { + out[key] = redacted; + redactedKeys.push(key); + continue; + } + } + out[key] = value; + } + + return { config: out, redactedKeys: redactedKeys.sort() }; +} diff --git a/packages/spec/src/data/driver/common.zod.ts b/packages/spec/src/data/driver/common.zod.ts index 3f763229bd..524212bc03 100644 --- a/packages/spec/src/data/driver/common.zod.ts +++ b/packages/spec/src/data/driver/common.zod.ts @@ -168,8 +168,10 @@ export const URL_EMBEDDED_CREDENTIAL_REFUSED = (key: string): string => * mangles (postgres/mongo multi-host `user:pass@h1:5432,h2:5432/db`, bare * `:memory:`, `file:` paths), and a detector that throws on the exact inputs it * must judge would fail open. The boundaries below are RFC 3986's, and match - * the read-path redactor (`service-datasource`'s `redactUrlPassword`) so the - * write door refuses precisely the material the read door redacts: + * the read-path redactor (`redactUrlPassword` in this package's + * `data/datasource-credential-redaction.ts`, moved from `service-datasource` + * by #8300) so the write door refuses precisely the material the read door + * redacts: * * - the authority is what follows `//` (scheme-relative included), up to the * first `/`, `?` or `#` — a `:` or `@` in a path or query is never userinfo; diff --git a/packages/spec/src/data/driver/driver-credential-refusal.test.ts b/packages/spec/src/data/driver/driver-credential-refusal.test.ts index 3cd30ad794..975d4815fe 100644 --- a/packages/spec/src/data/driver/driver-credential-refusal.test.ts +++ b/packages/spec/src/data/driver/driver-credential-refusal.test.ts @@ -312,9 +312,10 @@ describe('urlUserinfoPassword — the shared value-level parse (#8082)', () => { }); it('userinfo ends at the LAST `@`, so a malformed literal-`@` password is caught whole', () => { - // Mirrors the read-path redactor's boundary (service-datasource - // `redactUrlPassword`): a URL malformed in exactly the way that hides part - // of a password must not be the case that goes unjudged. + // Mirrors the read-path redactor's boundary (`redactUrlPassword`, now in + // this package's `data/datasource-credential-redaction.ts` — #8300): a URL + // malformed in exactly the way that hides part of a password must not be + // the case that goes unjudged. expect(urlUserinfoPassword('postgres://u:p@ss@host/db')).toBe('p@ss'); }); diff --git a/packages/spec/src/data/index.ts b/packages/spec/src/data/index.ts index f91704d9e4..5bd1240998 100644 --- a/packages/spec/src/data/index.ts +++ b/packages/spec/src/data/index.ts @@ -156,6 +156,12 @@ export * from './datasource.zod'; // they were merely the shapes authors were TOLD to write against. export * from './driver/index'; +// The ONE definition of "what is a credential key" in a driver config, derived +// from the contracts above, plus the read-path redaction built on it (#8300 — +// moved here from service-datasource so the datasource-admin read path and the +// metadata read path's per-type redaction hook share a single security list). +export * from './datasource-credential-redaction'; + // External Datasource Federation — SQL↔field type compatibility (ADR-0015) export * from './type-compat'; export * from './external-catalog.zod'; diff --git a/packages/spec/src/kernel/index.ts b/packages/spec/src/kernel/index.ts index b65cc29165..b9d5c1802e 100644 --- a/packages/spec/src/kernel/index.ts +++ b/packages/spec/src/kernel/index.ts @@ -32,6 +32,9 @@ export * from './metadata-protection.zod'; // a persisted body or a strict re-parse (#4326, cloud#971). export * from './metadata-read-decorations'; export * from './metadata-type-schemas'; +// Per-type read-path redaction seam (#8300) — the registry #8154's metadata +// read exits consume, beside the schema/actions registries it mirrors. +export * from './metadata-type-redaction'; // Pre-parse unknown-key walker over EVERY metadata collection (#3786). Lives // here, not in data/, because covering every type means importing every schema. export * from './metadata-authoring-lint'; diff --git a/packages/spec/src/kernel/metadata-type-redaction.test.ts b/packages/spec/src/kernel/metadata-type-redaction.test.ts new file mode 100644 index 0000000000..20b8321a01 --- /dev/null +++ b/packages/spec/src/kernel/metadata-type-redaction.test.ts @@ -0,0 +1,141 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * #8300 — the per-type metadata read-path redaction seam. + * + * The blocks below hold, in order: the register/lookup round trip (the seam's + * contract, mirroring `registerMetadataTypeSchema`); the FAIL-CLOSED wiring of + * the built-in `datasource` redactor (#8300's central measurement: plugin-init + * registration is fail-open because the admin plugin is opt-in while + * `sys_metadata` rows and the `/meta` read exits exist without it — so the + * built-in must be present with ZERO registration calls); and the + * absence-vs-empty distinction #8154's consumer depends on (no redactor + * registered ⇒ `undefined`; redactor ran with nothing to hide ⇒ + * `redactedKeys: []` — collapsing the two would make "looks protected" and + * "is protected" indistinguishable again). + */ + +import { describe, expect, it } from 'vitest'; + +import { + getMetadataTypeRedactor, + listMetadataTypeRedactorTypes, + registerMetadataTypeRedactor, + type MetadataRedactionResult, + type MetadataTypeRedactor, +} from './metadata-type-redaction'; + +describe('register/lookup round trip (the registry pattern of registerMetadataTypeSchema)', () => { + it('a registered redactor is returned for its type', () => { + const redactor: MetadataTypeRedactor = (item) => ({ + item: { ...item, clientSecret: undefined }, + redactedKeys: ['clientSecret'], + }); + registerMetadataTypeRedactor('sso_provider_test', redactor); + expect(getMetadataTypeRedactor('sso_provider_test')).toBe(redactor); + expect(listMetadataTypeRedactorTypes()).toContain('sso_provider_test'); + }); + + it('re-registration replaces (idempotent registry, same as the schema seam)', () => { + const first: MetadataTypeRedactor = (item) => ({ item, redactedKeys: [] }); + const second: MetadataTypeRedactor = (item) => ({ item, redactedKeys: [] }); + registerMetadataTypeRedactor('replace_me_test', first); + registerMetadataTypeRedactor('replace_me_test', second); + expect(getMetadataTypeRedactor('replace_me_test')).toBe(second); + // One entry, not two. + expect(listMetadataTypeRedactorTypes().filter((t) => t === 'replace_me_test')).toHaveLength(1); + }); + + it('a registered redactor overrides a built-in, exactly as registered schemas do', () => { + const builtin = getMetadataTypeRedactor('datasource'); + expect(builtin).toBeDefined(); + const override: MetadataTypeRedactor = (item) => ({ item, redactedKeys: [] }); + registerMetadataTypeRedactor('datasource', override); + try { + expect(getMetadataTypeRedactor('datasource')).toBe(override); + } finally { + // Restore the built-in for the rest of the suite — the registry is + // module-level state shared across tests. + registerMetadataTypeRedactor('datasource', builtin!); + } + }); +}); + +describe('FAIL-CLOSED: the datasource redactor is a BUILT-IN, not a plugin registration', () => { + it('is resolvable with zero registration calls — no opt-in plugin in sight', () => { + // The #8300 measurement this pins: registering from + // `DatasourceAdminServicePlugin.init` is fail-open (the plugin is opt-in; + // the rows and read exits exist without it). If this lookup ever starts + // answering `undefined` on a fresh module load, cleartext would serve + // while looking protected — the worst available outcome. + const redactor = getMetadataTypeRedactor('datasource'); + expect(redactor).toBeDefined(); + expect(listMetadataTypeRedactorTypes()).toContain('datasource'); + }); + + it('redacts a legacy stored row through the ONE credential-key definition', () => { + const stored = { + name: 'legacy_pg', + driver: 'postgres', + config: { + host: 'db.internal', + database: 'app', + username: 'admin', + password: 'hunter2', + url: 'postgresql://admin:hunter2@db.internal:5432/app', + }, + _diagnostics: { valid: false, issues: [{ path: ['config', 'password'] }] }, + }; + const result = getMetadataTypeRedactor('datasource')!(stored) as MetadataRedactionResult; + + expect(result.item.config).toEqual({ + host: 'db.internal', + database: 'app', + username: 'admin', + url: 'postgresql://admin@db.internal:5432/app', + }); + expect(JSON.stringify(result.item)).not.toContain('hunter2'); + expect(result.redactedKeys).toEqual(['config.password', 'config.url']); + // `_diagnostics` is load-bearing (#8154: the valid:false badge is the + // migration inventory) — the redactor must pass it through untouched. + expect(result.item._diagnostics).toBe(stored._diagnostics); + // Pure: the STORED body keeps its credential; the connect path reads it. + expect(stored.config.password).toBe('hunter2'); + expect(stored.config.url).toBe('postgresql://admin:hunter2@db.internal:5432/app'); + }); + + it('turso: alias spellings and the still-writable encryptionKey are covered end-to-end', () => { + const result = getMetadataTypeRedactor('datasource')!({ + name: 't', + driver: 'turso', + config: { url: 'libsql://db.turso.io', authToken: 'jwt-token', encryptionKey: 'aes', passwd: 'x' }, + }); + expect(result.item.config).toEqual({ url: 'libsql://db.turso.io' }); + expect(result.redactedKeys).toEqual(['config.authToken', 'config.encryptionKey', 'config.passwd']); + }); + + it('an item with no config object is passed through as-is', () => { + const noConfig = { name: 'x', driver: 'postgres' }; + expect(getMetadataTypeRedactor('datasource')!(noConfig)).toEqual({ + item: noConfig, + redactedKeys: [], + }); + const arrayConfig = { name: 'x', driver: 'postgres', config: ['not', 'an', 'object'] }; + expect(getMetadataTypeRedactor('datasource')!(arrayConfig).item).toBe(arrayConfig); + }); +}); + +describe('absence is distinguishable from "nothing to redact" (#8154 consumer contract)', () => { + it('a type with no redactor answers undefined — a fact, not a failure', () => { + expect(getMetadataTypeRedactor('object')).toBeUndefined(); + expect(getMetadataTypeRedactor('view')).toBeUndefined(); + expect(getMetadataTypeRedactor('type-that-does-not-exist')).toBeUndefined(); + }); + + it('a redactor that finds nothing answers [] with the item served intact', () => { + const clean = { name: 'clean', driver: 'postgres', config: { host: 'h', database: 'd' } }; + const result = getMetadataTypeRedactor('datasource')!(clean); + expect(result.redactedKeys).toEqual([]); + expect(result.item).toEqual(clean); + }); +}); diff --git a/packages/spec/src/kernel/metadata-type-redaction.ts b/packages/spec/src/kernel/metadata-type-redaction.ts new file mode 100644 index 0000000000..a87baa1af3 --- /dev/null +++ b/packages/spec/src/kernel/metadata-type-redaction.ts @@ -0,0 +1,147 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Metadata Type → read-path redactor registry (#8300, the enabling half of + * #8154's security invariant: stored credentials must never serve cleartext). + * + * Same shape and pattern as the two sibling registries in + * `metadata-type-schemas.ts` (`registerMetadataTypeSchema` / + * `registerMetadataTypeActions`): a built-in map, a runtime-extensible overlay, + * one accessor, and a snapshot enumerator. It lives HERE — not in + * `metadata-protocol` — because the type owners that must register a redactor + * are service packages (`service-datasource` today, SSO next), and **no + * service or connector package depends on `@objectstack/metadata-protocol`** + * (measured in #8300; a registry there would be unreachable from every package + * that must call it). `@objectstack/spec/kernel` is the seam every type owner + * already imports. + * + * ## Consumer contract (#8154 — the metadata read path) + * + * Every metadata read exit that serves a stored body (`getMetaItems`, + * `getMetaItem`, `getMetaItemLayered` in both layers) should resolve the + * type's redactor with {@link getMetadataTypeRedactor} and apply it to each + * item before serving. Two ordering rules the consumer owns, measured on + * #8154: + * + * - `_diagnostics` MUST be computed on the raw stored body BEFORE redaction — + * computing after flips `valid:false` → `valid:true` and destroys the + * migration-inventory badge. + * - The stored record is never mutated: a redactor is pure, and the connect + * path (or any other raw-record consumer) keeps reading cleartext exactly + * as before. Redaction is a read-path SERVING act. + * + * The write-path carry-forward (a redacted-body PUT must not persist credential + * deletion) is also #8154's, deliberately not represented here. + * + * ## Why `datasource` is a BUILT-IN, not a plugin registration + * + * The obvious registration site — `DatasourceAdminServicePlugin.init`, where + * the sibling `registerMetadataTypeActions` call lives — is measured + * **fail-open** (#8300): that plugin is opt-in, while `sys_metadata` rows and + * the `/meta` read exits exist without it, so a host storing datasource rows + * without the plugin would serve cleartext *with the hook installed and + * looking healthy*. The redaction derivation lives in this same package + * (`data/datasource-credential-redaction.ts`), so the honest, non-opt-in + * wiring is a built-in entry: present the moment this module loads, on every + * composition that can serve a datasource row, with nothing to forget. + */ + +import { redactDatasourceConfig } from '../data/datasource-credential-redaction'; + +/** What a {@link MetadataTypeRedactor} returns: the servable item, and what was withheld. */ +export interface MetadataRedactionResult { + /** The item with credential material removed. A NEW object — never the input mutated. */ + item: Record; + /** + * Dotted item-relative paths whose value was removed or rewritten + * (e.g. `config.password`), sorted. `[]` means the redactor RAN and found + * nothing to hide — distinguishable from "no redactor registered", which is + * {@link getMetadataTypeRedactor} answering `undefined`. Serving this beside + * the item is the difference between a caller that knows a credential is + * being withheld and one that infers it from an absence. + */ + redactedKeys: string[]; +} + +/** + * A per-type read-path redactor: takes a stored metadata item body, returns + * the servable projection of it. MUST be pure (no mutation of the input, no + * I/O) — it runs on every read exit, against the raw stored body. + */ +export type MetadataTypeRedactor = (item: Record) => MetadataRedactionResult; + +/** + * The built-in `datasource` redactor: redacts the driver `config` through the + * one credential-key definition in `data/datasource-credential-redaction.ts` + * (derived from each driver's `z.never()` contract + the pre-#8078 alias list + * + turso's `encryptionKey`), and leaves every other key of the item — + * `_diagnostics` included — byte-for-byte. + */ +function redactDatasourceItem(item: Record): MetadataRedactionResult { + const config = item.config; + if (!config || typeof config !== 'object' || Array.isArray(config)) { + return { item, redactedKeys: [] }; + } + const { config: redacted, redactedKeys } = redactDatasourceConfig( + item.driver, + config as Record, + ); + if (redactedKeys.length === 0) return { item, redactedKeys: [] }; + return { + item: { ...item, config: redacted }, + redactedKeys: redactedKeys.map((key) => `config.${key}`), + }; +} + +/** + * Built-in mapping from metadata type identifier → its read-path redactor. + * A type omitted here serves its stored body unredacted (most types hold no + * secret). See the module note for why `datasource` is wired here rather than + * registered from the opt-in admin plugin. + */ +const BUILTIN_METADATA_TYPE_REDACTORS: Record = { + datasource: redactDatasourceItem, +}; + +/** Runtime-extensible overlay populated via `registerMetadataTypeRedactor`. */ +const EXTRA_METADATA_TYPE_REDACTORS = new Map(); + +/** + * Look up the read-path redactor for a metadata type. + * + * Returns the user-registered override if any, otherwise the built-in + * redactor. Returns `undefined` for a type with no redactor — which the + * consumer must treat as "serve as-is", never as an error: absence of a + * registration is a fact about the type, not a failure. + */ +export function getMetadataTypeRedactor(type: string): MetadataTypeRedactor | undefined { + return EXTRA_METADATA_TYPE_REDACTORS.get(type) ?? BUILTIN_METADATA_TYPE_REDACTORS[type]; +} + +/** + * Register (or replace) the read-path redactor for a metadata type. + * + * A plugin whose metadata type stores secret material (an SSO seat storing + * client secrets, a connector storing API keys) should call this from its + * **`init(ctx)`** — the same site the sibling {@link + * registerMetadataTypeSchema} documents — so every metadata read exit starts + * withholding that material. Idempotent; a later registration for the same + * type replaces the earlier one, built-ins included. + * + * ⚠️ Registering from an OPT-IN plugin protects only compositions that install + * the plugin. If the type's rows can exist in `sys_metadata` without the + * plugin (the #8300 datasource measurement), the redactor belongs in + * `BUILTIN_METADATA_TYPE_REDACTORS` in this package instead — a redaction + * that looks installed but isn't loaded is the worst available outcome, + * because it reads as protected. + */ +export function registerMetadataTypeRedactor(type: string, redactor: MetadataTypeRedactor): void { + EXTRA_METADATA_TYPE_REDACTORS.set(type, redactor); +} + +/** Snapshot of every type that currently has a redactor (built-in + extras), sorted. */ +export function listMetadataTypeRedactorTypes(): string[] { + const types = new Set(Object.keys(BUILTIN_METADATA_TYPE_REDACTORS)); + for (const t of EXTRA_METADATA_TYPE_REDACTORS.keys()) types.add(t); + return Array.from(types).sort(); +}