Skip to content

feat(spec): per-type metadata redaction seam in kernel, one credential-key definition in data (#8300)#8530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
qq9340100 merged 4 commits into from
Aug 13, 2026
Merged

feat(spec): per-type metadata redaction seam in kernel, one credential-key definition in data (#8300) #8530

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
77f07fc
feat(spec): per-type metadata redaction seam in kernel + one credenti…
claude Aug 13, 2026
aca3b3c
Merge remote-tracking branch 'origin/main' into claude/issue-8300-met…
claude Aug 13, 2026
af848fb
merge origin/main (os-regen artifacts taken from main; regeneration f…
claude Aug 13, 2026
6ac1f2d
chore(spec): regenerate api-surface + export-origins on the merged tr…
claude Aug 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .changeset/metadata-redaction-seam.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
---
"@objectstack/spec": minor
"@objectstack/service-datasource": patch
---

feat(spec): per-type metadata read-path redaction seam in `kernel`, and ONE definition of "what is a credential key" in `data` (#8300)

Two additions to `@objectstack/spec`, both enabling #8154's security invariant
(stored credentials must never serve cleartext on the metadata read path):

- **`kernel/metadata-type-redaction.ts`** — `registerMetadataTypeRedactor` /
`getMetadataTypeRedactor` / `listMetadataTypeRedactorTypes`, the same
built-in-map + runtime-overlay registry pattern as its siblings
`registerMetadataTypeSchema` and `registerMetadataTypeActions`. The
`datasource` redactor is wired as a **built-in** (present the moment the
module loads), because registering it from the opt-in datasource-admin
plugin is measured fail-open: `sys_metadata` rows and the `/meta` read exits
exist without that plugin.
- **`data/datasource-credential-redaction.ts`** — the credential-key
derivation and read-path redaction previously in
`@objectstack/service-datasource` (`refusedCredentialKeys`,
`redactableConfigKeys`, `redactUrlPassword`, `redactDatasourceConfig`,
`RedactedDatasourceConfig`), moved here so the datasource-admin read path
and the metadata read path share one security list. The key set is derived
from each driver's own `z.never()` contract plus the pre-#8078 alias list
and turso's still-writable `encryptionKey` — byte-equal to what the
service-datasource original derived, pinned by test.

`@objectstack/service-datasource` re-exports the moved names from
`@objectstack/spec/data` (existing imports keep compiling; behaviour
unchanged) and keeps `restoreRedactedConfig`, the admin service's write-path
inverse.

<!-- adr-0087: not-required (no-migration-prescription) additive new exports plus a same-name re-export move; no authorable key, stored shape, or consumer-visible behaviour changes, so there is nothing for an upgrader to migrate -->
255 changes: 30 additions & 225 deletions packages/services/service-datasource/src/datasource-config-redaction.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,242 +2,47 @@

/**
* Read-path credential redaction for a datasource's driver `config` (#8081,
* the services half of #7990).
* the services half of #7990) — and the write-path inverse that keeps the
* redaction from turning "Save" into credential deletion.
*
* #8078 closed the WRITE door: `config.password` / `config.authToken` are
* declared-unwritable (`z.never()`) on every driver that has them, so no new
* row can carry an inline credential. It could not close the READ door, and it
* did not try: rows written before it still carry cleartext, and
* `DatasourceAdminService.getDatasource()` handed `config` back verbatim while
* its own doc comment claimed the credential had been stripped. This module is
* the strip that comment described.
* ## Where the definition lives now (#8300)
*
* ## What counts as a credential here
* The derivation half — what counts as a credential key, and the read-path
* redaction built on it — MOVED to `@objectstack/spec/data`
* (`datasource-credential-redaction.ts`), so that this package and the
* metadata read path (#8154, via the `kernel/metadata-type-redaction.ts`
* seam) share ONE security list instead of two derived copies that must
* agree. The re-exports below keep every existing consumer of this module
* compiling unchanged; the moved module's header carries the full rationale
* (three key sources, the unknown-driver posture, URL-userinfo boundaries).
*
* Three sources, in descending order of authority:
*
* 1. **Derived from the driver's own contract** — a config key whose schema is
* `z.never()` IS the shape #8078 gave a refused inline credential
* (`refusedInlineCredentialKey`), so reading the schema is reading the
* refusal list rather than re-typing it. A driver that refuses a new
* credential key tomorrow is covered here the day it lands, which a
* hand-maintained list in this package would not be.
* 2. **Former alias spellings** ({@link FORMER_CREDENTIAL_ALIASES}) — `passwd`
* / `pwd` / `token` / `jwt` / `auth_token` / `authtoken` used to be
* `aliases` that the parse RENAMED onto the canonical key; #8078 moved them
* to `guidance`, which refuses them. Neither spelling appears in the schema
* shape, and a stored row never went through the parse that would have
* renamed it — the wizard persists through `metadata.register`, whose
* validation is a structural name/label check. So the only place these can
* still be found is exactly the place this module reads: a stored row.
* 3. **Credential-shaped keys that are still WRITABLE** ({@link
* STILL_WRITABLE_CREDENTIAL_KEYS}) — today just turso's `encryptionKey`, an
* AES-256 key the binder has no slot for (#8081 scope item 4, which owns
* the decision about giving it one). Redacting it on READ neither grants
* nor removes that slot: the key stays writable, stays stored, and stays
* injected at connect. It simply stops being served back in cleartext,
* which is the one question this module answers.
*
* For a driver the platform ships no contract for, source 1 is empty — the
* registry is saying "nothing to check against", not "nothing to protect". The
* canonical spellings are therefore redacted by NAME for unknown drivers too.
* That asymmetry with the write gate (which deliberately lets an unknown
* driver's config through untouched) is intentional: declining to REFUSE an
* unrecognised key is a boundary choice about authoring, while serving a key
* literally named `password` back in cleartext is a leak under any boundary.
*
* ## URL-embedded credentials
*
* A `postgresql://user:pass@host/db` in `config.url` carries the same secret as
* `config.password`. When this module landed, refusing it was explicitly
* UNRULED (#7990/#8078 pinned the acceptance as a fact); #8082 has since ruled
* it (maintainer 2026-08-12, Option A), and the WRITE door now refuses a URL
* userinfo password via the spec's shared value-level parse
* (`urlUserinfoPassword`, `@objectstack/spec` `data/driver/common.zod.ts`).
* This module is still the READ half: a scrub that dropped `config.password`
* and then served the identical credential one key over would be a scrub in
* name only — the same "claims a protection it does not perform" shape #8081
* exists to end. So the read path redacts the PASSWORD COMPONENT of a URL's
* userinfo and leaves everything else, including the username, byte-for-byte —
* and the redacted shape it serves (`user@host`) is exactly what the write
* door still accepts, which is what keeps an untouched "Save" on a legacy row
* working.
*
* ## Why redaction must be reversible
* ## What stays here: {@link restoreRedactedConfig}
*
* `getDatasource()` feeds the Studio edit form, and `updateDatasource()` takes
* that form's `config` back as a whole-object patch. A scrub with no inverse
* would therefore turn every "Save" on an unmodified form into silent credential
* DELETION — trading a disclosure bug for a data-loss bug. {@link
* restoreRedactedConfig} is that inverse, and it is the same rule the secret
* path next to it has always used ("preserve the existing `credentialsRef`
* unless a new secret rewraps it"), applied to the material this module hides.
*/

import { getDriverConfigSchema } from '@objectstack/spec/data';

/**
* Canonical inline-credential spellings, used for a driver whose contract this
* platform does not ship. Kept in sync with the schemas by
* `datasource-config-redaction.test.ts`, which asserts every `z.never()` key
* across every builtin driver appears here — so a new refused key cannot land
* without this fallback learning it.
*/
const CANONICAL_CREDENTIAL_KEYS = ['password', 'authToken'] as const;

/**
* Pre-#8078 alias spellings of the keys above. A row written through the wizard
* (which does not parse) can hold these verbatim; a row written through an
* authoring door had them renamed onto the canonical key before storage.
* would therefore turn every "Save" on an unmodified form into silent
* credential DELETION — trading a disclosure bug for a data-loss bug.
* {@link restoreRedactedConfig} is that inverse, and it is the same rule the
* secret path next to it has always used ("preserve the existing
* `credentialsRef` unless a new secret rewraps it"), applied to the material
* the redaction hides. It stays in this package because restoration is a
* write policy of the admin service's own edit round-trip, not a spec-derived
* fact — and the generic metadata write door's equivalent carry-forward is
* #8154's, deliberately not built here.
*/
const FORMER_CREDENTIAL_ALIASES = [
'passwd',
'pwd',
'token',
'jwt',
'auth_token',
'authtoken',
] as const;

/**
* Credential-shaped config keys that remain WRITABLE by deliberate spec choice,
* and so are never found by the `z.never()` derivation.
*
* `encryptionKey` (turso) is an AES-256 key for the local database file. #8078
* left it writable because the datasource secret binder injects exactly one
* secret slot and `external.credentialsRef` resolution cannot target a second
* one; giving it a slot is #8081 scope item 4 and is NOT decided here.
*/
const STILL_WRITABLE_CREDENTIAL_KEYS: Record<string, readonly string[]> = {
turso: ['encryptionKey'],
};

/** Unwrap `.optional()` / `.default()` / `.nullable()` down to the base type. */
function baseTypeOf(schema: unknown): string | undefined {
let node: any = schema;
for (let depth = 0; node && depth < 10; depth += 1) {
const def = node.def ?? node._def;
const type: string | undefined = def?.type;
if (!type) return undefined;
if (type === 'optional' || type === 'default' || type === 'nullable' || type === 'readonly') {
node = def.innerType;
continue;
}
return type;
}
return undefined;
}

/**
* The inline-credential keys a driver's own contract declares unwritable.
*
* Empty for a driver with no shipped contract — see the module note on why the
* canonical spellings are still redacted in that case.
*/
export function refusedCredentialKeys(driver: unknown): string[] {
let shape: Record<string, unknown> | undefined;
try {
const schema: any = getDriverConfigSchema(driver as never);
const raw = schema?.shape;
shape = typeof raw === 'function' ? raw() : raw;
} catch {
return [];
}
if (!shape) return [];
return Object.entries(shape)
.filter(([, member]) => baseTypeOf(member) === 'never')
.map(([key]) => key);
}
import { redactableConfigKeys, redactUrlPassword } from '@objectstack/spec/data';

/** Every config key this module hides for `driver`, canonical + alias + writable-but-secret. */
export function redactableConfigKeys(driver: unknown): string[] {
const derived = refusedCredentialKeys(driver);
const canonical = derived.length > 0 ? derived : [...CANONICAL_CREDENTIAL_KEYS];
const stillWritable = typeof driver === 'string' ? (STILL_WRITABLE_CREDENTIAL_KEYS[driver] ?? []) : [];
return [...new Set([...canonical, ...FORMER_CREDENTIAL_ALIASES, ...stillWritable])];
}

/**
* `scheme://[user[:password]@]rest`. Anchored, and every class excludes `/?#`
* so a password-looking substring in a path or query cannot be mistaken for one
* — `https://host/a:b@c` has no userinfo and must come back untouched.
*
* The password group deliberately ALLOWS `@` and is greedy, which (with
* backtracking) makes the match end at the LAST `@` before the path — the
* userinfo boundary RFC 3986 actually defines. A lazier class stopping at the
* first `@` would split `postgres://u:p@ss@host/db` after `p`, leave `ss@host`
* in place, and publish a fragment of the password while looking redacted.
* Such a URL is malformed (a literal `@` in userinfo must be `%40`), which is
* precisely why it must not be the case that decides how much leaks.
*/
const URL_USERINFO_RE = /^([a-z][a-z0-9+.\-]*:\/\/)([^/?#@:]*)(:[^/?#]*)@/i;

/**
* Strip the password component from a URL's userinfo, preserving the scheme,
* the username, and everything from the host onward.
*
* Returns the input unchanged when there is nothing to strip, which is what
* makes "did this value change?" a usable test for whether a credential was
* present.
*/
export function redactUrlPassword(value: string): string {
return value.replace(URL_USERINFO_RE, (_m, scheme: string, user: string) => `${scheme}${user}@`);
}

/** A driver `config` with its credential material removed, and what was removed. */
export interface RedactedDatasourceConfig {
config: Record<string, unknown>;
/**
* Config keys whose value was removed or rewritten, sorted. Serving this
* alongside the redacted config is the difference between a caller that knows
* a credential is being withheld and one that infers it from an absence.
*/
redactedKeys: string[];
}

/**
* Remove every stored credential from a driver `config` for serving on a read
* path.
*
* Pure: the input object is never mutated, so a caller holding the stored
* record (the connect path does) is unaffected.
*/
export function redactDatasourceConfig(
driver: unknown,
config: Record<string, unknown> | undefined,
): RedactedDatasourceConfig {
if (!config || typeof config !== 'object') return { config: {}, redactedKeys: [] };

const hidden = new Set(redactableConfigKeys(driver));
const out: Record<string, unknown> = {};
const redactedKeys: string[] = [];

for (const [key, value] of Object.entries(config)) {
if (hidden.has(key)) {
// Dropped, not masked. A mask would round-trip back through the wizard as
// a literal new password, and post-#8078 the canonical spellings would
// then be REFUSED at the write door — turning an untouched "Save" into an
// error the author cannot act on. An absent key is the shape the form
// already understands from `hasSecret`.
if (value !== undefined) redactedKeys.push(key);
continue;
}
if (typeof value === 'string') {
const redacted = redactUrlPassword(value);
if (redacted !== value) {
out[key] = redacted;
redactedKeys.push(key);
continue;
}
}
out[key] = value;
}

return { config: out, redactedKeys: redactedKeys.sort() };
}
export {
refusedCredentialKeys,
redactableConfigKeys,
redactUrlPassword,
redactDatasourceConfig,
type RedactedDatasourceConfig,
} from '@objectstack/spec/data';

/**
* Re-apply the credential material {@link redactDatasourceConfig} hid, for a
* Re-apply the credential material `redactDatasourceConfig` hid, for a
* patch that is round-tripping a previously-read config back to the store.
*
* The rule is deliberately narrow: stored material is carried forward ONLY
Expand Down
5 changes: 5 additions & 0 deletions packages/spec/api-surface/data.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -472,6 +472,7 @@
"RecordSurface (type)",
"RecordSurfaceOptions (interface)",
"RecordSurfaceViewport (type)",
"RedactedDatasourceConfig (interface)",
"ReferenceIdValue (type)",
"ReferenceIdValueSchema (const)",
"ReferenceResolution (type)",
Expand DownExpand Up@@ -702,10 +703,14 @@
"platformProvisionsStorage (function)",
"provisionPrimary (function)",
"readAutonumberCounter (function)",
"redactDatasourceConfig (function)",
"redactUrlPassword (function)",
"redactableConfigKeys (function)",
"reduceFilterKeyVerdict (function)",
"reduceFilterVerdict (function)",
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedInlineCredentialKey (function)",
"renderAutonumber (function)",
"resolveAutonumberFormat (function)",
Expand Down
5 changes: 5 additions & 0 deletions packages/spec/api-surface/kernel.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -226,7 +226,9 @@
"MetadataQueryResultSchema (const)",
"MetadataQuerySchema (const)",
"MetadataReadDecoration (type)",
"MetadataRedactionResult (interface)",
"MetadataType (type)",
"MetadataTypeRedactor (type)",
"MetadataTypeRegistryEntry (type)",
"MetadataTypeRegistryEntryParsed (type)",
"MetadataTypeRegistryEntrySchema (const)",
Expand DownExpand Up@@ -492,6 +494,7 @@
"featureGatePredicate (function)",
"getMetadataCreateSeed (function)",
"getMetadataTypeActions (function)",
"getMetadataTypeRedactor (function)",
"getMetadataTypeSchema (function)",
"isConsumerInstallable (function)",
"isKnownPlatformCapability (function)",
Expand All@@ -500,10 +503,12 @@
"lintUnknownStackKeys (function)",
"listLintableAuthoringCollections (function)",
"listMetadataCreateSeedTypes (function)",
"listMetadataTypeRedactorTypes (function)",
"listMetadataTypeSchemaTypes (function)",
"listUnregisteredKindSchemaTypes (function)",
"lowerRequiresFeature (function)",
"registerMetadataTypeActions (function)",
"registerMetadataTypeRedactor (function)",
"registerMetadataTypeSchema (function)",
"resolveLockState (function)",
"stripReadDecorations (function)",
Expand Down
5 changes: 5 additions & 0 deletions packages/spec/export-origins/data.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -472,6 +472,7 @@
"RecordSurface": "src/data/record-surface.ts#RecordSurface (type)",
"RecordSurfaceOptions": "src/data/record-surface.ts#RecordSurfaceOptions (interface)",
"RecordSurfaceViewport": "src/data/record-surface.ts#RecordSurfaceViewport (type)",
"RedactedDatasourceConfig": "src/data/datasource-credential-redaction.ts#RedactedDatasourceConfig (interface)",
"ReferenceIdValue": "src/data/field-value.zod.ts#ReferenceIdValue (type)",
"ReferenceIdValueSchema": "src/data/field-value.zod.ts#ReferenceIdValueSchema (const)",
"ReferenceResolution": "src/data/seed-loader.zod.ts#ReferenceResolution (type)",
Expand DownExpand Up@@ -702,10 +703,14 @@
"platformProvisionsStorage": "src/data/injected-system-column-provenance.ts#platformProvisionsStorage (function)",
"provisionPrimary": "src/data/display-name.ts#provisionPrimary (function)",
"readAutonumberCounter": "src/data/autonumber-format.ts#readAutonumberCounter (function)",
"redactDatasourceConfig": "src/data/datasource-credential-redaction.ts#redactDatasourceConfig (function)",
"redactUrlPassword": "src/data/datasource-credential-redaction.ts#redactUrlPassword (function)",
"redactableConfigKeys": "src/data/datasource-credential-redaction.ts#redactableConfigKeys (function)",
"reduceFilterKeyVerdict": "src/data/filter-verdict.ts#reduceFilterKeyVerdict (function)",
"reduceFilterVerdict": "src/data/filter-verdict.ts#reduceFilterVerdict (function)",
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
"resolveAutonumberFormat": "src/data/autonumber-format.ts#resolveAutonumberFormat (function)",
Expand Down
Loading
Loading