From 2f02ca6ae1f2d11f27d214f1ce9d7cf051971069 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 02:45:06 +0000 Subject: [PATCH] feat(spec): treat nested datasource-config credential positions identically to the top-level keys they mirror A credential under the very spelling the top level refuses and redacts - one object level down (options.auth.token, options.pool.password, tunnel.password on a contract-less driver) - was accepted at publish and served by every datasource read door in cleartext with redactedConfigKeys: []. The nested judgment was a hand-enumerated per-driver path table on the read side and absent on the write side, while the top level was derived from the driver contract. Both sides now consume one derivation: the canonical spellings and former aliases move to driver/common.zod.ts (CREDENTIAL_KEY_SPELLINGS, the bottom of the import graph); the read scrub applies the name judgment and the URL composite at every object depth for every driver; the write door's passthrough walk refuses the same spellings at any depth; refusedCredentialPaths walks nested object shapes for z.never leaves; arrays stay off the walk on both doors (row-shaped data is not config). passthroughSecretPaths remains only as the client-measured residue. restoreRedactedConfig is now derived from the redactor's own redactedPaths, so every current and future redaction source is mirrored on the untouched-Save round trip by construction. Semantic migration entry datasource-config-options-nested-credential-spelling-refused (major 18) carries the authored-artifact upgrade. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01PBjwYLS6BciTQW3c9xQiD2 --- .../nested-credential-redaction-class.md | 59 +++++ content/docs/references/data/driver-mongo.mdx | 2 +- .../datasource-config-redaction.test.ts | 75 ++++++ .../src/datasource-config-redaction.ts | 66 +++-- packages/spec/api-surface/data.json | 6 + packages/spec/export-origins/data.json | 6 + .../datasource-credential-redaction.test.ts | 136 ++++++++++ .../data/datasource-credential-redaction.ts | 243 +++++++++++++----- packages/spec/src/data/driver/common.zod.ts | 131 ++++++++++ .../driver/driver-credential-refusal.test.ts | 80 ++++++ packages/spec/src/data/driver/mongo.zod.ts | 9 +- .../kernel/metadata-type-redaction.test.ts | 27 ++ ...ions-nested-credential-spelling-refused.ts | 37 +++ packages/spec/src/migrations/registry.ts | 33 +++ 14 files changed, 812 insertions(+), 98 deletions(-) create mode 100644 .changeset/nested-credential-redaction-class.md create mode 100644 packages/spec/src/migrations/entries/semantic/18.datasource-config-options-nested-credential-spelling-refused.ts diff --git a/.changeset/nested-credential-redaction-class.md b/.changeset/nested-credential-redaction-class.md new file mode 100644 index 0000000000..b854668543 --- /dev/null +++ b/.changeset/nested-credential-redaction-class.md @@ -0,0 +1,59 @@ +--- +'@objectstack/spec': minor +'@objectstack/service-datasource': patch +--- + +feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405) + +**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A +credential under the very spelling the top level refuses and redacts — one +object level down (`options.auth.token`, `options.pool.password`, a +`tunnel.password` on a contract-less driver) — was accepted at publish and +served back by every datasource read door in cleartext with +`redactedConfigKeys: []`, because the read side's nested judgment was only the +hand-enumerated `passthroughSecretPaths` table and the write side had no nested +name judgment at all. + +Both sides are now derived from ONE source instead of hand-maintained: + +- The canonical credential spellings and former aliases moved to + `driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the + driver-schema import graph — so the write door's passthrough walk and the + read redactor consume the same list (#8300's no-second-copy posture applied + to the list itself). +- **Read door** (`redactDatasourceConfig`, behind both consumers — the + datasource-admin routes and the kernel per-type redaction hook): the + credential-name judgment and the URL composite (userinfo + query params) now + run at EVERY object depth, for every driver, contract-less included. Nested + removals are reported as dotted paths in `redactedKeys`, plus a new + `redactedPaths` field carrying exact segments. `passthroughSecretPaths` + remains only as the residue it always should have been: CLIENT-MEASURED + secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no + top-level key. +- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a + credential-spelled key is refused at any object depth of the mongodb + `options` passthrough, with a prescription that does not inherit the + `auth.password`-only "wins over" reassurance. The measured `auth.password` + refusal keeps its own message; nothing is double-reported. +- **Schema derivation walked at depth**: `refusedCredentialPaths` / + `refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the + top level, so a driver contract that refuses a key inside a nested object + shape is covered the day it lands (none exists today — pinned per driver). +- **Arrays are off the walk** on both doors — the same structural line + `valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's + `initialData` seeds) keeps its own fields without a per-driver exclusion + list. +- `restoreRedactedConfig` (service-datasource) is now DERIVED from the + redactor instead of mirroring it rule by rule: it grafts stored material + back wherever the patch is indistinguishable from what the read path served, + so an untouched "Save" on an affected legacy row keeps its stored material + for every current and future redaction source, and an author's edit always + wins. The metadata write door's generic `carryForwardRedactedValues` already + walks the dotted paths and needs no change. + +Semantic migration entry +`datasource-config-options-nested-credential-spelling-refused` (protocol major +18) carries the authored-artifact upgrade: remove the nested key, or bind the +real secret through `external.credentialsRef` / the connection form. + + diff --git a/content/docs/references/data/driver-mongo.mdx b/content/docs/references/data/driver-mongo.mdx index 1bb82a55cd..912b184510 100644 --- a/content/docs/references/data/driver-mongo.mdx +++ b/content/docs/references/data/driver-mongo.mdx @@ -48,7 +48,7 @@ MongoDB Connection Configuration | **username** | `string` | optional | Authentication user | | **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` | | **authSource** | `string` | optional | Authentication database | -| **options** | `Record` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. | +| **options** | `Record` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. | --- diff --git a/packages/services/service-datasource/src/__tests__/datasource-config-redaction.test.ts b/packages/services/service-datasource/src/__tests__/datasource-config-redaction.test.ts index 34f2bacfd2..cbd799ab11 100644 --- a/packages/services/service-datasource/src/__tests__/datasource-config-redaction.test.ts +++ b/packages/services/service-datasource/src/__tests__/datasource-config-redaction.test.ts @@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door', ).rejects.toThrow(/options\.auth\.password/); }); }); + +describe('nested credential positions OFF the passthrough table — the class control, at this door', () => { + /** + * The nested-position finding, at the admin door: a credential spelling one + * object level down from the key the top level hides — deliberately NOT a + * `passthroughSecretPaths` row — used to be served by `getDatasource()` in + * cleartext with `redactedConfigKeys: []`, and accepted by the write gate in + * silence (measured on the pre-fix build). Regression cases against table + * rows were already green and prove nothing about this class. + */ + const OFF_TABLE_MONGO: StoredDatasource = { + name: 'off_table_mongo', + driver: 'mongodb', + origin: 'runtime', + config: { + database: 'events', + options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } }, + }, + }; + + it('read path: the off-table nested credential does not reach the caller, and is named', async () => { + const { service } = makeService([OFF_TABLE_MONGO]); + const read = await service.getDatasource('off_table_mongo'); + expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE'); + expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } }); + expect(read!.redactedConfigKeys).toContain('options.auth.token'); + }); + + it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => { + const { service, records } = makeService([OFF_TABLE_MONGO]); + const read = await service.getDatasource('off_table_mongo'); + await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' }); + expect(records[0].label).toBe('Renamed'); + expect((records[0].config!.options as any).auth).toEqual({ + username: 'app', + token: 'eyJhbGci.OFFTABLE.y', + }); + }); + + it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => { + const { service } = makeService([OFF_TABLE_MONGO]); + await expect( + service.updateDatasource('off_table_mongo', { + config: { + database: 'events', + options: { auth: { username: 'app', token: 'typed-new-secret' } }, + }, + }), + ).rejects.toThrow(/options\.auth\.token/); + }); + + it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => { + const stored: StoredDatasource = { + name: 'vendor_ds', + driver: 'com.vendor.custom', + origin: 'runtime', + config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } }, + }; + const { service, records } = makeService([stored]); + const read = await service.getDatasource('vendor_ds'); + expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db'); + expect(read!.redactedConfigKeys).toContain('replication.url'); + // Untouched round-trip: the served (redacted) URL is indistinguishable + // from the stored one once redacted, so the stored value is carried back. + await service.updateDatasource('vendor_ds', { config: read!.config }); + expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db'); + // An author who rewrites the nested URL by hand still WINS. + const edited = { + ...read!.config, + replication: { url: 'postgresql://svc@other-replica/db' }, + }; + await service.updateDatasource('vendor_ds', { config: edited }); + expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db'); + }); +}); diff --git a/packages/services/service-datasource/src/datasource-config-redaction.ts b/packages/services/service-datasource/src/datasource-config-redaction.ts index 66e39265a7..368c2c80a3 100644 --- a/packages/services/service-datasource/src/datasource-config-redaction.ts +++ b/packages/services/service-datasource/src/datasource-config-redaction.ts @@ -31,11 +31,7 @@ * #8154's, deliberately not built here. */ -import { - passthroughSecretPaths, - redactableConfigKeys, - redactUrlCredentials, -} from '@objectstack/spec/data'; +import { redactDatasourceConfig } from '@objectstack/spec/data'; export { refusedCredentialKeys, @@ -56,6 +52,24 @@ export { * where the patch is indistinguishable from what the read path served — an * absent key, or a URL that matches the stored URL once redacted. Anything the * author actually changed wins, including clearing a URL's password by hand. + * A patch whose CONTAINER for a nested leaf is removed is the author's word + * too (they deleted the block), so nothing is grafted there. + * + * ## Derived from the redactor, not restated beside it + * + * This function used to mirror the read path rule by rule — one loop per + * redaction source, each a copy that could silently fall behind (the docblock + * threat on every one of them: "a redaction the restore side did not mirror + * turns an untouched Save into silent credential deletion"). The nested- + * position fix made the read path recursive, which would have added two more + * loops — so the mirroring is now structural instead: compute what the read + * path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`), + * and for every redacted path graft the stored value back exactly where the + * patch still matches the served projection. A future redaction source is + * mirrored here by construction, with nothing to forget. (Same inversion the + * metadata door's generic `carryForwardRedactedValues` performs; this one + * consumes the redactor's exact `redactedPaths` segments, so a stored key + * with a literal dot cannot be mis-split.) * * What this does NOT do is let a patch set a refused key: `assertValidConfig` * still runs on the merged record, so a caller that types `password` into the @@ -69,43 +83,25 @@ export function restoreRedactedConfig( if (!patch || typeof patch !== 'object') return patch; if (!stored || typeof stored !== 'object') return patch; - const hidden = new Set(redactableConfigKeys(driver)); + const served = redactDatasourceConfig(driver, stored); const out: Record = { ...patch }; - for (const key of hidden) { - // Only when the patch does not speak to the key at all. A patch that DOES - // carry it is the author's word, and (for a refused spelling) is about to - // be refused on its own merits rather than quietly overwritten here. - if (!(key in out) && stored[key] !== undefined) out[key] = stored[key]; - } - - for (const [key, storedValue] of Object.entries(stored)) { - if (hidden.has(key) || typeof storedValue !== 'string') continue; - // The SAME composite the read path applies (userinfo password + #8337 - // credential query parameters) — a redaction this compare did not mirror - // would make the untouched "Save" it exists for delete the credential. - const redactedStored = redactUrlCredentials(storedValue); - // Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore. - if (redactedStored === storedValue) continue; - if (out[key] === redactedStored) out[key] = storedValue; - } - - // The passthrough spellings (#9040) — the nested material the read path - // drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The - // same narrow rule as the top-level keys, translated per leaf: restore ONLY - // when the patch's container for the leaf exists but does not speak to the - // leaf at all — exactly what the read path served. A patch carrying the leaf - // is the author's word (a typed-in `auth.password` is then refused by the - // #9040 write gate on its own merits); a patch with the CONTAINER removed is - // the author's word too (they deleted the block), so nothing is grafted. - for (const path of passthroughSecretPaths(driver)) { + for (const path of served.redactedPaths) { const storedLeaf = valueAt(stored, path); if (storedLeaf === undefined) continue; const parentPath = path.slice(0, -1); const leafKey = path[path.length - 1] as string; - const patchParent = valueAt(out, parentPath); + const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath); if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue; - if (leafKey in (patchParent as Record)) continue; + // What the read path served at this position: `undefined` for a dropped + // key, the rewritten string for a URL redaction. The patch speaks for the + // author exactly where it DIFFERS from that projection. + const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath); + const servedLeaf = + servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent) + ? (servedParent as Record)[leafKey] + : undefined; + if ((patchParent as Record)[leafKey] !== servedLeaf) continue; graftAt(out, path, storedLeaf); } diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index 4e577207a4..ff491722a3 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -56,12 +56,14 @@ "BulkWriteHookPhase (type)", "BulkWritePayloadScope (type)", "CALENDAR_DATE_TYPES (const)", + "CANONICAL_CREDENTIAL_KEYS (const)", "CLOCK_TIME_TYPES (const)", "COMPUTED_VALUE_TYPES (const)", "CONTEXT_TOKENS (const)", "CONTEXT_TOKEN_DESCRIPTIONS (const)", "CONTEXT_TOKEN_SUGGESTIONS (const)", "CONTEXT_TOKEN_WRAPPED_RE (const)", + "CREDENTIAL_KEY_SPELLINGS (const)", "CREDENTIAL_URL_QUERY_PARAMS (const)", "CREDENTIAL_URL_QUERY_PARAM_NAMES (const)", "CalendarDateValue (type)", @@ -258,6 +260,7 @@ "FILTER_TEXT_CASES (const)", "FILTER_TEXT_ROWS (const)", "FILTER_TOKEN_WRAPPED_RE (const)", + "FORMER_CREDENTIAL_ALIASES (const)", "FeedFilterMode (type)", "FeedItemType (type)", "Field (type)", @@ -452,6 +455,7 @@ "PAGINATION_UNORDERED_CASES (const)", "PAGINATION_ZERO_LIMIT_CASES (const)", "PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)", + "PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)", "PaginationConformanceCase (interface)", "PaginationConformanceRow (interface)", "PerOperationRequiredPermissions (type)", @@ -750,6 +754,8 @@ "referenceTargetOf (function)", "referencedFields (function)", "refusedCredentialKeys (function)", + "refusedCredentialPaths (function)", + "refusedCredentialPathsOfSchema (function)", "refusedInlineCredentialKey (function)", "refusedPassthroughSecretPaths (function)", "renderAutonumber (function)", diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index 60e669e3e9..07b1b97cf9 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -56,12 +56,14 @@ "BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)", "BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)", "CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)", + "CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)", "CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)", "COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)", "CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)", "CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)", "CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)", "CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)", + "CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)", "CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)", "CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)", "CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)", @@ -258,6 +260,7 @@ "FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)", "FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)", "FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)", + "FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)", "FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)", "FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)", "Field": "src/data/field.zod.ts#Field (type)", @@ -452,6 +455,7 @@ "PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)", "PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)", "PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)", + "PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)", "PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)", "PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)", "PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)", @@ -750,6 +754,8 @@ "referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)", "referencedFields": "src/data/autonumber-format.ts#referencedFields (function)", "refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)", + "refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)", + "refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)", "refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)", "refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)", "renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)", diff --git a/packages/spec/src/data/datasource-credential-redaction.test.ts b/packages/spec/src/data/datasource-credential-redaction.test.ts index 694fcfb614..aa33309da7 100644 --- a/packages/spec/src/data/datasource-credential-redaction.test.ts +++ b/packages/spec/src/data/datasource-credential-redaction.test.ts @@ -26,8 +26,11 @@ import { describe, expect, it } from 'vitest'; +import { z } from 'zod'; + import { BUILTIN_DRIVER_IDS, + CREDENTIAL_KEY_SPELLINGS, CREDENTIAL_URL_QUERY_PARAM_NAMES, getDriverConfigSchema, urlCredentialQueryParams, @@ -42,6 +45,8 @@ import { redactUrlPassword, redactableConfigKeys, refusedCredentialKeys, + refusedCredentialPaths, + refusedCredentialPathsOfSchema, refusedPassthroughSecretPaths, } from './datasource-credential-redaction'; @@ -368,3 +373,134 @@ describe('passthrough secret redaction (#9040) — the nested spellings the key- } }); }); + +/** + * The nested-position class — the finding this suite exists to hold closed. + * + * Before the fix, the credential-name judgment ran at the TOP level only and + * the nested side was ONLY the hand-enumerated `passthroughSecretPaths` table: + * a credential under the very spelling the top level hides, one object level + * down, was served back cleartext with `redactedKeys: []` (measured on the + * pre-fix build: `options.auth.token`, `options.pool.password`, + * `tunnel.password` on a contract-less driver, and a nested URL userinfo + * password all leaked). Every case below is a position deliberately ABSENT + * from that table — the mandated non-empty control: regression cases against + * table paths were already green and prove nothing about this class. + */ +describe('nested credential positions OFF the passthrough table — the class control', () => { + it('mongodb: drops a nested credential SPELLING the table does not list (`options.auth.token`)', () => { + expect(passthroughSecretPaths('mongodb').map((p) => p.join('.'))) + .not.toContain('options.auth.token'); + const { config, redactedKeys } = redactDatasourceConfig('mongodb', { + database: 'app', + options: { auth: { username: 'svc', token: 'eyJhbGci.x.y' }, replicaSet: 'rs0' }, + }); + expect(config).toEqual({ + database: 'app', + options: { auth: { username: 'svc' }, replicaSet: 'rs0' }, + }); + expect(redactedKeys).toEqual(['options.auth.token']); + }); + + it('mongodb: drops a credential spelling nested one level deeper than the table knows', () => { + expect(passthroughSecretPaths('mongodb').map((p) => p.join('.'))) + .not.toContain('options.pool.password'); + const { config, redactedKeys } = redactDatasourceConfig('mongodb', { + database: 'app', + options: { pool: { password: 'hunter2', min: 1 } }, + }); + expect(config).toEqual({ database: 'app', options: { pool: { min: 1 } } }); + expect(redactedKeys).toEqual(['options.pool.password']); + }); + + it('a contract-less driver has nested credential spellings hidden too — no table row exists at all', () => { + expect(passthroughSecretPaths('com.vendor.custom')).toEqual([]); + const { config, redactedKeys } = redactDatasourceConfig('com.vendor.custom', { + endpoint: 'x', + tunnel: { password: 'hunter2', host: 'bastion' }, + }); + expect(config).toEqual({ endpoint: 'x', tunnel: { host: 'bastion' } }); + expect(redactedKeys).toEqual(['tunnel.password']); + }); + + it('a NESTED string value gets the same URL composite the top level gets', () => { + const { config, redactedKeys } = redactDatasourceConfig('com.vendor.custom', { + replication: { url: 'postgresql://svc:hunter2@replica/db', lag: 5 }, + }); + expect(config).toEqual({ + replication: { url: 'postgresql://svc@replica/db', lag: 5 }, + }); + expect(redactedKeys).toEqual(['replication.url']); + }); + + it('`redactedPaths` carries the same removals as exact segments, index-aligned', () => { + const { redactedKeys, redactedPaths } = redactDatasourceConfig('mongodb', { + password: 'top', + options: { auth: { token: 'nested' } }, + }); + expect(redactedKeys).toEqual(['options.auth.token', 'password']); + expect(redactedPaths).toEqual([['options', 'auth', 'token'], ['password']]); + }); + + it('is pure at depth — the nested containers of the stored input are never mutated', () => { + const stored = { + options: { auth: { username: 'svc', token: 'eyJ' }, pool: { password: 'x' } }, + }; + const snapshot = JSON.parse(JSON.stringify(stored)); + redactDatasourceConfig('mongodb', stored); + expect(stored).toEqual(snapshot); + }); + + it('arrays are OFF the walk — row-shaped data keeps its own `password` FIELD (the seed carve-out, structurally)', () => { + // memory's `initialData` holds rows the driver SERVES; a scrub that + // reached into them would corrupt data, which is not this module's + // question. The boundary is structural (arrays end the walk — the same + // line the write door's `valueAtPath` draws), so it needs no per-driver + // exclusion list. + const stored = { + initialData: { users: [{ name: 'u1', password: 'seed-row-value' }] }, + }; + const { config, redactedKeys } = redactDatasourceConfig('memory', stored); + expect(config).toEqual(stored); + expect(redactedKeys).toEqual([]); + }); +}); + +/** + * The DERIVED nested refusals — "reading the schema is reading the refusal + * list", extended below the top level. No builtin driver declares a nested + * `z.never()` today (pinned per driver below, measured not assumed), so the + * nested branch is proved against a constructed schema. + */ +describe('refusedCredentialPaths — the schema derivation, walked at depth', () => { + it('finds a nested z.never() leaf in an object shape, at its exact path', () => { + const schema = z.object({ + host: z.string(), + proxy: z.object({ + host: z.string(), + secretToken: z.never().optional(), + }).optional(), + }); + expect(refusedCredentialPathsOfSchema(schema)).toEqual([['proxy', 'secretToken']]); + }); + + it('depth-1 projection equals refusedCredentialKeys for every builtin driver — and no driver declares deeper refusals today', () => { + for (const id of BUILTIN_DRIVER_IDS as readonly string[]) { + const paths = refusedCredentialPaths(id); + expect(paths.map((p) => p.join('.')), id).toEqual(refusedCredentialKeys(id)); + expect(paths.every((p) => p.length === 1), id).toBe(true); + } + expect(refusedCredentialPaths('not-a-real-driver')).toEqual([]); + }); + + it('the ONE spelling list both doors consume covers canonical + aliases, and the read fallback carries all of it', () => { + // `CREDENTIAL_KEY_SPELLINGS` (driver/common.zod.ts) is what the write + // door's passthrough walk refuses; the read path's name set must hide at + // least those spellings at every depth, or the doors disagree about a + // nested position — the drift #8300 exists to prevent. + const fallback = new Set(redactableConfigKeys('a-driver-with-no-contract')); + for (const name of CREDENTIAL_KEY_SPELLINGS) { + expect(fallback.has(name), name).toBe(true); + } + }); +}); diff --git a/packages/spec/src/data/datasource-credential-redaction.ts b/packages/spec/src/data/datasource-credential-redaction.ts index 2101e61940..c9d1965c3d 100644 --- a/packages/spec/src/data/datasource-credential-redaction.ts +++ b/packages/spec/src/data/datasource-credential-redaction.ts @@ -85,34 +85,21 @@ */ import { + CANONICAL_CREDENTIAL_KEYS, CREDENTIAL_URL_QUERY_PARAM_NAMES, credentialQueryParamOf, + FORMER_CREDENTIAL_ALIASES, MONGO_OPTIONS_CREDENTIAL_PATHS, } from './driver/common.zod'; import { getDriverConfigSchema, resolveDriverId } from './driver/config-registry.zod'; -/** - * Canonical inline-credential spellings, used for a driver whose contract this - * platform does not ship. Kept in sync with the schemas by - * `datasource-credential-redaction.test.ts`, which asserts every `z.never()` - * key across every builtin driver appears here — so a new refused key cannot - * land without this fallback learning it. - */ -const CANONICAL_CREDENTIAL_KEYS = ['password', 'authToken'] as const; - -/** - * Pre-#8078 alias spellings of the keys above. A row written through the wizard - * (which does not parse) can hold these verbatim; a row written through an - * authoring door had them renamed onto the canonical key before storage. - */ -const FORMER_CREDENTIAL_ALIASES = [ - 'passwd', - 'pwd', - 'token', - 'jwt', - 'auth_token', - 'authtoken', -] as const; +// The canonical spellings and former aliases MOVED to `driver/common.zod.ts` +// with the nested-position fix: the write door's passthrough walk +// (`credentialFreeMongoOptions`) must judge the same names this read path +// judges, and this module sits above `driver/` in the import graph — a second +// copy down there would be exactly the duplicated security list #8300 rejects. +// The sync pin (every builtin `z.never()` key appears in the canonical list) +// lives unchanged in `datasource-credential-redaction.test.ts`. /** * Credential-shaped config keys that remain WRITABLE by deliberate spec choice, @@ -129,8 +116,18 @@ const STILL_WRITABLE_CREDENTIAL_KEYS: Record = { /** * Secret-bearing paths inside a driver's passthrough `config` slot — the - * FOURTH spelling of the stored credential (#9040), nested where the top-level - * key-name scrub cannot see it. + * FOURTH spelling of the stored credential (#9040). + * + * Since the nested-position finding this table is a RESIDUE, not the nested + * judgment: the credential-name scrub runs at every object depth (see + * {@link redactDatasourceConfig}), so what belongs here is only the + * CLIENT-MEASURED secret spellings that mirror no top-level key + * (`proxyPassword`, `key`, `passphrase`, …) — names no schema ships and no + * shared spelling list can derive, established by measuring the client the + * driver spreads the slot into, never inferred from documentation. A nested + * position missing from this table is no longer served cleartext by default; + * it is cleartext only if it ALSO mirrors no credential spelling, which is + * exactly the class a measurement must decide. * * Only mongo declares a passthrough today (`options`, spread verbatim into * `MongoClientOptions`); postgres/mysql/turso/sqlite/memory have closed @@ -202,8 +199,8 @@ export function refusedPassthroughSecretPaths(driver: unknown): readonly (readon : []; } -/** Unwrap `.optional()` / `.default()` / `.nullable()` down to the base type. */ -function baseTypeOf(schema: unknown): string | undefined { +/** Unwrap `.optional()` / `.default()` / `.nullable()` down to the base node. */ +function baseNodeOf(schema: unknown): any { let node: any = schema; for (let depth = 0; node && depth < 10; depth += 1) { const def = node.def ?? node._def; @@ -213,11 +210,77 @@ function baseTypeOf(schema: unknown): string | undefined { node = def.innerType; continue; } - return type; + return node; } return undefined; } +/** Unwrap `.optional()` / `.default()` / `.nullable()` down to the base type. */ +function baseTypeOf(schema: unknown): string | undefined { + const node = baseNodeOf(schema); + const def = node?.def ?? node?._def; + return def?.type; +} + +/** The `shape` record of an object-typed schema node, or `undefined`. */ +function shapeOf(schema: unknown): Record | undefined { + const raw = (schema as any)?.shape; + const shape = typeof raw === 'function' ? raw() : raw; + return shape && typeof shape === 'object' ? (shape as Record) : undefined; +} + +/** + * Every path (any depth) in `schema`'s object shape whose leaf is `z.never()` + * — the nested extension of the {@link refusedCredentialKeys} derivation, so a + * driver contract that refuses a credential key inside a nested object shape + * is covered by the read scrub the day it lands, exactly like a top-level one. + * + * The walk descends OBJECT members only: a record slot + * (`z.record(z.string(), z.unknown())`) is the schema saying "untyped from + * here down", and the untyped region is judged by NAME instead (see the + * recursion in {@link redactDatasourceConfig}). Depth-capped defensively — + * driver config shapes are shallow and non-recursive. + * + * Exported for the derivation pin: no builtin driver declares a nested + * `z.never()` today, so the nested branch is provable only against a + * constructed schema. + */ +export function refusedCredentialPathsOfSchema(schema: unknown): (readonly string[])[] { + const out: string[][] = []; + const walk = (shape: Record, prefix: readonly string[], depth: number): void => { + if (depth > 8) return; + for (const [key, member] of Object.entries(shape)) { + const node = baseNodeOf(member); + const def = node?.def ?? node?._def; + const type: string | undefined = def?.type; + if (type === 'never') { + out.push([...prefix, key]); + continue; + } + if (type === 'object') { + const inner = shapeOf(node) ?? shapeOf(def); + if (inner) walk(inner, [...prefix, key], depth + 1); + } + } + }; + const shape = shapeOf(schema); + if (shape) walk(shape, [], 0); + return out; +} + +/** + * The refused credential POSITIONS a driver's own contract declares, at any + * depth — {@link refusedCredentialKeys} is the depth-1 projection of this. + * Empty for a driver with no shipped contract, like the key derivation. + */ +export function refusedCredentialPaths(driver: unknown): (readonly string[])[] { + try { + return refusedCredentialPathsOfSchema(getDriverConfigSchema(driver)); + } catch { + return []; + } +} + /** * The inline-credential keys a driver's own contract declares unwritable. * @@ -328,16 +391,58 @@ export function redactUrlCredentials(value: string): string { export interface RedactedDatasourceConfig { config: Record; /** - * Config keys whose value was removed or rewritten, sorted. Serving this - * alongside the redacted config is the difference between a caller that knows - * a credential is being withheld and one that infers it from an absence. + * Config keys whose value was removed or rewritten, as dotted paths + * (`password`, `options.auth.password`), sorted. Serving this alongside the + * redacted config is the difference between a caller that knows a credential + * is being withheld and one that infers it from an absence. */ redactedKeys: string[]; + /** + * The same removals as exact SEGMENT arrays, index-aligned with + * {@link redactedKeys}. The dotted spelling is ambiguous the moment a stored + * key carries a literal dot; the write-path inverses restore by walking + * containers, so they consume this form and cannot mis-split. + */ + redactedPaths: (readonly string[])[]; } /** * Remove every stored credential from a driver `config` for serving on a read - * path. + * path — at EVERY object depth, not only the top level. + * + * Four removal sources, applied in order: + * + * 1. The credential-name judgment ({@link redactableConfigKeys}) at every + * object depth. It was top-level-only until the nested-position finding: + * a credential under the very spelling the top level hides — one object + * level down (`options.auth.token`, `tunnel.password`) — was served back + * cleartext with `redactedKeys: []`, because the nested side was only the + * hand-enumerated path table of source 4. The name set is ONE list for + * every depth, and the same list the write door's passthrough walk + * refuses, so a nested position is treated identically to the top-level + * key it mirrors. + * 2. The URL composite ({@link redactUrlCredentials}) on every STRING value, + * again at every depth — a `postgresql://u:pass@host/db` nested inside a + * record slot carries the same secret it carries at the top level. + * 3. The DERIVED nested refusals ({@link refusedCredentialPaths}) — object + * shapes inside a driver contract whose leaf is `z.never()`. None exist + * today; the walk is what keeps "reading the schema is reading the + * refusal list" true at depth the day one lands. + * 4. The passthrough spellings (#9040, {@link passthroughSecretPaths}): the + * CLIENT-MEASURED secret names (`proxyPassword`, `key`, `passphrase`, …) + * that mirror no top-level key, so neither the schema nor the name set can + * derive them. The table is the residue for exactly that class — an entry + * whose leaf IS credential-spelled (`options.auth.password`) is already + * covered by source 1 and stays listed only because the table doubles as + * the write-path inverse's contract. + * + * Boundary shared by every walker on this surface (the write door's + * `valueAtPath`, `withoutPath` here): **arrays are off the walk.** An array in + * driver config is row-shaped data (memory's `initialData` seeds), and + * redacting a seeded row's own `password` FIELD would corrupt data the driver + * serves — which is not this module's question. The boundary is structural, + * so the seed-data carve-out needs no per-driver list, and both doors draw it + * in the same place. * * Pure: the input object is never mutated, so a caller holding the stored * record (the connect path does) is unaffected. @@ -346,48 +451,66 @@ export function redactDatasourceConfig( driver: unknown, config: Record | undefined, ): RedactedDatasourceConfig { - if (!config || typeof config !== 'object') return { config: {}, redactedKeys: [] }; + if (!config || typeof config !== 'object') return { config: {}, redactedKeys: [], redactedPaths: [] }; const hidden = new Set(redactableConfigKeys(driver)); - const out: Record = {}; - const redactedKeys: string[] = []; + const removed: (readonly string[])[] = []; - for (const [key, value] of Object.entries(config)) { - if (hidden.has(key)) { - // Dropped, not masked. A mask would round-trip back through the wizard as - // a literal new password, and post-#8078 the canonical spellings would - // then be REFUSED at the write door — turning an untouched "Save" into an - // error the author cannot act on. An absent key is the shape the form - // already understands from `hasSecret`. - if (value !== undefined) redactedKeys.push(key); - continue; - } - if (typeof value === 'string') { - const redacted = redactUrlCredentials(value); - if (redacted !== value) { - out[key] = redacted; - redactedKeys.push(key); + const scrub = (node: Record, prefix: readonly string[]): Record => { + const out: Record = {}; + for (const [key, value] of Object.entries(node)) { + const path = [...prefix, key]; + if (hidden.has(key)) { + // Dropped, not masked. A mask would round-trip back through the wizard + // as a literal new password, and post-#8078 the canonical spellings + // would then be REFUSED at the write door — turning an untouched + // "Save" into an error the author cannot act on. An absent key is the + // shape the form already understands from `hasSecret`. + if (value !== undefined) removed.push(path); continue; } + if (typeof value === 'string') { + const redacted = redactUrlCredentials(value); + if (redacted !== value) { + out[key] = redacted; + removed.push(path); + continue; + } + out[key] = value; + continue; + } + if (value && typeof value === 'object' && !Array.isArray(value)) { + out[key] = scrub(value as Record, path); + continue; + } + out[key] = value; } - out[key] = value; - } + return out; + }; + + let scrubbed = scrub(config, []); - // The passthrough spellings (#9040): nested secret material the top-level - // key-name scrub cannot see. Dropped, not masked, for the same round-trip - // reason as the inline keys; each removal is reported as its DOTTED path - // (`options.auth.password`), which is the shape the metadata write door's - // generic carry-forward (`carryForwardRedactedValues`) already walks. - let scrubbed: Record = out; - for (const path of passthroughSecretPaths(driver)) { + // Sources 3 and 4: positional removals. Reported as their exact paths; a + // path source 1 already emptied is a no-op here (`dropped: false`), so + // nothing is reported twice. + const positional = [ + ...refusedCredentialPaths(driver).filter((path) => path.length > 1), + ...passthroughSecretPaths(driver), + ]; + for (const path of positional) { const [next, dropped] = withoutPath(scrubbed, path); if (dropped) { scrubbed = next; - redactedKeys.push(path.join('.')); + removed.push(path); } } - return { config: scrubbed, redactedKeys: redactedKeys.sort() }; + removed.sort((a, b) => (a.join('.') < b.join('.') ? -1 : a.join('.') > b.join('.') ? 1 : 0)); + return { + config: scrubbed, + redactedKeys: removed.map((path) => path.join('.')), + redactedPaths: removed, + }; } /** diff --git a/packages/spec/src/data/driver/common.zod.ts b/packages/spec/src/data/driver/common.zod.ts index b1618f1339..6c60193e79 100644 --- a/packages/spec/src/data/driver/common.zod.ts +++ b/packages/spec/src/data/driver/common.zod.ts @@ -411,6 +411,62 @@ export const MONGO_OPTIONS_CREDENTIAL_PATHS: readonly (readonly string[])[] = [ ['auth', 'password'], ]; +/** + * Canonical inline-credential spellings — the key names the shipped driver + * contracts declare unwritable (`z.never()`), spelled out for the two places + * that must judge a key by NAME because no schema types the position: a driver + * the platform ships no contract for, and the interior of a passthrough + * `config` slot (`z.record(z.string(), z.unknown())` — the schema itself says + * "untyped from here down"). + * + * Lived in `data/datasource-credential-redaction.ts` (read side only) until + * the nested-position finding: a credential-spelled key one level DOWN from the + * top-level key it mirrors was accepted at publish and served back cleartext, + * because the read side's list could not be reached from this module (import + * direction) and the write side had no list at all. This module is the bottom + * of the driver-schema import graph, so the ONE spelling list lives here and + * both doors read it — the #8300 posture (never two copies of a security + * list), applied to the list itself. + * + * Kept in sync with the schemas by `datasource-credential-redaction.test.ts`, + * which asserts every `z.never()` key across every builtin driver appears + * here — so a driver refusing a new credential key cannot land without this + * list learning it. + */ +export const CANONICAL_CREDENTIAL_KEYS = ['password', 'authToken'] as const; + +/** + * Pre-#8078 alias spellings of the keys above. They used to be `aliases` that + * the parse RENAMED onto the canonical key; #8078 moved them to `guidance`, + * which refuses them at the top level. A stored row written through a door + * that does not parse (the wizard persists through `metadata.register`, whose + * validation is a structural name/label check) can hold them verbatim — at the + * top level or nested — which is why the read path judges these names at + * every depth. + */ +export const FORMER_CREDENTIAL_ALIASES = [ + 'passwd', + 'pwd', + 'token', + 'jwt', + 'auth_token', + 'authtoken', +] as const; + +/** + * Every credential KEY SPELLING the platform judges by name — canonical plus + * former aliases. The one list behind the nested halves of both doors: the + * write door's passthrough walk ({@link credentialFreeMongoOptions}) refuses a + * non-empty string under any of these names at any depth, and the read + * redactor (`data/datasource-credential-redaction.ts`) hides the same names at + * the same depths, so publish-time refusal and read-time redaction treat a + * nested position identically to the top-level key it mirrors. + */ +export const CREDENTIAL_KEY_SPELLINGS: readonly string[] = [ + ...CANONICAL_CREDENTIAL_KEYS, + ...FORMER_CREDENTIAL_ALIASES, +]; + /** The value at `path` inside a record-ish value, or `undefined` off the walk. */ function valueAtPath(value: unknown, path: readonly string[]): unknown { let node: unknown = value; @@ -421,6 +477,34 @@ function valueAtPath(value: unknown, path: readonly string[]): unknown { return node; } +/** + * Refusal prescription for a credential-SPELLED key at an arbitrary nested + * position inside the driver-options passthrough — the nested mirror of the + * top-level inline refusal, for positions the measured list above does not + * name. + * + * Wording constraints, inherited and one added: + * + * - No `${…}` placeholder advice (measured broken escape, #8078/#8336) — + * same as every sibling message. + * - It must NOT promise the bound secret "wins over" this value: that claim + * is measured for `auth.password` only (#8696). For any other nested + * position nothing is measured to read the value at all — which is the + * point the message makes instead: the value performs no function the + * author can observe, while sitting cleartext at rest. + */ +export const PASSTHROUGH_NESTED_CREDENTIAL_REFUSED = (path: string): string => + `\`${path}\` is spelled like a credential and is not accepted in the driver-options ` + + 'passthrough: the datasource is persisted whole into `sys_metadata`, which is served back ' + + 'by the ordinary data API, so a nested credential lands in cleartext at rest exactly like ' + + 'an inline `password` one level up — and unlike `auth.password`, no measured client ' + + 'behaviour reads this position, so the value buys nothing for what it leaks. Remove it; if ' + + "a real secret needs to reach the connection, bind it instead: the Setup → Datasources " + + "connection form's secret field hands it to the datasource secret binder, which encrypts " + + 'it into `sys_secret` and stores only an opaque handle at `external.credentialsRef`. ' + + 'Placeholders are no escape either: a `${…}` span anywhere in `options` is itself refused ' + + 'at publish.'; + /** * Attach the #9040 passthrough-credential refusal to mongo's `options` slot. * @@ -429,12 +513,33 @@ function valueAtPath(value: unknown, path: readonly string[]): unknown { * `superRefine`s judging the same value independently, an input violating both * reports both, and neither changes the other's semantics. Each finding is * reported at its own path so the author is pointed at the exact entry. + * + * Two refusal sources, disjoint by construction: + * + * 1. The MEASURED paths ({@link MONGO_OPTIONS_CREDENTIAL_PATHS}) — positions + * the client resolves into a login credential the binder substitutes + * (#9040's original walk, message unchanged). + * 2. The nested NAME judgment — a non-empty string under a key spelled like + * a credential ({@link CREDENTIAL_KEY_SPELLINGS}), at ANY object depth of + * the passthrough. Before this walk, `options.auth.password` was refused + * while the byte-identical secret at `options.auth.token` or + * `options.pool.password` was accepted in silence and sat cleartext in + * `sys_metadata` — the nested twin of the top-level key refusal, judged + * from the same one spelling list the read-path redactor consumes. + * + * Boundaries the walk shares with every other walker on this surface: arrays + * are off the walk ({@link valueAtPath} posture), only a NON-EMPTY STRING is + * refused (`auth.password`'s own boundary — an empty string is the passthrough + * twin of `user:@host`, and a non-string is not a secret a client accepts), + * and a path already refused by source 1 is not reported twice. */ export function credentialFreeMongoOptions(schema: S, key: string) { return schema.superRefine((value, ctx) => { + const reported = new Set(); for (const path of MONGO_OPTIONS_CREDENTIAL_PATHS) { const leaf = valueAtPath(value, path); if (typeof leaf === 'string' && leaf.length > 0) { + reported.add(path.join('.')); ctx.addIssue({ code: 'custom', path: [...path], @@ -442,6 +547,32 @@ export function credentialFreeMongoOptions(schema: S, key: }); } } + const walk = (node: unknown, prefix: readonly string[]): void => { + if (!node || typeof node !== 'object' || Array.isArray(node)) return; + for (const [name, child] of Object.entries(node as Record)) { + const path = [...prefix, name]; + if ( + CREDENTIAL_KEY_SPELLINGS.includes(name) + && typeof child === 'string' + && child.length > 0 + && !reported.has(path.join('.')) + ) { + ctx.addIssue({ + code: 'custom', + path, + message: PASSTHROUGH_NESTED_CREDENTIAL_REFUSED([key, ...path].join('.')), + }); + continue; + } + walk(child, path); + } + }; + // The judgment applies from the record's own top level down: every key + // under `options` is already one level below the config keys the driver + // schema types, so `options.password` mirrors the refused top-level + // `password` exactly as `options.auth.password` does (no measured client + // option carries any of these spellings at any depth). + walk(value, []); }); } diff --git a/packages/spec/src/data/driver/driver-credential-refusal.test.ts b/packages/spec/src/data/driver/driver-credential-refusal.test.ts index 6490173650..fa3a3ee098 100644 --- a/packages/spec/src/data/driver/driver-credential-refusal.test.ts +++ b/packages/spec/src/data/driver/driver-credential-refusal.test.ts @@ -704,6 +704,86 @@ describe('mongo options passthrough — credential refusal (#9040)', () => { }); }); +/** + * The nested NAME mirror of the refusal above — the class the nested-position + * finding closed. `options.auth.password` was refused while the byte-identical + * secret one spelling over (`options.auth.token`) or one level over + * (`options.pool.password`) was accepted in silence and sat cleartext in + * `sys_metadata`. The walk now judges every credential SPELLING + * (`CREDENTIAL_KEY_SPELLINGS` — the same one list the read redactor hides) at + * every object depth of the passthrough, so publish-time refusal treats a + * nested position identically to the top-level key it mirrors. + * + * Envelope note (same as the sibling pins): the zod issue's `code` and its + * pathed location are the whole envelope at this layer. + */ +describe('mongo options passthrough — nested credential-SPELLED keys refused at any depth', () => { + const VALID = { database: 'events', host: 'mongo.internal', username: 'svc' } as const; + const refusalAt = (options: Record, dotted: string) => { + const result = MongoConfigSchema.safeParse({ ...VALID, options }); + if (result.success) return undefined; + return result.error.issues.find((i) => i.path.join('.') === dotted); + }; + + it('refuses `auth.token` — the off-table position the read door also served cleartext', () => { + const issue = refusalAt({ auth: { username: 'app', token: 'eyJhbGci.x.y' } }, 'options.auth.token'); + expect(issue, 'refusal must be pathed at `options.auth.token`').toBeDefined(); + expect(issue!.code).toBe('custom'); + expect(issue!.message).toContain('`options.auth.token`'); + expect(issue!.message).toContain('cleartext at rest'); + expect(issue!.message).toContain('external.credentialsRef'); + // The "wins over" reassurance is measured for `auth.password` ONLY (#8696) + // — this message must not inherit it for a position nothing reads. + expect(issue!.message).not.toContain('wins over'); + }); + + it('refuses a credential spelling nested one level deeper than the measured table knows', () => { + const issue = refusalAt({ pool: { password: 'hunter2' } }, 'options.pool.password'); + expect(issue).toBeDefined(); + expect(issue!.message).toContain('`options.pool.password`'); + }); + + it('refuses a credential spelling at the record top level too — `options.password` mirrors `password`', () => { + const issue = refusalAt({ password: 'hunter2' }, 'options.password'); + expect(issue).toBeDefined(); + }); + + it('reports `auth.password` ONCE, with the measured message — the two walks do not double-report', () => { + const result = MongoConfigSchema.safeParse({ + ...VALID, + options: { auth: { username: 'app', password: 'hunter2' } }, + }); + expect(result.success).toBe(false); + const at = result.error!.issues.filter((i) => i.path.join('.') === 'options.auth.password'); + expect(at.length).toBe(1); + // The measured path keeps its own prescription — including the "wins over" + // reassurance that is TRUE for this position (#8696). + expect(at[0]!.message).toContain('wins over'); + }); + + it("shares `auth.password`'s value boundaries: empty and non-string are not refused", () => { + expect(refusalAt({ auth: { token: '' } }, 'options.auth.token')).toBeUndefined(); + expect(refusalAt({ auth: { token: 42 } }, 'options.auth.token')).toBeUndefined(); + // A name-hit OBJECT is a container, not a secret — its interior is walked. + const inner = refusalAt({ token: { password: 'x' } }, 'options.token.password'); + expect(inner).toBeDefined(); + }); + + it('does not refuse benign nested names, and arrays are off the walk (the shared boundary)', () => { + for (const options of [ + { auth: { username: 'app' }, replicaSet: 'rs0' }, + { authMechanismProperties: { SERVICE_NAME: 'mongodb' } }, + // An object inside an ARRAY is off every walker on this surface — the + // same structural line `valueAtPath`/`withoutPath` draw, which is what + // keeps row-shaped data out of a config judgment. + { hosts: [{ password: 'this-is-row-shaped-data-not-config' }] }, + ]) { + const result = MongoConfigSchema.safeParse({ ...VALID, options }); + expect(result.success, JSON.stringify(result.error?.issues)).toBe(true); + } + }); +}); + /** * The contradictory pair "`external.credentialsRef` bound + a mongo * `config.url` naming no user" is refused at the datasource level (#9041) — diff --git a/packages/spec/src/data/driver/mongo.zod.ts b/packages/spec/src/data/driver/mongo.zod.ts index 6241ffe71b..95dc7db34d 100644 --- a/packages/spec/src/data/driver/mongo.zod.ts +++ b/packages/spec/src/data/driver/mongo.zod.ts @@ -130,13 +130,18 @@ export const MongoConfigSchema = lazySchema(() => strictObject( * into `MongoCredentials`, so a passthrough password authenticated for real * while sitting cleartext in `sys_metadata`. A non-empty `auth.password` is * refused with the binder prescription; `auth.username` stays writable - * (#8876's asymmetry — a username is not credential material). + * (#8876's asymmetry — a username is not credential material). The + * nested-position finding widened the walk: a non-empty string under a + * credential-SPELLED key (`password`, `authToken`, and the former aliases) + * is refused at ANY object depth of the passthrough, so a nested position + * is treated identically to the top-level key it mirrors instead of + * accepting the same secret one level down. */ options: credentialFreeMongoOptions( placeholderFreeDeep(z.record(z.string(), z.unknown()), 'options'), 'options', ).optional() - .describe('Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they\'re redacted only when the datasource is read back, not refused at write.'), + .describe('Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they\'re redacted only when the datasource is read back, not refused at write.'), }) .describe('MongoDB Connection Configuration') .superRefine((cfg, ctx) => { diff --git a/packages/spec/src/kernel/metadata-type-redaction.test.ts b/packages/spec/src/kernel/metadata-type-redaction.test.ts index 20b8321a01..3ed48468c8 100644 --- a/packages/spec/src/kernel/metadata-type-redaction.test.ts +++ b/packages/spec/src/kernel/metadata-type-redaction.test.ts @@ -114,6 +114,33 @@ describe('FAIL-CLOSED: the datasource redactor is a BUILT-IN, not a plugin regis expect(result.redactedKeys).toEqual(['config.authToken', 'config.encryptionKey', 'config.passwd']); }); + it('a NESTED credential position OFF the passthrough table is withheld at this door too (the class control)', () => { + // The nested-position finding, measured at THIS consumer: a credential + // spelling one object level down (`options.auth.token` — deliberately not + // a `passthroughSecretPaths` row) used to flow through this hook verbatim + // with `redactedKeys: []`, so `/meta/datasource` — a door any + // authenticated caller reaches — served it cleartext. + const stored = { + name: 'events', + driver: 'mongodb', + config: { + database: 'app', + options: { auth: { username: 'svc', token: 'eyJhbGci.SECRET.y' }, replicaSet: 'rs0' }, + }, + }; + const result = getMetadataTypeRedactor('datasource')!(stored) as MetadataRedactionResult; + expect(JSON.stringify(result.item)).not.toContain('SECRET'); + expect(result.item.config).toEqual({ + database: 'app', + options: { auth: { username: 'svc' }, replicaSet: 'rs0' }, + }); + // Dotted, item-relative — the shape the generic write-door carry-forward + // (`carryForwardRedactedValues`) walks, nested paths included. + expect(result.redactedKeys).toEqual(['config.options.auth.token']); + // Pure: the stored body keeps its material for the connect path. + expect((stored.config.options.auth as Record).token).toBe('eyJhbGci.SECRET.y'); + }); + it('an item with no config object is passed through as-is', () => { const noConfig = { name: 'x', driver: 'postgres' }; expect(getMetadataTypeRedactor('datasource')!(noConfig)).toEqual({ diff --git a/packages/spec/src/migrations/entries/semantic/18.datasource-config-options-nested-credential-spelling-refused.ts b/packages/spec/src/migrations/entries/semantic/18.datasource-config-options-nested-credential-spelling-refused.ts new file mode 100644 index 0000000000..5b9b626cec --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/18.datasource-config-options-nested-credential-spelling-refused.ts @@ -0,0 +1,37 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +export const entry: SemanticMigration = { + id: 'datasource-config-options-nested-credential-spelling-refused', + surface: 'datasource.config.options.**: any credential-SPELLED key (`password`, `authToken`, ' + + 'or a former alias — `passwd`/`pwd`/`token`/`jwt`/`auth_token`/`authtoken`) holding a ' + + 'non-empty string at any object depth of the mongodb options passthrough', + replacement: 'remove the nested key (no measured client behaviour reads any such position ' + + 'other than `auth.password`, which has its own refusal); if a real secret must reach the ' + + 'connection, bind it — the Setup → Datasources connection form\'s secret field (encrypted ' + + 'into `sys_secret`, handle stored at `external.credentialsRef`) or a direct ' + + '`external.credentialsRef` reference', + reason: + 'The nested-position closure of the `datasource-config-mongo-options-credential-refused` ' + + 'family: that entry refused the one MEASURED login position (`options.auth.password`) and ' + + 'left every other nested spelling of the same secret accepted — `options.auth.token`, ' + + '`options.pool.password`, any credential-spelled key one object level down parsed green, ' + + 'persisted cleartext into `sys_metadata` (served back by the ordinary data API), and was ' + + 'served on the datasource read doors with `redactedConfigKeys: []` because the read-side ' + + 'nested judgment was a hand-enumerated path table. Publish now refuses a non-empty string ' + + 'under any credential SPELLING at any object depth of the passthrough — the same one ' + + 'spelling list the top level refuses and the read path redacts, so a nested position is ' + + 'treated identically to the top-level key it mirrors. Arrays are off the walk (row-shaped ' + + 'data is not config). The read path now also redacts these spellings at every depth, for ' + + 'every driver, and carries them forward on an untouched Save. There is no mechanical ' + + 'rewrite, for the same reason as the sibling credential entries: moving a value into ' + + '`sys_secret` requires a running secret binder, which a source transform cannot do — and ' + + 'unlike `auth.password`, a nested spelling at an unmeasured position buys nothing at ' + + 'connect, so the usual outcome is deletion, which only the author can confirm.', + acceptanceCriteria: + 'Every mongodb datasource parses with no non-empty credential-spelled string at any object ' + + 'depth of `config.options`; any real secret found there is re-bound through ' + + '`external.credentialsRef` (or the connection form) and the datasource still connects; no ' + + 'nested credential remains in any stored `sys_metadata` row or authored source.', +}; diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index a948c5dfc8..fdec7fc795 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -5709,6 +5709,39 @@ const step18: MigrationStep = { 'the connection form) with the username in its URL, and still connects; no passthrough ' + 'credential remains in any stored `sys_metadata` row or authored source.', }, + { + id: 'datasource-config-options-nested-credential-spelling-refused', + surface: 'datasource.config.options.**: any credential-SPELLED key (`password`, `authToken`, ' + + 'or a former alias — `passwd`/`pwd`/`token`/`jwt`/`auth_token`/`authtoken`) holding a ' + + 'non-empty string at any object depth of the mongodb options passthrough', + replacement: 'remove the nested key (no measured client behaviour reads any such position ' + + 'other than `auth.password`, which has its own refusal); if a real secret must reach the ' + + 'connection, bind it — the Setup → Datasources connection form\'s secret field (encrypted ' + + 'into `sys_secret`, handle stored at `external.credentialsRef`) or a direct ' + + '`external.credentialsRef` reference', + reason: + 'The nested-position closure of the `datasource-config-mongo-options-credential-refused` ' + + 'family: that entry refused the one MEASURED login position (`options.auth.password`) and ' + + 'left every other nested spelling of the same secret accepted — `options.auth.token`, ' + + '`options.pool.password`, any credential-spelled key one object level down parsed green, ' + + 'persisted cleartext into `sys_metadata` (served back by the ordinary data API), and was ' + + 'served on the datasource read doors with `redactedConfigKeys: []` because the read-side ' + + 'nested judgment was a hand-enumerated path table. Publish now refuses a non-empty string ' + + 'under any credential SPELLING at any object depth of the passthrough — the same one ' + + 'spelling list the top level refuses and the read path redacts, so a nested position is ' + + 'treated identically to the top-level key it mirrors. Arrays are off the walk (row-shaped ' + + 'data is not config). The read path now also redacts these spellings at every depth, for ' + + 'every driver, and carries them forward on an untouched Save. There is no mechanical ' + + 'rewrite, for the same reason as the sibling credential entries: moving a value into ' + + '`sys_secret` requires a running secret binder, which a source transform cannot do — and ' + + 'unlike `auth.password`, a nested spelling at an unmeasured position buys nothing at ' + + 'connect, so the usual outcome is deletion, which only the author can confirm.', + acceptanceCriteria: + 'Every mongodb datasource parses with no non-empty credential-spelled string at any object ' + + 'depth of `config.options`; any real secret found there is re-bound through ' + + '`external.credentialsRef` (or the connection form) and the datasource still connects; no ' + + 'nested credential remains in any stored `sys_metadata` row or authored source.', + }, { id: 'datasource-config-postgres-url-unparseable-refused', surface: 'datasource.config.url (postgres) — connection URLs the `pg` client cannot parse ' +