Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/nested-credential-redaction-class.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
---
'@objectstack/spec': minor
'@objectstack/service-datasource': patch
---

feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)

**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
credential under the very spelling the top level refuses and redacts — one
object level down (`options.auth.token`, `options.pool.password`, a
`tunnel.password` on a contract-less driver) — was accepted at publish and
served back by every datasource read door in cleartext with
`redactedConfigKeys: []`, because the read side's nested judgment was only the
hand-enumerated `passthroughSecretPaths` table and the write side had no nested
name judgment at all.

Both sides are now derived from ONE source instead of hand-maintained:

- The canonical credential spellings and former aliases moved to
`driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
driver-schema import graph — so the write door's passthrough walk and the
read redactor consume the same list (#8300's no-second-copy posture applied
to the list itself).
- **Read door** (`redactDatasourceConfig`, behind both consumers — the
datasource-admin routes and the kernel per-type redaction hook): the
credential-name judgment and the URL composite (userinfo + query params) now
run at EVERY object depth, for every driver, contract-less included. Nested
removals are reported as dotted paths in `redactedKeys`, plus a new
`redactedPaths` field carrying exact segments. `passthroughSecretPaths`
remains only as the residue it always should have been: CLIENT-MEASURED
secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
top-level key.
- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
credential-spelled key is refused at any object depth of the mongodb
`options` passthrough, with a prescription that does not inherit the
`auth.password`-only "wins over" reassurance. The measured `auth.password`
refusal keeps its own message; nothing is double-reported.
- **Schema derivation walked at depth**: `refusedCredentialPaths` /
`refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
top level, so a driver contract that refuses a key inside a nested object
shape is covered the day it lands (none exists today — pinned per driver).
- **Arrays are off the walk** on both doors — the same structural line
`valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
`initialData` seeds) keeps its own fields without a per-driver exclusion
list.
- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
redactor instead of mirroring it rule by rule: it grafts stored material
back wherever the patch is indistinguishable from what the read path served,
so an untouched "Save" on an affected legacy row keeps its stored material
for every current and future redaction source, and an author's edit always
wins. The metadata write door's generic `carryForwardRedactedValues` already
walks the dotted paths and needs no change.

Semantic migration entry
`datasource-config-options-nested-credential-spelling-refused` (protocol major
18) carries the authored-artifact upgrade: remove the nested key, or bind the
real secret through `external.credentialsRef` / the connection form.

<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ MongoDB Connection Configuration
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |


---
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
).rejects.toThrow(/options\.auth\.password/);
});
});

describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
/**
* The nested-position finding, at the admin door: a credential spelling one
* object level down from the key the top level hides — deliberately NOT a
* `passthroughSecretPaths` row — used to be served by `getDatasource()` in
* cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
* silence (measured on the pre-fix build). Regression cases against table
* rows were already green and prove nothing about this class.
*/
const OFF_TABLE_MONGO: StoredDatasource = {
name: 'off_table_mongo',
driver: 'mongodb',
origin: 'runtime',
config: {
database: 'events',
options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
},
};

it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
expect(read!.redactedConfigKeys).toContain('options.auth.token');
});

it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
const { service, records } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
expect(records[0].label).toBe('Renamed');
expect((records[0].config!.options as any).auth).toEqual({
username: 'app',
token: 'eyJhbGci.OFFTABLE.y',
});
});

it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
await expect(
service.updateDatasource('off_table_mongo', {
config: {
database: 'events',
options: { auth: { username: 'app', token: 'typed-new-secret' } },
},
}),
).rejects.toThrow(/options\.auth\.token/);
});

it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
const stored: StoredDatasource = {
name: 'vendor_ds',
driver: 'com.vendor.custom',
origin: 'runtime',
config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
};
const { service, records } = makeService([stored]);
const read = await service.getDatasource('vendor_ds');
expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
expect(read!.redactedConfigKeys).toContain('replication.url');
// Untouched round-trip: the served (redacted) URL is indistinguishable
// from the stored one once redacted, so the stored value is carried back.
await service.updateDatasource('vendor_ds', { config: read!.config });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
// An author who rewrites the nested URL by hand still WINS.
const edited = {
...read!.config,
replication: { url: 'postgresql://svc@other-replica/db' },
};
await service.updateDatasource('vendor_ds', { config: edited });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
});
});
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,11 +31,7 @@
* #8154's, deliberately not built here.
*/

import {
passthroughSecretPaths,
redactableConfigKeys,
redactUrlCredentials,
} from '@objectstack/spec/data';
import { redactDatasourceConfig } from '@objectstack/spec/data';

export {
refusedCredentialKeys,
Expand All@@ -56,6 +52,24 @@ export {
* where the patch is indistinguishable from what the read path served — an
* absent key, or a URL that matches the stored URL once redacted. Anything the
* author actually changed wins, including clearing a URL's password by hand.
* A patch whose CONTAINER for a nested leaf is removed is the author's word
* too (they deleted the block), so nothing is grafted there.
*
* ## Derived from the redactor, not restated beside it
*
* This function used to mirror the read path rule by rule — one loop per
* redaction source, each a copy that could silently fall behind (the docblock
* threat on every one of them: "a redaction the restore side did not mirror
* turns an untouched Save into silent credential deletion"). The nested-
* position fix made the read path recursive, which would have added two more
* loops — so the mirroring is now structural instead: compute what the read
* path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
* and for every redacted path graft the stored value back exactly where the
* patch still matches the served projection. A future redaction source is
* mirrored here by construction, with nothing to forget. (Same inversion the
* metadata door's generic `carryForwardRedactedValues` performs; this one
* consumes the redactor's exact `redactedPaths` segments, so a stored key
* with a literal dot cannot be mis-split.)
*
* What this does NOT do is let a patch set a refused key: `assertValidConfig`
* still runs on the merged record, so a caller that types `password` into the
Expand All@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
if (!patch || typeof patch !== 'object') return patch;
if (!stored || typeof stored !== 'object') return patch;

const hidden = new Set(redactableConfigKeys(driver));
const served = redactDatasourceConfig(driver, stored);
const out: Record<string, unknown> = { ...patch };

for (const key of hidden) {
// Only when the patch does not speak to the key at all. A patch that DOES
// carry it is the author's word, and (for a refused spelling) is about to
// be refused on its own merits rather than quietly overwritten here.
if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
}

for (const [key, storedValue] of Object.entries(stored)) {
if (hidden.has(key) || typeof storedValue !== 'string') continue;
// The SAME composite the read path applies (userinfo password + #8337
// credential query parameters) — a redaction this compare did not mirror
// would make the untouched "Save" it exists for delete the credential.
const redactedStored = redactUrlCredentials(storedValue);
// Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
if (redactedStored === storedValue) continue;
if (out[key] === redactedStored) out[key] = storedValue;
}

// The passthrough spellings (#9040) — the nested material the read path
// drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
// same narrow rule as the top-level keys, translated per leaf: restore ONLY
// when the patch's container for the leaf exists but does not speak to the
// leaf at all — exactly what the read path served. A patch carrying the leaf
// is the author's word (a typed-in `auth.password` is then refused by the
// #9040 write gate on its own merits); a patch with the CONTAINER removed is
// the author's word too (they deleted the block), so nothing is grafted.
for (const path of passthroughSecretPaths(driver)) {
for (const path of served.redactedPaths) {
const storedLeaf = valueAt(stored, path);
if (storedLeaf === undefined) continue;
const parentPath = path.slice(0, -1);
const leafKey = path[path.length - 1] as string;
const patchParent = valueAt(out, parentPath);
const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
if (leafKey in (patchParent as Record<string, unknown>)) continue;
// What the read path served at this position: `undefined` for a dropped
// key, the rewritten string for a URL redaction. The patch speaks for the
// author exactly where it DIFFERS from that projection.
const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
const servedLeaf =
servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
? (servedParent as Record<string, unknown>)[leafKey]
: undefined;
if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
graftAt(out, path, storedLeaf);
}

Expand Down
6 changes: 6 additions & 0 deletions packages/spec/api-surface/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase (type)",
"BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode (type)",
"FeedItemType (type)",
"Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase (interface)",
"PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths (function)",
"renderAutonumber (function)",
Expand Down
6 changes: 6 additions & 0 deletions packages/spec/export-origins/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
"BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
"FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
"Field": "src/data/field.zod.ts#Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
"PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/nested-credential-redaction-class.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
---
'@objectstack/spec': minor
'@objectstack/service-datasource': patch
---

feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)

**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
credential under the very spelling the top level refuses and redacts — one
object level down (`options.auth.token`, `options.pool.password`, a
`tunnel.password` on a contract-less driver) — was accepted at publish and
served back by every datasource read door in cleartext with
`redactedConfigKeys: []`, because the read side's nested judgment was only the
hand-enumerated `passthroughSecretPaths` table and the write side had no nested
name judgment at all.

Both sides are now derived from ONE source instead of hand-maintained:

- The canonical credential spellings and former aliases moved to
`driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
driver-schema import graph — so the write door's passthrough walk and the
read redactor consume the same list (#8300's no-second-copy posture applied
to the list itself).
- **Read door** (`redactDatasourceConfig`, behind both consumers — the
datasource-admin routes and the kernel per-type redaction hook): the
credential-name judgment and the URL composite (userinfo + query params) now
run at EVERY object depth, for every driver, contract-less included. Nested
removals are reported as dotted paths in `redactedKeys`, plus a new
`redactedPaths` field carrying exact segments. `passthroughSecretPaths`
remains only as the residue it always should have been: CLIENT-MEASURED
secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
top-level key.
- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
credential-spelled key is refused at any object depth of the mongodb
`options` passthrough, with a prescription that does not inherit the
`auth.password`-only "wins over" reassurance. The measured `auth.password`
refusal keeps its own message; nothing is double-reported.
- **Schema derivation walked at depth**: `refusedCredentialPaths` /
`refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
top level, so a driver contract that refuses a key inside a nested object
shape is covered the day it lands (none exists today — pinned per driver).
- **Arrays are off the walk** on both doors — the same structural line
`valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
`initialData` seeds) keeps its own fields without a per-driver exclusion
list.
- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
redactor instead of mirroring it rule by rule: it grafts stored material
back wherever the patch is indistinguishable from what the read path served,
so an untouched "Save" on an affected legacy row keeps its stored material
for every current and future redaction source, and an author's edit always
wins. The metadata write door's generic `carryForwardRedactedValues` already
walks the dotted paths and needs no change.

Semantic migration entry
`datasource-config-options-nested-credential-spelling-refused` (protocol major
18) carries the authored-artifact upgrade: remove the nested key, or bind the
real secret through `external.credentialsRef` / the connection form.

<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ MongoDB Connection Configuration
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |


---
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
).rejects.toThrow(/options\.auth\.password/);
});
});

describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
/**
* The nested-position finding, at the admin door: a credential spelling one
* object level down from the key the top level hides — deliberately NOT a
* `passthroughSecretPaths` row — used to be served by `getDatasource()` in
* cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
* silence (measured on the pre-fix build). Regression cases against table
* rows were already green and prove nothing about this class.
*/
const OFF_TABLE_MONGO: StoredDatasource = {
name: 'off_table_mongo',
driver: 'mongodb',
origin: 'runtime',
config: {
database: 'events',
options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
},
};

it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
expect(read!.redactedConfigKeys).toContain('options.auth.token');
});

it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
const { service, records } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
expect(records[0].label).toBe('Renamed');
expect((records[0].config!.options as any).auth).toEqual({
username: 'app',
token: 'eyJhbGci.OFFTABLE.y',
});
});

it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
await expect(
service.updateDatasource('off_table_mongo', {
config: {
database: 'events',
options: { auth: { username: 'app', token: 'typed-new-secret' } },
},
}),
).rejects.toThrow(/options\.auth\.token/);
});

it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
const stored: StoredDatasource = {
name: 'vendor_ds',
driver: 'com.vendor.custom',
origin: 'runtime',
config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
};
const { service, records } = makeService([stored]);
const read = await service.getDatasource('vendor_ds');
expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
expect(read!.redactedConfigKeys).toContain('replication.url');
// Untouched round-trip: the served (redacted) URL is indistinguishable
// from the stored one once redacted, so the stored value is carried back.
await service.updateDatasource('vendor_ds', { config: read!.config });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
// An author who rewrites the nested URL by hand still WINS.
const edited = {
...read!.config,
replication: { url: 'postgresql://svc@other-replica/db' },
};
await service.updateDatasource('vendor_ds', { config: edited });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
});
});
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,11 +31,7 @@
* #8154's, deliberately not built here.
*/

import {
passthroughSecretPaths,
redactableConfigKeys,
redactUrlCredentials,
} from '@objectstack/spec/data';
import { redactDatasourceConfig } from '@objectstack/spec/data';

export {
refusedCredentialKeys,
Expand All@@ -56,6 +52,24 @@ export {
* where the patch is indistinguishable from what the read path served — an
* absent key, or a URL that matches the stored URL once redacted. Anything the
* author actually changed wins, including clearing a URL's password by hand.
* A patch whose CONTAINER for a nested leaf is removed is the author's word
* too (they deleted the block), so nothing is grafted there.
*
* ## Derived from the redactor, not restated beside it
*
* This function used to mirror the read path rule by rule — one loop per
* redaction source, each a copy that could silently fall behind (the docblock
* threat on every one of them: "a redaction the restore side did not mirror
* turns an untouched Save into silent credential deletion"). The nested-
* position fix made the read path recursive, which would have added two more
* loops — so the mirroring is now structural instead: compute what the read
* path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
* and for every redacted path graft the stored value back exactly where the
* patch still matches the served projection. A future redaction source is
* mirrored here by construction, with nothing to forget. (Same inversion the
* metadata door's generic `carryForwardRedactedValues` performs; this one
* consumes the redactor's exact `redactedPaths` segments, so a stored key
* with a literal dot cannot be mis-split.)
*
* What this does NOT do is let a patch set a refused key: `assertValidConfig`
* still runs on the merged record, so a caller that types `password` into the
Expand All@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
if (!patch || typeof patch !== 'object') return patch;
if (!stored || typeof stored !== 'object') return patch;

const hidden = new Set(redactableConfigKeys(driver));
const served = redactDatasourceConfig(driver, stored);
const out: Record<string, unknown> = { ...patch };

for (const key of hidden) {
// Only when the patch does not speak to the key at all. A patch that DOES
// carry it is the author's word, and (for a refused spelling) is about to
// be refused on its own merits rather than quietly overwritten here.
if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
}

for (const [key, storedValue] of Object.entries(stored)) {
if (hidden.has(key) || typeof storedValue !== 'string') continue;
// The SAME composite the read path applies (userinfo password + #8337
// credential query parameters) — a redaction this compare did not mirror
// would make the untouched "Save" it exists for delete the credential.
const redactedStored = redactUrlCredentials(storedValue);
// Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
if (redactedStored === storedValue) continue;
if (out[key] === redactedStored) out[key] = storedValue;
}

// The passthrough spellings (#9040) — the nested material the read path
// drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
// same narrow rule as the top-level keys, translated per leaf: restore ONLY
// when the patch's container for the leaf exists but does not speak to the
// leaf at all — exactly what the read path served. A patch carrying the leaf
// is the author's word (a typed-in `auth.password` is then refused by the
// #9040 write gate on its own merits); a patch with the CONTAINER removed is
// the author's word too (they deleted the block), so nothing is grafted.
for (const path of passthroughSecretPaths(driver)) {
for (const path of served.redactedPaths) {
const storedLeaf = valueAt(stored, path);
if (storedLeaf === undefined) continue;
const parentPath = path.slice(0, -1);
const leafKey = path[path.length - 1] as string;
const patchParent = valueAt(out, parentPath);
const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
if (leafKey in (patchParent as Record<string, unknown>)) continue;
// What the read path served at this position: `undefined` for a dropped
// key, the rewritten string for a URL redaction. The patch speaks for the
// author exactly where it DIFFERS from that projection.
const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
const servedLeaf =
servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
? (servedParent as Record<string, unknown>)[leafKey]
: undefined;
if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
graftAt(out, path, storedLeaf);
}

Expand Down
6 changes: 6 additions & 0 deletions packages/spec/api-surface/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase (type)",
"BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode (type)",
"FeedItemType (type)",
"Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase (interface)",
"PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths (function)",
"renderAutonumber (function)",
Expand Down
6 changes: 6 additions & 0 deletions packages/spec/export-origins/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
"BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
"FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
"Field": "src/data/field.zod.ts#Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
"PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/nested-credential-redaction-class.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
---
'@objectstack/spec': minor
'@objectstack/service-datasource': patch
---

feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)

**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
credential under the very spelling the top level refuses and redacts — one
object level down (`options.auth.token`, `options.pool.password`, a
`tunnel.password` on a contract-less driver) — was accepted at publish and
served back by every datasource read door in cleartext with
`redactedConfigKeys: []`, because the read side's nested judgment was only the
hand-enumerated `passthroughSecretPaths` table and the write side had no nested
name judgment at all.

Both sides are now derived from ONE source instead of hand-maintained:

- The canonical credential spellings and former aliases moved to
`driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
driver-schema import graph — so the write door's passthrough walk and the
read redactor consume the same list (#8300's no-second-copy posture applied
to the list itself).
- **Read door** (`redactDatasourceConfig`, behind both consumers — the
datasource-admin routes and the kernel per-type redaction hook): the
credential-name judgment and the URL composite (userinfo + query params) now
run at EVERY object depth, for every driver, contract-less included. Nested
removals are reported as dotted paths in `redactedKeys`, plus a new
`redactedPaths` field carrying exact segments. `passthroughSecretPaths`
remains only as the residue it always should have been: CLIENT-MEASURED
secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
top-level key.
- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
credential-spelled key is refused at any object depth of the mongodb
`options` passthrough, with a prescription that does not inherit the
`auth.password`-only "wins over" reassurance. The measured `auth.password`
refusal keeps its own message; nothing is double-reported.
- **Schema derivation walked at depth**: `refusedCredentialPaths` /
`refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
top level, so a driver contract that refuses a key inside a nested object
shape is covered the day it lands (none exists today — pinned per driver).
- **Arrays are off the walk** on both doors — the same structural line
`valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
`initialData` seeds) keeps its own fields without a per-driver exclusion
list.
- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
redactor instead of mirroring it rule by rule: it grafts stored material
back wherever the patch is indistinguishable from what the read path served,
so an untouched "Save" on an affected legacy row keeps its stored material
for every current and future redaction source, and an author's edit always
wins. The metadata write door's generic `carryForwardRedactedValues` already
walks the dotted paths and needs no change.

Semantic migration entry
`datasource-config-options-nested-credential-spelling-refused` (protocol major
18) carries the authored-artifact upgrade: remove the nested key, or bind the
real secret through `external.credentialsRef` / the connection form.

<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ MongoDB Connection Configuration
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |


---
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
).rejects.toThrow(/options\.auth\.password/);
});
});

describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
/**
* The nested-position finding, at the admin door: a credential spelling one
* object level down from the key the top level hides — deliberately NOT a
* `passthroughSecretPaths` row — used to be served by `getDatasource()` in
* cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
* silence (measured on the pre-fix build). Regression cases against table
* rows were already green and prove nothing about this class.
*/
const OFF_TABLE_MONGO: StoredDatasource = {
name: 'off_table_mongo',
driver: 'mongodb',
origin: 'runtime',
config: {
database: 'events',
options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
},
};

it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
expect(read!.redactedConfigKeys).toContain('options.auth.token');
});

it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
const { service, records } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
expect(records[0].label).toBe('Renamed');
expect((records[0].config!.options as any).auth).toEqual({
username: 'app',
token: 'eyJhbGci.OFFTABLE.y',
});
});

it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
await expect(
service.updateDatasource('off_table_mongo', {
config: {
database: 'events',
options: { auth: { username: 'app', token: 'typed-new-secret' } },
},
}),
).rejects.toThrow(/options\.auth\.token/);
});

it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
const stored: StoredDatasource = {
name: 'vendor_ds',
driver: 'com.vendor.custom',
origin: 'runtime',
config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
};
const { service, records } = makeService([stored]);
const read = await service.getDatasource('vendor_ds');
expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
expect(read!.redactedConfigKeys).toContain('replication.url');
// Untouched round-trip: the served (redacted) URL is indistinguishable
// from the stored one once redacted, so the stored value is carried back.
await service.updateDatasource('vendor_ds', { config: read!.config });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
// An author who rewrites the nested URL by hand still WINS.
const edited = {
...read!.config,
replication: { url: 'postgresql://svc@other-replica/db' },
};
await service.updateDatasource('vendor_ds', { config: edited });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
});
});
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,11 +31,7 @@
* #8154's, deliberately not built here.
*/

import {
passthroughSecretPaths,
redactableConfigKeys,
redactUrlCredentials,
} from '@objectstack/spec/data';
import { redactDatasourceConfig } from '@objectstack/spec/data';

export {
refusedCredentialKeys,
Expand All@@ -56,6 +52,24 @@ export {
* where the patch is indistinguishable from what the read path served — an
* absent key, or a URL that matches the stored URL once redacted. Anything the
* author actually changed wins, including clearing a URL's password by hand.
* A patch whose CONTAINER for a nested leaf is removed is the author's word
* too (they deleted the block), so nothing is grafted there.
*
* ## Derived from the redactor, not restated beside it
*
* This function used to mirror the read path rule by rule — one loop per
* redaction source, each a copy that could silently fall behind (the docblock
* threat on every one of them: "a redaction the restore side did not mirror
* turns an untouched Save into silent credential deletion"). The nested-
* position fix made the read path recursive, which would have added two more
* loops — so the mirroring is now structural instead: compute what the read
* path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
* and for every redacted path graft the stored value back exactly where the
* patch still matches the served projection. A future redaction source is
* mirrored here by construction, with nothing to forget. (Same inversion the
* metadata door's generic `carryForwardRedactedValues` performs; this one
* consumes the redactor's exact `redactedPaths` segments, so a stored key
* with a literal dot cannot be mis-split.)
*
* What this does NOT do is let a patch set a refused key: `assertValidConfig`
* still runs on the merged record, so a caller that types `password` into the
Expand All@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
if (!patch || typeof patch !== 'object') return patch;
if (!stored || typeof stored !== 'object') return patch;

const hidden = new Set(redactableConfigKeys(driver));
const served = redactDatasourceConfig(driver, stored);
const out: Record<string, unknown> = { ...patch };

for (const key of hidden) {
// Only when the patch does not speak to the key at all. A patch that DOES
// carry it is the author's word, and (for a refused spelling) is about to
// be refused on its own merits rather than quietly overwritten here.
if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
}

for (const [key, storedValue] of Object.entries(stored)) {
if (hidden.has(key) || typeof storedValue !== 'string') continue;
// The SAME composite the read path applies (userinfo password + #8337
// credential query parameters) — a redaction this compare did not mirror
// would make the untouched "Save" it exists for delete the credential.
const redactedStored = redactUrlCredentials(storedValue);
// Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
if (redactedStored === storedValue) continue;
if (out[key] === redactedStored) out[key] = storedValue;
}

// The passthrough spellings (#9040) — the nested material the read path
// drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
// same narrow rule as the top-level keys, translated per leaf: restore ONLY
// when the patch's container for the leaf exists but does not speak to the
// leaf at all — exactly what the read path served. A patch carrying the leaf
// is the author's word (a typed-in `auth.password` is then refused by the
// #9040 write gate on its own merits); a patch with the CONTAINER removed is
// the author's word too (they deleted the block), so nothing is grafted.
for (const path of passthroughSecretPaths(driver)) {
for (const path of served.redactedPaths) {
const storedLeaf = valueAt(stored, path);
if (storedLeaf === undefined) continue;
const parentPath = path.slice(0, -1);
const leafKey = path[path.length - 1] as string;
const patchParent = valueAt(out, parentPath);
const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
if (leafKey in (patchParent as Record<string, unknown>)) continue;
// What the read path served at this position: `undefined` for a dropped
// key, the rewritten string for a URL redaction. The patch speaks for the
// author exactly where it DIFFERS from that projection.
const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
const servedLeaf =
servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
? (servedParent as Record<string, unknown>)[leafKey]
: undefined;
if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
graftAt(out, path, storedLeaf);
}

Expand Down
6 changes: 6 additions & 0 deletions packages/spec/api-surface/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase (type)",
"BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode (type)",
"FeedItemType (type)",
"Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase (interface)",
"PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths (function)",
"renderAutonumber (function)",
Expand Down
6 changes: 6 additions & 0 deletions packages/spec/export-origins/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
"BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
"FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
"Field": "src/data/field.zod.ts#Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
"PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/nested-credential-redaction-class.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
---
'@objectstack/spec': minor
'@objectstack/service-datasource': patch
---

feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)

**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
credential under the very spelling the top level refuses and redacts — one
object level down (`options.auth.token`, `options.pool.password`, a
`tunnel.password` on a contract-less driver) — was accepted at publish and
served back by every datasource read door in cleartext with
`redactedConfigKeys: []`, because the read side's nested judgment was only the
hand-enumerated `passthroughSecretPaths` table and the write side had no nested
name judgment at all.

Both sides are now derived from ONE source instead of hand-maintained:

- The canonical credential spellings and former aliases moved to
`driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
driver-schema import graph — so the write door's passthrough walk and the
read redactor consume the same list (#8300's no-second-copy posture applied
to the list itself).
- **Read door** (`redactDatasourceConfig`, behind both consumers — the
datasource-admin routes and the kernel per-type redaction hook): the
credential-name judgment and the URL composite (userinfo + query params) now
run at EVERY object depth, for every driver, contract-less included. Nested
removals are reported as dotted paths in `redactedKeys`, plus a new
`redactedPaths` field carrying exact segments. `passthroughSecretPaths`
remains only as the residue it always should have been: CLIENT-MEASURED
secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
top-level key.
- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
credential-spelled key is refused at any object depth of the mongodb
`options` passthrough, with a prescription that does not inherit the
`auth.password`-only "wins over" reassurance. The measured `auth.password`
refusal keeps its own message; nothing is double-reported.
- **Schema derivation walked at depth**: `refusedCredentialPaths` /
`refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
top level, so a driver contract that refuses a key inside a nested object
shape is covered the day it lands (none exists today — pinned per driver).
- **Arrays are off the walk** on both doors — the same structural line
`valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
`initialData` seeds) keeps its own fields without a per-driver exclusion
list.
- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
redactor instead of mirroring it rule by rule: it grafts stored material
back wherever the patch is indistinguishable from what the read path served,
so an untouched "Save" on an affected legacy row keeps its stored material
for every current and future redaction source, and an author's edit always
wins. The metadata write door's generic `carryForwardRedactedValues` already
walks the dotted paths and needs no change.

Semantic migration entry
`datasource-config-options-nested-credential-spelling-refused` (protocol major
18) carries the authored-artifact upgrade: remove the nested key, or bind the
real secret through `external.credentialsRef` / the connection form.

<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ MongoDB Connection Configuration
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |


---
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
).rejects.toThrow(/options\.auth\.password/);
});
});

describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
/**
* The nested-position finding, at the admin door: a credential spelling one
* object level down from the key the top level hides — deliberately NOT a
* `passthroughSecretPaths` row — used to be served by `getDatasource()` in
* cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
* silence (measured on the pre-fix build). Regression cases against table
* rows were already green and prove nothing about this class.
*/
const OFF_TABLE_MONGO: StoredDatasource = {
name: 'off_table_mongo',
driver: 'mongodb',
origin: 'runtime',
config: {
database: 'events',
options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
},
};

it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
expect(read!.redactedConfigKeys).toContain('options.auth.token');
});

it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
const { service, records } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
expect(records[0].label).toBe('Renamed');
expect((records[0].config!.options as any).auth).toEqual({
username: 'app',
token: 'eyJhbGci.OFFTABLE.y',
});
});

it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
await expect(
service.updateDatasource('off_table_mongo', {
config: {
database: 'events',
options: { auth: { username: 'app', token: 'typed-new-secret' } },
},
}),
).rejects.toThrow(/options\.auth\.token/);
});

it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
const stored: StoredDatasource = {
name: 'vendor_ds',
driver: 'com.vendor.custom',
origin: 'runtime',
config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
};
const { service, records } = makeService([stored]);
const read = await service.getDatasource('vendor_ds');
expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
expect(read!.redactedConfigKeys).toContain('replication.url');
// Untouched round-trip: the served (redacted) URL is indistinguishable
// from the stored one once redacted, so the stored value is carried back.
await service.updateDatasource('vendor_ds', { config: read!.config });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
// An author who rewrites the nested URL by hand still WINS.
const edited = {
...read!.config,
replication: { url: 'postgresql://svc@other-replica/db' },
};
await service.updateDatasource('vendor_ds', { config: edited });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
});
});
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,11 +31,7 @@
* #8154's, deliberately not built here.
*/

import {
passthroughSecretPaths,
redactableConfigKeys,
redactUrlCredentials,
} from '@objectstack/spec/data';
import { redactDatasourceConfig } from '@objectstack/spec/data';

export {
refusedCredentialKeys,
Expand All@@ -56,6 +52,24 @@ export {
* where the patch is indistinguishable from what the read path served — an
* absent key, or a URL that matches the stored URL once redacted. Anything the
* author actually changed wins, including clearing a URL's password by hand.
* A patch whose CONTAINER for a nested leaf is removed is the author's word
* too (they deleted the block), so nothing is grafted there.
*
* ## Derived from the redactor, not restated beside it
*
* This function used to mirror the read path rule by rule — one loop per
* redaction source, each a copy that could silently fall behind (the docblock
* threat on every one of them: "a redaction the restore side did not mirror
* turns an untouched Save into silent credential deletion"). The nested-
* position fix made the read path recursive, which would have added two more
* loops — so the mirroring is now structural instead: compute what the read
* path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
* and for every redacted path graft the stored value back exactly where the
* patch still matches the served projection. A future redaction source is
* mirrored here by construction, with nothing to forget. (Same inversion the
* metadata door's generic `carryForwardRedactedValues` performs; this one
* consumes the redactor's exact `redactedPaths` segments, so a stored key
* with a literal dot cannot be mis-split.)
*
* What this does NOT do is let a patch set a refused key: `assertValidConfig`
* still runs on the merged record, so a caller that types `password` into the
Expand All@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
if (!patch || typeof patch !== 'object') return patch;
if (!stored || typeof stored !== 'object') return patch;

const hidden = new Set(redactableConfigKeys(driver));
const served = redactDatasourceConfig(driver, stored);
const out: Record<string, unknown> = { ...patch };

for (const key of hidden) {
// Only when the patch does not speak to the key at all. A patch that DOES
// carry it is the author's word, and (for a refused spelling) is about to
// be refused on its own merits rather than quietly overwritten here.
if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
}

for (const [key, storedValue] of Object.entries(stored)) {
if (hidden.has(key) || typeof storedValue !== 'string') continue;
// The SAME composite the read path applies (userinfo password + #8337
// credential query parameters) — a redaction this compare did not mirror
// would make the untouched "Save" it exists for delete the credential.
const redactedStored = redactUrlCredentials(storedValue);
// Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
if (redactedStored === storedValue) continue;
if (out[key] === redactedStored) out[key] = storedValue;
}

// The passthrough spellings (#9040) — the nested material the read path
// drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
// same narrow rule as the top-level keys, translated per leaf: restore ONLY
// when the patch's container for the leaf exists but does not speak to the
// leaf at all — exactly what the read path served. A patch carrying the leaf
// is the author's word (a typed-in `auth.password` is then refused by the
// #9040 write gate on its own merits); a patch with the CONTAINER removed is
// the author's word too (they deleted the block), so nothing is grafted.
for (const path of passthroughSecretPaths(driver)) {
for (const path of served.redactedPaths) {
const storedLeaf = valueAt(stored, path);
if (storedLeaf === undefined) continue;
const parentPath = path.slice(0, -1);
const leafKey = path[path.length - 1] as string;
const patchParent = valueAt(out, parentPath);
const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
if (leafKey in (patchParent as Record<string, unknown>)) continue;
// What the read path served at this position: `undefined` for a dropped
// key, the rewritten string for a URL redaction. The patch speaks for the
// author exactly where it DIFFERS from that projection.
const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
const servedLeaf =
servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
? (servedParent as Record<string, unknown>)[leafKey]
: undefined;
if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
graftAt(out, path, storedLeaf);
}

Expand Down
6 changes: 6 additions & 0 deletions packages/spec/api-surface/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase (type)",
"BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode (type)",
"FeedItemType (type)",
"Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase (interface)",
"PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths (function)",
"renderAutonumber (function)",
Expand Down
6 changes: 6 additions & 0 deletions packages/spec/export-origins/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
"BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
"FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
"Field": "src/data/field.zod.ts#Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
"PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/nested-credential-redaction-class.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
---
'@objectstack/spec': minor
'@objectstack/service-datasource': patch
---

feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)

**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
credential under the very spelling the top level refuses and redacts — one
object level down (`options.auth.token`, `options.pool.password`, a
`tunnel.password` on a contract-less driver) — was accepted at publish and
served back by every datasource read door in cleartext with
`redactedConfigKeys: []`, because the read side's nested judgment was only the
hand-enumerated `passthroughSecretPaths` table and the write side had no nested
name judgment at all.

Both sides are now derived from ONE source instead of hand-maintained:

- The canonical credential spellings and former aliases moved to
`driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
driver-schema import graph — so the write door's passthrough walk and the
read redactor consume the same list (#8300's no-second-copy posture applied
to the list itself).
- **Read door** (`redactDatasourceConfig`, behind both consumers — the
datasource-admin routes and the kernel per-type redaction hook): the
credential-name judgment and the URL composite (userinfo + query params) now
run at EVERY object depth, for every driver, contract-less included. Nested
removals are reported as dotted paths in `redactedKeys`, plus a new
`redactedPaths` field carrying exact segments. `passthroughSecretPaths`
remains only as the residue it always should have been: CLIENT-MEASURED
secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
top-level key.
- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
credential-spelled key is refused at any object depth of the mongodb
`options` passthrough, with a prescription that does not inherit the
`auth.password`-only "wins over" reassurance. The measured `auth.password`
refusal keeps its own message; nothing is double-reported.
- **Schema derivation walked at depth**: `refusedCredentialPaths` /
`refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
top level, so a driver contract that refuses a key inside a nested object
shape is covered the day it lands (none exists today — pinned per driver).
- **Arrays are off the walk** on both doors — the same structural line
`valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
`initialData` seeds) keeps its own fields without a per-driver exclusion
list.
- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
redactor instead of mirroring it rule by rule: it grafts stored material
back wherever the patch is indistinguishable from what the read path served,
so an untouched "Save" on an affected legacy row keeps its stored material
for every current and future redaction source, and an author's edit always
wins. The metadata write door's generic `carryForwardRedactedValues` already
walks the dotted paths and needs no change.

Semantic migration entry
`datasource-config-options-nested-credential-spelling-refused` (protocol major
18) carries the authored-artifact upgrade: remove the nested key, or bind the
real secret through `external.credentialsRef` / the connection form.

<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ MongoDB Connection Configuration
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |


---
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
).rejects.toThrow(/options\.auth\.password/);
});
});

describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
/**
* The nested-position finding, at the admin door: a credential spelling one
* object level down from the key the top level hides — deliberately NOT a
* `passthroughSecretPaths` row — used to be served by `getDatasource()` in
* cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
* silence (measured on the pre-fix build). Regression cases against table
* rows were already green and prove nothing about this class.
*/
const OFF_TABLE_MONGO: StoredDatasource = {
name: 'off_table_mongo',
driver: 'mongodb',
origin: 'runtime',
config: {
database: 'events',
options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
},
};

it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
expect(read!.redactedConfigKeys).toContain('options.auth.token');
});

it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
const { service, records } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
expect(records[0].label).toBe('Renamed');
expect((records[0].config!.options as any).auth).toEqual({
username: 'app',
token: 'eyJhbGci.OFFTABLE.y',
});
});

it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
await expect(
service.updateDatasource('off_table_mongo', {
config: {
database: 'events',
options: { auth: { username: 'app', token: 'typed-new-secret' } },
},
}),
).rejects.toThrow(/options\.auth\.token/);
});

it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
const stored: StoredDatasource = {
name: 'vendor_ds',
driver: 'com.vendor.custom',
origin: 'runtime',
config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
};
const { service, records } = makeService([stored]);
const read = await service.getDatasource('vendor_ds');
expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
expect(read!.redactedConfigKeys).toContain('replication.url');
// Untouched round-trip: the served (redacted) URL is indistinguishable
// from the stored one once redacted, so the stored value is carried back.
await service.updateDatasource('vendor_ds', { config: read!.config });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
// An author who rewrites the nested URL by hand still WINS.
const edited = {
...read!.config,
replication: { url: 'postgresql://svc@other-replica/db' },
};
await service.updateDatasource('vendor_ds', { config: edited });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
});
});
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,11 +31,7 @@
* #8154's, deliberately not built here.
*/

import {
passthroughSecretPaths,
redactableConfigKeys,
redactUrlCredentials,
} from '@objectstack/spec/data';
import { redactDatasourceConfig } from '@objectstack/spec/data';

export {
refusedCredentialKeys,
Expand All@@ -56,6 +52,24 @@ export {
* where the patch is indistinguishable from what the read path served — an
* absent key, or a URL that matches the stored URL once redacted. Anything the
* author actually changed wins, including clearing a URL's password by hand.
* A patch whose CONTAINER for a nested leaf is removed is the author's word
* too (they deleted the block), so nothing is grafted there.
*
* ## Derived from the redactor, not restated beside it
*
* This function used to mirror the read path rule by rule — one loop per
* redaction source, each a copy that could silently fall behind (the docblock
* threat on every one of them: "a redaction the restore side did not mirror
* turns an untouched Save into silent credential deletion"). The nested-
* position fix made the read path recursive, which would have added two more
* loops — so the mirroring is now structural instead: compute what the read
* path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
* and for every redacted path graft the stored value back exactly where the
* patch still matches the served projection. A future redaction source is
* mirrored here by construction, with nothing to forget. (Same inversion the
* metadata door's generic `carryForwardRedactedValues` performs; this one
* consumes the redactor's exact `redactedPaths` segments, so a stored key
* with a literal dot cannot be mis-split.)
*
* What this does NOT do is let a patch set a refused key: `assertValidConfig`
* still runs on the merged record, so a caller that types `password` into the
Expand All@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
if (!patch || typeof patch !== 'object') return patch;
if (!stored || typeof stored !== 'object') return patch;

const hidden = new Set(redactableConfigKeys(driver));
const served = redactDatasourceConfig(driver, stored);
const out: Record<string, unknown> = { ...patch };

for (const key of hidden) {
// Only when the patch does not speak to the key at all. A patch that DOES
// carry it is the author's word, and (for a refused spelling) is about to
// be refused on its own merits rather than quietly overwritten here.
if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
}

for (const [key, storedValue] of Object.entries(stored)) {
if (hidden.has(key) || typeof storedValue !== 'string') continue;
// The SAME composite the read path applies (userinfo password + #8337
// credential query parameters) — a redaction this compare did not mirror
// would make the untouched "Save" it exists for delete the credential.
const redactedStored = redactUrlCredentials(storedValue);
// Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
if (redactedStored === storedValue) continue;
if (out[key] === redactedStored) out[key] = storedValue;
}

// The passthrough spellings (#9040) — the nested material the read path
// drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
// same narrow rule as the top-level keys, translated per leaf: restore ONLY
// when the patch's container for the leaf exists but does not speak to the
// leaf at all — exactly what the read path served. A patch carrying the leaf
// is the author's word (a typed-in `auth.password` is then refused by the
// #9040 write gate on its own merits); a patch with the CONTAINER removed is
// the author's word too (they deleted the block), so nothing is grafted.
for (const path of passthroughSecretPaths(driver)) {
for (const path of served.redactedPaths) {
const storedLeaf = valueAt(stored, path);
if (storedLeaf === undefined) continue;
const parentPath = path.slice(0, -1);
const leafKey = path[path.length - 1] as string;
const patchParent = valueAt(out, parentPath);
const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
if (leafKey in (patchParent as Record<string, unknown>)) continue;
// What the read path served at this position: `undefined` for a dropped
// key, the rewritten string for a URL redaction. The patch speaks for the
// author exactly where it DIFFERS from that projection.
const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
const servedLeaf =
servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
? (servedParent as Record<string, unknown>)[leafKey]
: undefined;
if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
graftAt(out, path, storedLeaf);
}

Expand Down
6 changes: 6 additions & 0 deletions packages/spec/api-surface/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase (type)",
"BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode (type)",
"FeedItemType (type)",
"Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase (interface)",
"PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths (function)",
"renderAutonumber (function)",
Expand Down
6 changes: 6 additions & 0 deletions packages/spec/export-origins/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
"BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
"FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
"Field": "src/data/field.zod.ts#Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
"PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/nested-credential-redaction-class.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
---
'@objectstack/spec': minor
'@objectstack/service-datasource': patch
---

feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)

**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
credential under the very spelling the top level refuses and redacts — one
object level down (`options.auth.token`, `options.pool.password`, a
`tunnel.password` on a contract-less driver) — was accepted at publish and
served back by every datasource read door in cleartext with
`redactedConfigKeys: []`, because the read side's nested judgment was only the
hand-enumerated `passthroughSecretPaths` table and the write side had no nested
name judgment at all.

Both sides are now derived from ONE source instead of hand-maintained:

- The canonical credential spellings and former aliases moved to
`driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
driver-schema import graph — so the write door's passthrough walk and the
read redactor consume the same list (#8300's no-second-copy posture applied
to the list itself).
- **Read door** (`redactDatasourceConfig`, behind both consumers — the
datasource-admin routes and the kernel per-type redaction hook): the
credential-name judgment and the URL composite (userinfo + query params) now
run at EVERY object depth, for every driver, contract-less included. Nested
removals are reported as dotted paths in `redactedKeys`, plus a new
`redactedPaths` field carrying exact segments. `passthroughSecretPaths`
remains only as the residue it always should have been: CLIENT-MEASURED
secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
top-level key.
- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
credential-spelled key is refused at any object depth of the mongodb
`options` passthrough, with a prescription that does not inherit the
`auth.password`-only "wins over" reassurance. The measured `auth.password`
refusal keeps its own message; nothing is double-reported.
- **Schema derivation walked at depth**: `refusedCredentialPaths` /
`refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
top level, so a driver contract that refuses a key inside a nested object
shape is covered the day it lands (none exists today — pinned per driver).
- **Arrays are off the walk** on both doors — the same structural line
`valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
`initialData` seeds) keeps its own fields without a per-driver exclusion
list.
- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
redactor instead of mirroring it rule by rule: it grafts stored material
back wherever the patch is indistinguishable from what the read path served,
so an untouched "Save" on an affected legacy row keeps its stored material
for every current and future redaction source, and an author's edit always
wins. The metadata write door's generic `carryForwardRedactedValues` already
walks the dotted paths and needs no change.

Semantic migration entry
`datasource-config-options-nested-credential-spelling-refused` (protocol major
18) carries the authored-artifact upgrade: remove the nested key, or bind the
real secret through `external.credentialsRef` / the connection form.

<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ MongoDB Connection Configuration
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |


---
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
).rejects.toThrow(/options\.auth\.password/);
});
});

describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
/**
* The nested-position finding, at the admin door: a credential spelling one
* object level down from the key the top level hides — deliberately NOT a
* `passthroughSecretPaths` row — used to be served by `getDatasource()` in
* cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
* silence (measured on the pre-fix build). Regression cases against table
* rows were already green and prove nothing about this class.
*/
const OFF_TABLE_MONGO: StoredDatasource = {
name: 'off_table_mongo',
driver: 'mongodb',
origin: 'runtime',
config: {
database: 'events',
options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
},
};

it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
expect(read!.redactedConfigKeys).toContain('options.auth.token');
});

it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
const { service, records } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
expect(records[0].label).toBe('Renamed');
expect((records[0].config!.options as any).auth).toEqual({
username: 'app',
token: 'eyJhbGci.OFFTABLE.y',
});
});

it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
await expect(
service.updateDatasource('off_table_mongo', {
config: {
database: 'events',
options: { auth: { username: 'app', token: 'typed-new-secret' } },
},
}),
).rejects.toThrow(/options\.auth\.token/);
});

it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
const stored: StoredDatasource = {
name: 'vendor_ds',
driver: 'com.vendor.custom',
origin: 'runtime',
config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
};
const { service, records } = makeService([stored]);
const read = await service.getDatasource('vendor_ds');
expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
expect(read!.redactedConfigKeys).toContain('replication.url');
// Untouched round-trip: the served (redacted) URL is indistinguishable
// from the stored one once redacted, so the stored value is carried back.
await service.updateDatasource('vendor_ds', { config: read!.config });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
// An author who rewrites the nested URL by hand still WINS.
const edited = {
...read!.config,
replication: { url: 'postgresql://svc@other-replica/db' },
};
await service.updateDatasource('vendor_ds', { config: edited });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
});
});
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,11 +31,7 @@
* #8154's, deliberately not built here.
*/

import {
passthroughSecretPaths,
redactableConfigKeys,
redactUrlCredentials,
} from '@objectstack/spec/data';
import { redactDatasourceConfig } from '@objectstack/spec/data';

export {
refusedCredentialKeys,
Expand All@@ -56,6 +52,24 @@ export {
* where the patch is indistinguishable from what the read path served — an
* absent key, or a URL that matches the stored URL once redacted. Anything the
* author actually changed wins, including clearing a URL's password by hand.
* A patch whose CONTAINER for a nested leaf is removed is the author's word
* too (they deleted the block), so nothing is grafted there.
*
* ## Derived from the redactor, not restated beside it
*
* This function used to mirror the read path rule by rule — one loop per
* redaction source, each a copy that could silently fall behind (the docblock
* threat on every one of them: "a redaction the restore side did not mirror
* turns an untouched Save into silent credential deletion"). The nested-
* position fix made the read path recursive, which would have added two more
* loops — so the mirroring is now structural instead: compute what the read
* path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
* and for every redacted path graft the stored value back exactly where the
* patch still matches the served projection. A future redaction source is
* mirrored here by construction, with nothing to forget. (Same inversion the
* metadata door's generic `carryForwardRedactedValues` performs; this one
* consumes the redactor's exact `redactedPaths` segments, so a stored key
* with a literal dot cannot be mis-split.)
*
* What this does NOT do is let a patch set a refused key: `assertValidConfig`
* still runs on the merged record, so a caller that types `password` into the
Expand All@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
if (!patch || typeof patch !== 'object') return patch;
if (!stored || typeof stored !== 'object') return patch;

const hidden = new Set(redactableConfigKeys(driver));
const served = redactDatasourceConfig(driver, stored);
const out: Record<string, unknown> = { ...patch };

for (const key of hidden) {
// Only when the patch does not speak to the key at all. A patch that DOES
// carry it is the author's word, and (for a refused spelling) is about to
// be refused on its own merits rather than quietly overwritten here.
if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
}

for (const [key, storedValue] of Object.entries(stored)) {
if (hidden.has(key) || typeof storedValue !== 'string') continue;
// The SAME composite the read path applies (userinfo password + #8337
// credential query parameters) — a redaction this compare did not mirror
// would make the untouched "Save" it exists for delete the credential.
const redactedStored = redactUrlCredentials(storedValue);
// Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
if (redactedStored === storedValue) continue;
if (out[key] === redactedStored) out[key] = storedValue;
}

// The passthrough spellings (#9040) — the nested material the read path
// drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
// same narrow rule as the top-level keys, translated per leaf: restore ONLY
// when the patch's container for the leaf exists but does not speak to the
// leaf at all — exactly what the read path served. A patch carrying the leaf
// is the author's word (a typed-in `auth.password` is then refused by the
// #9040 write gate on its own merits); a patch with the CONTAINER removed is
// the author's word too (they deleted the block), so nothing is grafted.
for (const path of passthroughSecretPaths(driver)) {
for (const path of served.redactedPaths) {
const storedLeaf = valueAt(stored, path);
if (storedLeaf === undefined) continue;
const parentPath = path.slice(0, -1);
const leafKey = path[path.length - 1] as string;
const patchParent = valueAt(out, parentPath);
const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
if (leafKey in (patchParent as Record<string, unknown>)) continue;
// What the read path served at this position: `undefined` for a dropped
// key, the rewritten string for a URL redaction. The patch speaks for the
// author exactly where it DIFFERS from that projection.
const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
const servedLeaf =
servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
? (servedParent as Record<string, unknown>)[leafKey]
: undefined;
if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
graftAt(out, path, storedLeaf);
}

Expand Down
6 changes: 6 additions & 0 deletions packages/spec/api-surface/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase (type)",
"BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode (type)",
"FeedItemType (type)",
"Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase (interface)",
"PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths (function)",
"renderAutonumber (function)",
Expand Down
6 changes: 6 additions & 0 deletions packages/spec/export-origins/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
"BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
"FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
"Field": "src/data/field.zod.ts#Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
"PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/nested-credential-redaction-class.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
---
'@objectstack/spec': minor
'@objectstack/service-datasource': patch
---

feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)

**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
credential under the very spelling the top level refuses and redacts — one
object level down (`options.auth.token`, `options.pool.password`, a
`tunnel.password` on a contract-less driver) — was accepted at publish and
served back by every datasource read door in cleartext with
`redactedConfigKeys: []`, because the read side's nested judgment was only the
hand-enumerated `passthroughSecretPaths` table and the write side had no nested
name judgment at all.

Both sides are now derived from ONE source instead of hand-maintained:

- The canonical credential spellings and former aliases moved to
`driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
driver-schema import graph — so the write door's passthrough walk and the
read redactor consume the same list (#8300's no-second-copy posture applied
to the list itself).
- **Read door** (`redactDatasourceConfig`, behind both consumers — the
datasource-admin routes and the kernel per-type redaction hook): the
credential-name judgment and the URL composite (userinfo + query params) now
run at EVERY object depth, for every driver, contract-less included. Nested
removals are reported as dotted paths in `redactedKeys`, plus a new
`redactedPaths` field carrying exact segments. `passthroughSecretPaths`
remains only as the residue it always should have been: CLIENT-MEASURED
secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
top-level key.
- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
credential-spelled key is refused at any object depth of the mongodb
`options` passthrough, with a prescription that does not inherit the
`auth.password`-only "wins over" reassurance. The measured `auth.password`
refusal keeps its own message; nothing is double-reported.
- **Schema derivation walked at depth**: `refusedCredentialPaths` /
`refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
top level, so a driver contract that refuses a key inside a nested object
shape is covered the day it lands (none exists today — pinned per driver).
- **Arrays are off the walk** on both doors — the same structural line
`valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
`initialData` seeds) keeps its own fields without a per-driver exclusion
list.
- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
redactor instead of mirroring it rule by rule: it grafts stored material
back wherever the patch is indistinguishable from what the read path served,
so an untouched "Save" on an affected legacy row keeps its stored material
for every current and future redaction source, and an author's edit always
wins. The metadata write door's generic `carryForwardRedactedValues` already
walks the dotted paths and needs no change.

Semantic migration entry
`datasource-config-options-nested-credential-spelling-refused` (protocol major
18) carries the authored-artifact upgrade: remove the nested key, or bind the
real secret through `external.credentialsRef` / the connection form.

<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ MongoDB Connection Configuration
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |


---
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
).rejects.toThrow(/options\.auth\.password/);
});
});

describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
/**
* The nested-position finding, at the admin door: a credential spelling one
* object level down from the key the top level hides — deliberately NOT a
* `passthroughSecretPaths` row — used to be served by `getDatasource()` in
* cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
* silence (measured on the pre-fix build). Regression cases against table
* rows were already green and prove nothing about this class.
*/
const OFF_TABLE_MONGO: StoredDatasource = {
name: 'off_table_mongo',
driver: 'mongodb',
origin: 'runtime',
config: {
database: 'events',
options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
},
};

it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
expect(read!.redactedConfigKeys).toContain('options.auth.token');
});

it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
const { service, records } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
expect(records[0].label).toBe('Renamed');
expect((records[0].config!.options as any).auth).toEqual({
username: 'app',
token: 'eyJhbGci.OFFTABLE.y',
});
});

it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
await expect(
service.updateDatasource('off_table_mongo', {
config: {
database: 'events',
options: { auth: { username: 'app', token: 'typed-new-secret' } },
},
}),
).rejects.toThrow(/options\.auth\.token/);
});

it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
const stored: StoredDatasource = {
name: 'vendor_ds',
driver: 'com.vendor.custom',
origin: 'runtime',
config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
};
const { service, records } = makeService([stored]);
const read = await service.getDatasource('vendor_ds');
expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
expect(read!.redactedConfigKeys).toContain('replication.url');
// Untouched round-trip: the served (redacted) URL is indistinguishable
// from the stored one once redacted, so the stored value is carried back.
await service.updateDatasource('vendor_ds', { config: read!.config });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
// An author who rewrites the nested URL by hand still WINS.
const edited = {
...read!.config,
replication: { url: 'postgresql://svc@other-replica/db' },
};
await service.updateDatasource('vendor_ds', { config: edited });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
});
});
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,11 +31,7 @@
* #8154's, deliberately not built here.
*/

import {
passthroughSecretPaths,
redactableConfigKeys,
redactUrlCredentials,
} from '@objectstack/spec/data';
import { redactDatasourceConfig } from '@objectstack/spec/data';

export {
refusedCredentialKeys,
Expand All@@ -56,6 +52,24 @@ export {
* where the patch is indistinguishable from what the read path served — an
* absent key, or a URL that matches the stored URL once redacted. Anything the
* author actually changed wins, including clearing a URL's password by hand.
* A patch whose CONTAINER for a nested leaf is removed is the author's word
* too (they deleted the block), so nothing is grafted there.
*
* ## Derived from the redactor, not restated beside it
*
* This function used to mirror the read path rule by rule — one loop per
* redaction source, each a copy that could silently fall behind (the docblock
* threat on every one of them: "a redaction the restore side did not mirror
* turns an untouched Save into silent credential deletion"). The nested-
* position fix made the read path recursive, which would have added two more
* loops — so the mirroring is now structural instead: compute what the read
* path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
* and for every redacted path graft the stored value back exactly where the
* patch still matches the served projection. A future redaction source is
* mirrored here by construction, with nothing to forget. (Same inversion the
* metadata door's generic `carryForwardRedactedValues` performs; this one
* consumes the redactor's exact `redactedPaths` segments, so a stored key
* with a literal dot cannot be mis-split.)
*
* What this does NOT do is let a patch set a refused key: `assertValidConfig`
* still runs on the merged record, so a caller that types `password` into the
Expand All@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
if (!patch || typeof patch !== 'object') return patch;
if (!stored || typeof stored !== 'object') return patch;

const hidden = new Set(redactableConfigKeys(driver));
const served = redactDatasourceConfig(driver, stored);
const out: Record<string, unknown> = { ...patch };

for (const key of hidden) {
// Only when the patch does not speak to the key at all. A patch that DOES
// carry it is the author's word, and (for a refused spelling) is about to
// be refused on its own merits rather than quietly overwritten here.
if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
}

for (const [key, storedValue] of Object.entries(stored)) {
if (hidden.has(key) || typeof storedValue !== 'string') continue;
// The SAME composite the read path applies (userinfo password + #8337
// credential query parameters) — a redaction this compare did not mirror
// would make the untouched "Save" it exists for delete the credential.
const redactedStored = redactUrlCredentials(storedValue);
// Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
if (redactedStored === storedValue) continue;
if (out[key] === redactedStored) out[key] = storedValue;
}

// The passthrough spellings (#9040) — the nested material the read path
// drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
// same narrow rule as the top-level keys, translated per leaf: restore ONLY
// when the patch's container for the leaf exists but does not speak to the
// leaf at all — exactly what the read path served. A patch carrying the leaf
// is the author's word (a typed-in `auth.password` is then refused by the
// #9040 write gate on its own merits); a patch with the CONTAINER removed is
// the author's word too (they deleted the block), so nothing is grafted.
for (const path of passthroughSecretPaths(driver)) {
for (const path of served.redactedPaths) {
const storedLeaf = valueAt(stored, path);
if (storedLeaf === undefined) continue;
const parentPath = path.slice(0, -1);
const leafKey = path[path.length - 1] as string;
const patchParent = valueAt(out, parentPath);
const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
if (leafKey in (patchParent as Record<string, unknown>)) continue;
// What the read path served at this position: `undefined` for a dropped
// key, the rewritten string for a URL redaction. The patch speaks for the
// author exactly where it DIFFERS from that projection.
const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
const servedLeaf =
servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
? (servedParent as Record<string, unknown>)[leafKey]
: undefined;
if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
graftAt(out, path, storedLeaf);
}

Expand Down
6 changes: 6 additions & 0 deletions packages/spec/api-surface/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase (type)",
"BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode (type)",
"FeedItemType (type)",
"Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase (interface)",
"PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths (function)",
"renderAutonumber (function)",
Expand Down
6 changes: 6 additions & 0 deletions packages/spec/export-origins/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
"BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
"FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
"Field": "src/data/field.zod.ts#Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
"PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .changeset/nested-credential-redaction-class.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
---
'@objectstack/spec': minor
'@objectstack/service-datasource': patch
---

feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)

**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
credential under the very spelling the top level refuses and redacts — one
object level down (`options.auth.token`, `options.pool.password`, a
`tunnel.password` on a contract-less driver) — was accepted at publish and
served back by every datasource read door in cleartext with
`redactedConfigKeys: []`, because the read side's nested judgment was only the
hand-enumerated `passthroughSecretPaths` table and the write side had no nested
name judgment at all.

Both sides are now derived from ONE source instead of hand-maintained:

- The canonical credential spellings and former aliases moved to
`driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
driver-schema import graph — so the write door's passthrough walk and the
read redactor consume the same list (#8300's no-second-copy posture applied
to the list itself).
- **Read door** (`redactDatasourceConfig`, behind both consumers — the
datasource-admin routes and the kernel per-type redaction hook): the
credential-name judgment and the URL composite (userinfo + query params) now
run at EVERY object depth, for every driver, contract-less included. Nested
removals are reported as dotted paths in `redactedKeys`, plus a new
`redactedPaths` field carrying exact segments. `passthroughSecretPaths`
remains only as the residue it always should have been: CLIENT-MEASURED
secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
top-level key.
- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
credential-spelled key is refused at any object depth of the mongodb
`options` passthrough, with a prescription that does not inherit the
`auth.password`-only "wins over" reassurance. The measured `auth.password`
refusal keeps its own message; nothing is double-reported.
- **Schema derivation walked at depth**: `refusedCredentialPaths` /
`refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
top level, so a driver contract that refuses a key inside a nested object
shape is covered the day it lands (none exists today — pinned per driver).
- **Arrays are off the walk** on both doors — the same structural line
`valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
`initialData` seeds) keeps its own fields without a per-driver exclusion
list.
- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
redactor instead of mirroring it rule by rule: it grafts stored material
back wherever the patch is indistinguishable from what the read path served,
so an untouched "Save" on an affected legacy row keeps its stored material
for every current and future redaction source, and an author's edit always
wins. The metadata write door's generic `carryForwardRedactedValues` already
walks the dotted paths and needs no change.

Semantic migration entry
`datasource-config-options-nested-credential-spelling-refused` (protocol major
18) carries the authored-artifact upgrade: remove the nested key, or bind the
real secret through `external.credentialsRef` / the connection form.

<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ MongoDB Connection Configuration
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |


---
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
).rejects.toThrow(/options\.auth\.password/);
});
});

describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
/**
* The nested-position finding, at the admin door: a credential spelling one
* object level down from the key the top level hides — deliberately NOT a
* `passthroughSecretPaths` row — used to be served by `getDatasource()` in
* cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
* silence (measured on the pre-fix build). Regression cases against table
* rows were already green and prove nothing about this class.
*/
const OFF_TABLE_MONGO: StoredDatasource = {
name: 'off_table_mongo',
driver: 'mongodb',
origin: 'runtime',
config: {
database: 'events',
options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
},
};

it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
expect(read!.redactedConfigKeys).toContain('options.auth.token');
});

it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
const { service, records } = makeService([OFF_TABLE_MONGO]);
const read = await service.getDatasource('off_table_mongo');
await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
expect(records[0].label).toBe('Renamed');
expect((records[0].config!.options as any).auth).toEqual({
username: 'app',
token: 'eyJhbGci.OFFTABLE.y',
});
});

it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
const { service } = makeService([OFF_TABLE_MONGO]);
await expect(
service.updateDatasource('off_table_mongo', {
config: {
database: 'events',
options: { auth: { username: 'app', token: 'typed-new-secret' } },
},
}),
).rejects.toThrow(/options\.auth\.token/);
});

it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
const stored: StoredDatasource = {
name: 'vendor_ds',
driver: 'com.vendor.custom',
origin: 'runtime',
config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
};
const { service, records } = makeService([stored]);
const read = await service.getDatasource('vendor_ds');
expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
expect(read!.redactedConfigKeys).toContain('replication.url');
// Untouched round-trip: the served (redacted) URL is indistinguishable
// from the stored one once redacted, so the stored value is carried back.
await service.updateDatasource('vendor_ds', { config: read!.config });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
// An author who rewrites the nested URL by hand still WINS.
const edited = {
...read!.config,
replication: { url: 'postgresql://svc@other-replica/db' },
};
await service.updateDatasource('vendor_ds', { config: edited });
expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
});
});
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,11 +31,7 @@
* #8154's, deliberately not built here.
*/

import {
passthroughSecretPaths,
redactableConfigKeys,
redactUrlCredentials,
} from '@objectstack/spec/data';
import { redactDatasourceConfig } from '@objectstack/spec/data';

export {
refusedCredentialKeys,
Expand All@@ -56,6 +52,24 @@ export {
* where the patch is indistinguishable from what the read path served — an
* absent key, or a URL that matches the stored URL once redacted. Anything the
* author actually changed wins, including clearing a URL's password by hand.
* A patch whose CONTAINER for a nested leaf is removed is the author's word
* too (they deleted the block), so nothing is grafted there.
*
* ## Derived from the redactor, not restated beside it
*
* This function used to mirror the read path rule by rule — one loop per
* redaction source, each a copy that could silently fall behind (the docblock
* threat on every one of them: "a redaction the restore side did not mirror
* turns an untouched Save into silent credential deletion"). The nested-
* position fix made the read path recursive, which would have added two more
* loops — so the mirroring is now structural instead: compute what the read
* path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
* and for every redacted path graft the stored value back exactly where the
* patch still matches the served projection. A future redaction source is
* mirrored here by construction, with nothing to forget. (Same inversion the
* metadata door's generic `carryForwardRedactedValues` performs; this one
* consumes the redactor's exact `redactedPaths` segments, so a stored key
* with a literal dot cannot be mis-split.)
*
* What this does NOT do is let a patch set a refused key: `assertValidConfig`
* still runs on the merged record, so a caller that types `password` into the
Expand All@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
if (!patch || typeof patch !== 'object') return patch;
if (!stored || typeof stored !== 'object') return patch;

const hidden = new Set(redactableConfigKeys(driver));
const served = redactDatasourceConfig(driver, stored);
const out: Record<string, unknown> = { ...patch };

for (const key of hidden) {
// Only when the patch does not speak to the key at all. A patch that DOES
// carry it is the author's word, and (for a refused spelling) is about to
// be refused on its own merits rather than quietly overwritten here.
if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
}

for (const [key, storedValue] of Object.entries(stored)) {
if (hidden.has(key) || typeof storedValue !== 'string') continue;
// The SAME composite the read path applies (userinfo password + #8337
// credential query parameters) — a redaction this compare did not mirror
// would make the untouched "Save" it exists for delete the credential.
const redactedStored = redactUrlCredentials(storedValue);
// Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
if (redactedStored === storedValue) continue;
if (out[key] === redactedStored) out[key] = storedValue;
}

// The passthrough spellings (#9040) — the nested material the read path
// drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
// same narrow rule as the top-level keys, translated per leaf: restore ONLY
// when the patch's container for the leaf exists but does not speak to the
// leaf at all — exactly what the read path served. A patch carrying the leaf
// is the author's word (a typed-in `auth.password` is then refused by the
// #9040 write gate on its own merits); a patch with the CONTAINER removed is
// the author's word too (they deleted the block), so nothing is grafted.
for (const path of passthroughSecretPaths(driver)) {
for (const path of served.redactedPaths) {
const storedLeaf = valueAt(stored, path);
if (storedLeaf === undefined) continue;
const parentPath = path.slice(0, -1);
const leafKey = path[path.length - 1] as string;
const patchParent = valueAt(out, parentPath);
const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
if (leafKey in (patchParent as Record<string, unknown>)) continue;
// What the read path served at this position: `undefined` for a dropped
// key, the rewritten string for a URL redaction. The patch speaks for the
// author exactly where it DIFFERS from that projection.
const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
const servedLeaf =
servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
? (servedParent as Record<string, unknown>)[leafKey]
: undefined;
if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
graftAt(out, path, storedLeaf);
}

Expand Down
6 changes: 6 additions & 0 deletions packages/spec/api-surface/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase (type)",
"BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode (type)",
"FeedItemType (type)",
"Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase (interface)",
"PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf (function)",
"referencedFields (function)",
"refusedCredentialKeys (function)",
"refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths (function)",
"renderAutonumber (function)",
Expand Down
6 changes: 6 additions & 0 deletions packages/spec/export-origins/data.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -56,12 +56,14 @@
"BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
"BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
"CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
"CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
"CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
"COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
"CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
"CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
"CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
"CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
"CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
"CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
"CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
"CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
Expand DownExpand Up@@ -258,6 +260,7 @@
"FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
"FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
"FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
"FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
"FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
"FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
"Field": "src/data/field.zod.ts#Field (type)",
Expand DownExpand Up@@ -452,6 +455,7 @@
"PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
"PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
"PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
"PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
"PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
"PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
"PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
Expand DownExpand Up@@ -750,6 +754,8 @@
"referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
"referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
"refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
"refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
"refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
"refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
"refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
"renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",
Expand Down
Loading
Loading