Skip to content

feat(spec): IDataEngine datasource-def contract catches up to the engine — external.credentialsRef accepted, listDatasourceDefs declared - #12949

Merged
os-trump merged 3 commits into
mainfrom
claude/issue-12805-datasource-contract-catchup
Aug 28, 2026
Merged

feat(spec): IDataEngine datasource-def contract catches up to the engine — external.credentialsRef accepted, listDatasourceDefs declared#12949
os-trump merged 3 commits into
mainfrom
claude/issue-12805-datasource-contract-catchup

Conversation

@os-trump

Copy link
Copy Markdown
Collaborator

Fixes#12805

What

The published IDataEngine contract catches up to declared-and-enforced engine reality (#12758 remains the engine-side change this follows):

  1. registerDatasourceDef now accepts external.credentialsRef — the secrets-store handle ObjectQL has accepted and retained since Widen registerDatasourceDef to retain external.credentialsRef, so the sys_secret reference union can answer family 3 without the caller — schedule BEFORE #8103's deletion half #12758. Before this, a caller typed against the published contract was refused with TS2353 at the consumer seam for a value the runtime keeps.
  2. New optional member listDatasourceDefs returning EngineDatasourceDef[] — the read-back of the same registry, the fourth datasource-lifecycle member under the 2026-08-25 finding: two consumer-local structural DataEngineLike re-declarations remain after the #11493 deletion (datasource-admin-plugin, service-analytics) #11833 ruling's item-4 precedent (which adjudicated the first three onto this contract in the change for IDataEngine contract adoption ruled by #11833: declare resolveEffectiveDatasource + getDriverForObject (optional members), and give getObject a real return contract #12248).
  3. Both members share the newly exported EngineDatasourceDef interface. Both stay OPTIONAL — only engines that own a datasource registry answer.

Regenerated artifacts: api-surface/contracts.json and export-origins/contracts.json (one added line each: the new interface). Changeset: minor for @objectstack/spec, following the additive-public-surface precedent (root-entry-nameability-data-delta.md).

Judgment calls (the three the triage grading named)

  1. listDatasourceDefs lands on IDataEngine, not IObjectQLEngine — follows the ruling's item-4 precedent rather than reopening it. The member meets the same evidence bar as the three already there: implemented on ObjectQL (engine.ts, listDatasourceDefs, shipped by Widen registerDatasourceDef to retain external.credentialsRef, so the sys_secret reference union can answer family 3 without the caller — schedule BEFORE #8103's deletion half #12758), cross-package pull exists (Wire the sys_secret reference union to self-serve family 3 from ObjectQL.listDatasourceDefs(), and correct the three prose sites #12758 falsified #12804 is the consumer: a sys_secret reference sweep must ask the engine which code-declared datasources hold a handle, since code-declared datasources never reach the metadata store), and without the declared member that consumer must name the engine class concretely or re-declare a consumer-local structural type — the pattern the finding: two consumer-local structural DataEngineLike re-declarations remain after the #11493 deletion (datasource-admin-plugin, service-analytics) #11833 sweep retired. Optionality preserved: same population as registerDatasourceDef ("only engines that own a datasource registry answer"). My measurement agrees with the precedent; no fork to report.
  2. Return element type: a named exported EngineDatasourceDef in the contract file, used by BOTH the parameter and the return element — not two inline literals. Evidence: the engine itself names DatasourceDef because "three copies of one shape is a second de-facto contract that drifts silently" (engine.ts docblock); at the contract level two members now share the shape, and an unnamed element type would push Wire the sys_secret reference union to self-serve family 3 from ObjectQL.listDatasourceDefs(), and correct the three prose sites #12758 falsified #12804's sweep back into indexed-access gymnastics or a consumer-local structural re-declaration — the exact retired pattern. The name follows the contract's existing Engine-prefixed family (EngineQueryOptions, EngineUpdateOptions). It is deliberately NOT named DatasourceDef: @objectstack/objectql exports a structurally identical type of that name which this contract cannot import (dependency direction); the distinct name keeps the two declarations tellable apart until the engine converges on the contract's. The keys mirror exactly what the engine retains (name, schemaMode, external.allowWrites, external.credentialsRef) — a deliberate SUBSET of ExternalDatasourceSettingsSchema, not invention (its docblock says so and forbids growing it).
  3. The pin (data-engine.test.ts) grows its positive case AND gains negatives, in this PR: the positive arm now registers a def carrying credentialsRef; the existing name-required negative stays; a NEW negative pins that the widening admits exactly the ruled key (an inline credentials key is still refused) — matching the block's own both-directions convention (the markDatasourceUnavailable kind-union case). New cases pin: all FOUR members optional; register parameter and list return share ONE declaration (mutual-extends against EngineDatasourceDef); the Wire the sys_secret reference union to self-serve family 3 from ObjectQL.listDatasourceDefs(), and correct the three prose sites #12758 falsified #12804 sweep expressible through the data-slot contract type alone; implementations answering nameless or inline-credential defs refused.

Verification — the consumer seam, both directions

The engine's typecheck is green either way (method-parameter bivariance), so it is NOT offered as proof — it was run as corroboration only (@objectstack/objectql typecheck exit 0; datasource-def-credentials-ref.test.ts 9/9 passed).

The proof is a compile probe typed against the published contract (resolved through the package exports map to the built dist declarations), run at both endpoints:

  • BEFORE (dist built at BASE 750fff5): probe A — a fresh literal carrying external.credentialsRef passed through registerDatasourceDef on an IDataEngine-typed value → error TS2353: 'credentialsRef' does not exist in type '{ allowWrites?: boolean | undefined; }'. Probe B — listDatasourceDefs called through the data-slot contract type → error TS2551: Property 'listDatasourceDefs' does not exist on type 'IDataEngine'.
  • AFTER (dist rebuilt from this branch, build-input-hash changed 038b95c8… → 688ef4a9…): the same probe file compiles with zero diagnostics (tsc exit 0).
  • Negative leg on the rebuilt dist (proves the probe read the NEW declarations, and that the widening is exactly one key): an undeclared inline credentials key → error TS2561 … does not exist in type '{ allowWrites?: boolean | undefined; credentialsRef?: string | undefined; }' — the printed target type itself names the widened member.

Spec tree at the probe head is byte-identical to this PR's final head (the merge with current main brought no packages/spec/src change), so the probe evidence stands at 6c7577623.

Gates (local, at final head 6c75776)

  • pnpm --filter @objectstack/spec build · typecheck (incl. check:test-typecheck: OK) · targeted vitest src/contracts/data-engine.test.ts + src/contracts/core-service-contracts.test.ts: 2 files, 39 tests passed. Both edited files confirmed inside the compiled tsc programs via listFiles (the new expect-error pins are live, not phantom).
  • check:generated: 2 artifacts proved stale (api-surface, export-origins), regenerated with --fix, re-checked clean.
  • Gate union re-derived with scripts/pm/dispatch-gates.mjs (no hand-fed paths) and run: check:authorable-surfacecheck:docscheck:livenesscheck:strictness-ledgercheck:empty-statecheck:variant-docscheck:skill-refscheck:exported-anycheck:dual-source-exportscheck:doc-authoringcheck:spec-parsed-aliascheck:changeset-gate-self-testscheck:cross-package-test-inputscheck:merge-drivercheck:objectql-double-limitcheck:page-declaration-shapecheck:published-filescheck:slot-lookupcheck:test-source-aliascheck:type-source-resolutioncheck:query-options-erasurecheck:engine-double-contractcheck:where-matchercheck:objectui-changesetcheck:pm-half-statescheck:type-check-coveragecheck:nul-bytescheck:dev-prereqscheck:doc-formula-expressionscheck:type-check-debt (--re-measure verdict: "31 ledger entries re-measured … none above its recorded number") plus the adr-0087 / changeset-no-major / empty-changeset / ci-filter-parity / comment-mask-adoption / plugin-teardown-shape / release-rehearsal-clone self-test node scripts — all exit 0, statuses captured before any pipe. scripts/pm/check-half-states.mjs exits 3 (no GitHub credential in this container — its own output states "no reading at all"; a board patrol, not a tree gate).

Scope notes

Generated by Claude Code


Generated by Claude Code

…ine — external.credentialsRef accepted, listDatasourceDefs declared
Part of #12805 (report and regenerated artifacts follow).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LpRNHxWZgSUgVnFT9mQQo4
…sourceDef; add changeset
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LpRNHxWZgSUgVnFT9mQQo4
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec, touching 5 documentable anchor(s). ⚠️2 changed file(s) yielded no anchor (packages/spec/api-surface/contracts.json, packages/spec/export-origins/contracts.json), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

12 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/ai/actions-as-tools.mdx(via IDataEngine (symbol))
  • content/docs/ai/knowledge-rag.mdx(via IDataEngine (symbol))
  • content/docs/data-modeling/drivers.mdx(via schemaMode (symbol))
  • content/docs/data-modeling/external-datasources.mdx(via schemaMode (symbol))
  • content/docs/kernel/contracts/data-engine.mdx(via IDataEngine (symbol))
  • content/docs/kernel/contracts/index.mdx(via IDataEngine (symbol))
  • content/docs/kernel/index.mdx(via IDataEngine (symbol))
  • content/docs/kernel/runtime-services/data-service.mdx(via IDataEngine (symbol))
  • content/docs/kernel/services-checklist.mdx(via IDataEngine (symbol))
  • content/docs/kernel/services.mdx(via IDataEngine (symbol))
  • content/docs/protocol/knowledge.mdx(via IDataEngine (symbol))
  • content/docs/protocol/objectql/query-syntax.mdx(via IDataEngine (symbol))

2 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/implementation-status.mdx(via IDataEngine (symbol))
  • content/docs/releases/v17.mdx(via IDataEngine (symbol), schemaMode (symbol))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 2 changed file(s) yielded no anchor (packages/spec/api-surface/contracts.json, packages/spec/export-origins/contracts.json) — pages documenting those are invisible to this run
  • 1 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 126 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 8beb3deaf6755311ed6ef88dcce4b5dcbebbdac3packageMentionDocs.

Which tree this was computed on

This run read content/docs from 86bfd8ea369f6fddd15e480004a58937d0fd0b13 — the merge of head 6c7577623edd7c8d3eb44b6341df2fa28721fd6b into base 8beb3deaf6755311ed6ef88dcce4b5dcbebbdac3, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 86bfd8ea369f6fddd15e480004a58937d0fd0b13 && git checkout 86bfd8ea369f6fddd15e480004a58937d0fd0b13
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 8beb3deaf6755311ed6ef88dcce4b5dcbebbdac3 6c7577623edd7c8d3eb44b6341df2fa28721fd6b && git checkout -B drift-repro 8beb3deaf6755311ed6ef88dcce4b5dcbebbdac3 && git merge --no-ff 6c7577623edd7c8d3eb44b6341df2fa28721fd6b
node scripts/docs-audit/affected-docs.mjs --json 8beb3deaf6755311ed6ef88dcce4b5dcbebbdac3

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 8beb3deaf6755311ed6ef88dcce4b5dcbebbdac3 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation tests tooling labels Aug 28, 2026
@os-trump
os-trump marked this pull request as ready for review August 28, 2026 12:53
@os-trump
os-trump enabled auto-merge August 28, 2026 12:53
@os-trump
os-trump added this pull request to the merge queueAug 28, 2026
os-trump pushed a commit that referenced this pull request Aug 28, 2026
Patch round on #12414/PR #12899: CI flagged check:export-origins STALE
on ai.json/kernel.json/root.json/ui.json -- the seven re-exports this
branch adds regenerated api-surface but not the sibling export-origins
artifact when the branch was first merged with origin/main (the #12949
precedent this diff follows regenerates both).
Root cause, precisely: merging origin/main (which had since landed
#12718, retiring PreviewModeConfig) picked up main's source changes but
the two GENERATED artifacts derived from that merged source were never
rebuilt -- kernel.json's api-surface still listed the retired
PreviewModeConfig/PreviewModeConfigParsed/PreviewModeConfigSchema, and
export-origins hadn't gained entries for this branch's seven new
re-exports at all.
Fix: pnpm --filter @objectstack/spec build (fresh dist reflecting both
this branch's re-exports and #12718's retirement), then
gen:export-origins + gen:api-surface. Read the resulting diff per the
gate's own #4411 dual-source-trap remedy: each of the seven names'
newly-gained export-origins entry was cross-checked against its origin
on every sibling entry that already carried it (root vs /system vs /ui,
kernel vs /data, /ai and /ui vs /data and /automation) -- all match
byte-for-byte, so this is the harmless re-export shape, never dual-source.
kernel.json's api-surface diff is exactly the PreviewModeConfig removal
#12718 made; nothing from this branch's own repair moved.
Verified: check:export-origins, check:dual-source-exports and
check:generated (14/14 generated artifacts) all green on the merged +
rebuilt tree.
Part of #12414
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LpRNHxWZgSUgVnFT9mQQo4
Merged via the queue into main with commit 4f6325dAug 28, 2026
41 checks passed
@os-trump
os-trump deleted the claude/issue-12805-datasource-contract-catchup branch August 28, 2026 13:52
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/mteststooling

Projects

None yet

2 participants

@os-trump@claude