feat(service-datasource): external-datasource driver return contract fails compile-time on mis-shaped drivers (#11123 option C)

.changeset/datasource-driver-handle-introspection-contract.md

@@ -0,0 +1,38 @@
+---
+"@objectstack/service-datasource": minor
+---
+
+feat(service-datasource): `DatasourceDriverHandle.introspectSchema` declares the spec introspection contract, so a mis-shaped custom driver fails to compile naming the wrong field (#11381, option C of the #11123 ruling)
+
+**BREAKING** for TypeScript hosts that build custom external-datasource
+drivers, shipped as `minor` under the repo's launch-window convention for
+breaking changes.
+
+`DatasourceDriverHandle.introspectSchema` — the seam every host-built driver
+crosses, since the framework deliberately ships no driver-by-id registry —
+was typed `Promise<unknown>`. The `isPrimary` → `primaryKey` retirement
+(#11124, shipped in 17.2.0) named the compiler as the channel that reaches
+every affected consumer, but against an `unknown` return that channel
+provably never fired: a host driver spelling the per-column primary-key flag
+`isPrimary`, or returning `{ tables }` with no `dialect`/`introspectedAt`,
+compiled clean, and the mis-shape surfaced only as a federated table whose
+records silently could not be located or updated.
+
+The member now declares `Promise<IntrospectedSchema>` — the one introspection
+contract in `packages/spec` (`contracts/schema-diff-service.ts`). A
+mis-shaped driver is refused at compile time, at the offending field:
+`Property 'primaryKey' is missing in type '…' but required in type
+'IntrospectedColumn'`, and on a fresh literal additionally `'isPrimary' does
+not exist in type 'IntrospectedColumn'`. A driver that already returns the
+spec shape — or a richer declared type extending it, the driver-sql /
+objectql pattern (table-level `primaryKeys`, per-column `maxLength`) —
+compiles unchanged.
+
+Runtime behaviour does not change. The `primaryKeyReader` compatibility belt
+in `ExternalDatasourceService` keeps absorbing the retired spelling from
+producers no compiler reaches (drivers already built against older versions,
+plain-JS drivers, casts). Removing that belt is #11123 option B — a later,
+separate step gated on this tightening being released and the retirement
+being published — and is not part of this change.
+
+<!-- adr-0087: not-required (runtime-interface-only packages/services/service-datasource/src/contracts/datasource-driver-factory.ts#DatasourceDriverHandle) the tightened member is a published runtime TypeScript interface describing a driver handle's capability surface — not a metadata surface. There is no Zod schema, no `packages/spec` declaration of the old `Promise<unknown>` signature, and no stored representation of a driver handle, so `objectstack migrate meta` has nothing to rewrite; the channel that reaches every affected consumer is the compiler, precisely and at every site. -->

packages/services/service-datasource/src/__tests__/datasource-driver-handle-contract.test.ts

@@ -0,0 +1,199 @@
+// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * DatasourceDriverHandle.introspectSchema — the return contract is the spec
+ * introspection shape, enforced by tsc (#11381, option C of the #11123
+ * ruling, recorded 2026-08-23).
+ *
+ * Until #11381 the member was typed `Promise<unknown>`, so the one channel
+ * the `isPrimary` → `primaryKey` retirement (#11124) named as its migration
+ * path — the compiler, "precisely and at every site" — provably never fired
+ * for the seam's OPEN producer population: host-built drivers
+ * (`datasource-driver-factory.ts` deliberately ships no driver registry). A
+ * driver spelling the primary-key flag wrong compiled clean and produced a
+ * federated table whose records silently could not be located or updated.
+ *
+ * Every mis-shape pin below is resolved by tsc, not by vitest: reverting the
+ * tightening (`introspectSchema?(): Promise<IntrospectedSchema>` back to
+ * `Promise<unknown>`) makes each `@ts-expect-error` directive unused, and an
+ * unused directive is itself an error, so
+ * `pnpm --filter @objectstack/service-datasource typecheck` goes red — and
+ * the first pin asserts the member's exact type, which reds directly on the
+ * same revert. This package carries no test-typecheck-debt ledger entry, so
+ * zero errors is the measured baseline these pins move away from. The
+ * `expect()` calls only give the assertions a home vitest will run.
+ *
+ * What these pins deliberately do NOT claim: any runtime behaviour change.
+ * The `primaryKeyReader` compatibility belt in
+ * `external-datasource-service.ts` still absorbs the retired spelling from
+ * producers no compiler reaches (already-built drivers, plain JS, casts);
+ * its own suite pins that. Removing the belt is #11123 option B, gated on
+ * this tightening being released AND the retirement being published.
+ */
+
+import { describe, expect, expectTypeOf, it } from 'vitest';
+import type { DatasourceDriverHandle } from '../contracts/datasource-driver-factory.js';
+import type {
+ IntrospectedColumn,
+ IntrospectedSchema,
+ IntrospectedTable,
+} from '@objectstack/spec/contracts';
+
+/** A well-formed schema in the spec spelling, reused by the green cases. */
+function specShapedSchema(): IntrospectedSchema {
+ return {
+ tables: {
+ customers: {
+ name: 'customers',
+ columns: [
+ { name: 'id', type: 'varchar', nullable: false, primaryKey: true },
+ { name: 'email', type: 'varchar', nullable: true, primaryKey: false },
+ ],
+ },
+ },
+ dialect: 'sqlite',
+ introspectedAt: '2026-08-23T00:00:00.000Z',
+ };
+}
+
+describe('DatasourceDriverHandle.introspectSchema return contract (#11381)', () => {
+ it('declares exactly the spec introspection shape — not unknown', () => {
+ // The strongest pin in the file: this line reads the member's type off the
+ // CONTRACT and reds directly if the tightening is reverted to
+ // `Promise<unknown>` (or widened to any other spelling), independently of
+ // every `@ts-expect-error` below.
+ expectTypeOf<
+ ReturnType<NonNullable<DatasourceDriverHandle['introspectSchema']>>
+ >().toEqualTypeOf<Promise<IntrospectedSchema>>();
+ });
+
+ it('accepts a correctly-shaped driver, written in the spec spelling', async () => {
+ // A host-built handle authored fresh against this version: per-column
+ // `primaryKey`, schema-level `dialect` + `introspectedAt`. This literal
+ // failing to compile is the regression.
+ const handle: DatasourceDriverHandle = {
+ introspectSchema: async () => specShapedSchema(),
+ };
+ const schema = await handle.introspectSchema!();
+ expect(schema.tables.customers.columns[0].primaryKey).toBe(true);
+ });
+
+ it('gives the consumer a typed result — the probe read is no longer unknown', async () => {
+ const handle: DatasourceDriverHandle = {
+ introspectSchema: async () => specShapedSchema(),
+ };
+ // Before #11381 this expression was `unknown` and every consumer either
+ // cast or re-narrowed; now `dialect` reads straight off the declared type.
+ expectTypeOf(handle.introspectSchema!()).resolves.toEqualTypeOf<IntrospectedSchema>();
+ expect((await handle.introspectSchema!()).dialect).toBe('sqlite');
+ });
+
+ it('still accepts a RICHER driver whose declared type extends the spec contract', async () => {
+ // The in-tree pattern: driver-sql / objectql declare their introspection
+ // types as EXTENSIONS of the spec contract (table-level `primaryKeys`,
+ // per-column `maxLength`, …). Return-type covariance admits those extras
+ // on any non-literal value, so such a driver needs no edit — and the
+ // `primaryKeyReader` belt keeps reading `table.primaryKeys` as the one
+ // carrier of composite-key ORDER.
+ interface RicherColumn extends IntrospectedColumn {
+ maxLength?: number | string;
+ }
+ interface RicherTable extends IntrospectedTable {
+ columns: RicherColumn[];
+ primaryKeys: string[];
+ }
+ interface RicherSchema extends IntrospectedSchema {
+ tables: Record<string, RicherTable>;
+ }
+ const richer = async (): Promise<RicherSchema> => ({
+ tables: {
+ orders: {
+ name: 'orders',
+ columns: [
+ { name: 'order_id', type: 'varchar', nullable: false, primaryKey: true, maxLength: 36 },
+ { name: 'line_no', type: 'integer', nullable: false, primaryKey: true },
+ ],
+ primaryKeys: ['order_id', 'line_no'],
+ },
+ },
+ dialect: 'postgres',
+ introspectedAt: '2026-08-23T00:00:00.000Z',
+ });
+ const handle: DatasourceDriverHandle = { introspectSchema: richer };
+ const schema = await handle.introspectSchema!();
+ expect(schema.tables.orders.columns).toHaveLength(2);
+ });
+
+ it('refuses the retired `isPrimary` spelling, naming the field', () => {
+ // The founding defect (#10676 → #11001 → #11123): a driver spelling the
+ // per-column flag `isPrimary`. tsc's rejection names the field on both
+ // instruments — the missing required key ("Property 'primaryKey' is
+ // missing in type … but required in type 'IntrospectedColumn'") and, on a
+ // fresh literal, the excess key ("'isPrimary' does not exist in type
+ // 'IntrospectedColumn'").
+ const misSpelledColumn: IntrospectedColumn = {
+ name: 'id',
+ type: 'varchar',
+ nullable: false,
+ // @ts-expect-error - the spec spelling is `primaryKey`; `isPrimary` does not exist in type 'IntrospectedColumn'
+ isPrimary: true,
+ };
+ expect(misSpelledColumn).toBeTruthy();
+
+ // The same mistake a whole driver up: pre-#11381 this handle compiled
+ // clean and lost the remote key at runtime; now it does not compile.
+ const legacyDriver = async () => ({
+ tables: {
+ customers: {
+ name: 'customers',
+ columns: [{ name: 'id', type: 'varchar', nullable: false, isPrimary: true }],
+ },
+ },
+ dialect: 'sqlite',
+ introspectedAt: '2026-08-23T00:00:00.000Z',
+ });
+ // @ts-expect-error - a driver whose columns spell the flag `isPrimary` no longer satisfies the handle
+ const misSpelledHandle: DatasourceDriverHandle = { introspectSchema: legacyDriver };
+ expect(misSpelledHandle).toBeTruthy();
+ });
+
+ it('refuses the `{ tables }`-only schema the driver actually shipped (#10998)', () => {
+ // Measured on the pre-retirement driver: `Object.keys()` of the schema
+ // was `["tables"]` — no `dialect`, no `introspectedAt` — so type mapping
+ // ran dialect-blind across the whole federation path. tsc now names both
+ // missing fields.
+ const bareTables = async () => ({
+ tables: {
+ customers: {
+ name: 'customers',
+ columns: [{ name: 'id', type: 'varchar', nullable: false, primaryKey: true }],
+ },
+ },
+ });
+ // @ts-expect-error - `dialect` and `introspectedAt` are required by the spec introspection contract
+ const tablesOnlyHandle: DatasourceDriverHandle = { introspectSchema: bareTables };
+ expect(tablesOnlyHandle).toBeTruthy();
+ });
+
+ it('refuses a table-level `primaryKeys` list written fresh against the BARE spec shape', () => {
+ // On the spec contract the primary-key fact is per-column. A table-level
+ // list is a driver-sql EXTENSION — legal on a declared extending type
+ // (previous green case), refused as an excess key on a fresh literal of
+ // the bare contract, so an author who reaches for the extension is told
+ // to declare it rather than having half a spelling absorbed silently.
+ const listOnlyTable: IntrospectedTable = {
+ name: 'customers',
+ columns: [{ name: 'id', type: 'varchar', nullable: false, primaryKey: false }],
+ // @ts-expect-error - `primaryKeys` does not exist in type 'IntrospectedTable'; declare an extending type or spell the fact per-column
+ primaryKeys: ['id'],
+ };
+ expect(listOnlyTable).toBeTruthy();
+ });
+
+ it('refuses a driver that only promises `unknown` — the pre-#11381 signature itself', () => {
+ const opaque = async (): Promise<unknown> => JSON.parse('{}');
+ // @ts-expect-error - `Promise<unknown>` is exactly the contract this member no longer has
+ const opaqueHandle: DatasourceDriverHandle = { introspectSchema: opaque };
+ expect(opaqueHandle).toBeTruthy();
+ });
+});

packages/services/service-datasource/src/contracts/datasource-driver-factory.ts

@@ -1,5 +1,7 @@
 // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
 
+import type { IntrospectedSchema } from '@objectstack/spec/contracts';
+
 /**
  * IDatasourceDriverFactory — host-provided capability that builds a live driver
  * from a connection spec (ADR-0015 Addendum §3.5).
@@ -76,6 +78,10 @@ export type DatasourceDriverOwnership = 'factory' | 'host';
  * A live (or lazily-connecting) driver handle. Intentionally structural and
  * fully optional so any concrete driver satisfies it — the admin service uses
  * whatever capabilities are present and skips the rest.
+ *
+ * "Fully optional" is about PRESENCE, not about shape: a member the driver
+ * does expose must honour that member's declared contract. The one place the
+ * distinction has already cost data is `introspectSchema` — see its own note.
  */
 export interface DatasourceDriverHandle {
  /** Open the connection / pool. */
@@ -86,8 +92,38 @@ export interface DatasourceDriverHandle {
  ownership?: DatasourceDriverOwnership;
  /** Cheap liveness round-trip (preferred for probes). */
  ping?(): Promise<unknown>;
- /** Introspect the live schema (fallback probe when `ping` is absent). */
- introspectSchema?(): Promise<unknown>;
+ /**
+ * Introspect the live schema (fallback probe when `ping` is absent, and the
+ * read the external-datasource federation path is built on).
+ *
+ * The return value is CONTRACTUAL: `packages/spec`'s one introspection
+ * shape — {@link IntrospectedSchema}, whose columns spell primary-key
+ * membership `primaryKey` and whose schema carries `dialect` and
+ * `introspectedAt`. This member was typed `Promise<unknown>` until #11381
+ * (option C of the #11123 ruling, 2026-08-23), which left the seam's OPEN
+ * producer population — host-built drivers, per this file's own header —
+ * unreachable by any compiler: a driver spelling the flag `isPrimary`, or
+ * emitting `{ tables }` alone, compiled clean, and the mis-shape surfaced
+ * only as a federated table whose records silently could not be located or
+ * updated. Typed against the spec contract, `tsc` now refuses a mis-shaped
+ * driver at the offending field (`Property 'primaryKey' is missing …` /
+ * `'isPrimary' does not exist in type 'IntrospectedColumn'`).
+ *
+ * Extra facts a richer driver carries stay legal — driver-sql's table-level
+ * `primaryKeys`, `foreignKeys`, per-column `maxLength` ride on declared
+ * types that EXTEND the spec contract, and assignability admits them on any
+ * non-literal value. What is refused is a WRONG spelling of a declared key,
+ * which is the defect class this seam has actually shipped.
+ *
+ * Runtime is deliberately untouched: the `primaryKeyReader` compatibility
+ * belt in `external-datasource-service.ts` keeps absorbing the retired
+ * `isPrimary` spelling from already-built drivers (plain-JS drivers and
+ * stale builds are reached by no compiler). Removing that belt is #11123
+ * option B — a later, separate step gated on this tightening being released
+ * AND the old spelling's retirement being published; it is not licensed by
+ * this type.
+ */
+ introspectSchema?(): Promise<IntrospectedSchema>;
  /** Liveness check on the underlying engine driver (probe fallback). */
  checkHealth?(): Promise<boolean>;
  /** Driver-reported server version, when available. */

packages/services/service-datasource/src/external-datasource-service.ts

@@ -171,15 +171,21 @@ const BUILTIN_COLUMNS = new Set(['id', 'created_at', 'updated_at']);
  * - It is still not dead code, because the producer population here is open
  * by design. `contracts/datasource-driver-factory.ts` says the framework
  * "ships no universal driver-by-id registry" — concrete drivers are built
- * by the HOST — and types the handle as `introspectSchema?(): Promise<unknown>`.
- * The retirement shipped as a BREAKING change whose stated migration
- * channel is the compiler, "precisely and at every site"; against an
- * `unknown` result that channel never fires, so a host-built driver still
- * emitting the old spelling is reached by nothing and would silently lose
- * its key here.
- * - The belt's clock has not run either: the union (#11001) and the
- * retirement (#11124) are BOTH still unconsumed changesets at `17.1.0`, so
- * no released version has ever emitted `primaryKey` from this driver.
+ * by the HOST. Since #11381 (option C of the #11123 ruling) the handle
+ * types `introspectSchema?(): Promise<IntrospectedSchema>` — the spec
+ * contract — so the retirement's stated migration channel, the compiler,
+ * finally reaches a host-built TypeScript driver "precisely and at every
+ * site" the moment it RECOMPILES against this version. Whom no compiler
+ * reaches, ever: drivers already built against older versions, plain-JS
+ * drivers, and casts. For that population the old spelling still arrives
+ * here at runtime, and this belt is what absorbs it.
+ * - The belt's clock HAS started: the union (#11001) and the retirement
+ * (#11124) were consumed into `17.2.0` (version-packages commit
+ * `e7d2cc67fd`, 2026-08-23), so the "unconsumed changesets at 17.1.0"
+ * argument this bullet used to carry is expired. Whether option B's gate —
+ * the retirement actually PUBLISHED — is met is a release-record question
+ * (npm, not this tree); B also waits on the #11381 tightening being
+ * released. Re-judge both there before touching the arm.
  *
  * Dropping the arm is therefore a NARROWING OF ACCEPTED INPUT rather than a
  * dead-code deletion, and wants the contract-review gate. Its only exercise is