feat(runtime): the standalone default datasource is a declaration, connected through the one datasource path (#3826)

.changeset/default-datasource-declared.md

@@ -0,0 +1,55 @@
+---
+"@objectstack/runtime": minor
+"@objectstack/service-datasource": minor
+"@objectstack/cli": patch
+---
+
+feat(runtime)!: the standalone `default` datasource is a declaration, connected through the one datasource path (#3826)
+
+ADR-0062 D1 asked for exactly one "definition → live driver" path. Construction
+converged earlier; the *connect + failure verdict* half did not — the standalone
+`default` driver was pre-built and smuggled into the engine as a `driver.*`
+kernel service, so "what if it cannot connect" lived in `ObjectQLEngine.init()`,
+a second implementation of the policy `DatasourceConnectionService` owns for
+every other datasource. #3741 → #3758 showed what two copies cost: a fix to one
+missed the other for three months.
+
+- **`createStandaloneStack` now emits a datasource DEFINITION**, not a driver.
+ URL→config translation and `mkdir` stay host concerns; the new
+ **`DefaultDatasourcePlugin`** (exported from `@objectstack/runtime`) connects
+ the definition at boot through the shared `DatasourceConnectionService` —
+ same driver factory, same failure verdict, same retained state. It must be
+ registered before `ObjectQLPlugin` (boot schema-sync needs the driver);
+ `createStandaloneStack` orders it correctly.
+- **`sqlite-wasm` joined the shared driver factory** (`sqlite-wasm` /
+ `wasm-sqlite` ids) — it was the last bespoke construction site.
+- **`bootCritical` on `ConnectableDatasource`**: the host declares a datasource
+ the platform cannot run without; a boot connect failure is then fatal
+ regardless of object bindings, sharing `OS_ALLOW_DRIVER_CONNECT_FAILURE` and
+ the `DEGRADED BOOT` banner with the engine-level guard. A connect policy that
+ denies a boot-critical datasource fails the boot loudly — the #3828 "denial is
+ not a failure" boundary was drawn for optional datasources.
+- **`connect(record, { asDefault: true })`**: registers the built driver as the
+ engine's default under its natural name (no `'default'` stamping — routing to
+ `default` goes through the engine's default-driver fallback, and the natural
+ name keeps logs/lookups byte-for-byte with the previous boot).
+- **`default` is a host-reserved name**: an app bundle declaring a datasource
+ named `default` is rejected at load (`AppPlugin`), and the runtime-admin
+ create rejects it too. It would shadow the host's primary datasource and, if
+ it passed the auto-connect gate, silently divert every unbound object.
+- The primary DB now shows a REAL `status` in Setup → Datasources (#3827) —
+ `ok` when connected, `error` + reason when the operator boots degraded.
+- `ObjectQLEngine.init()` is unchanged and keeps its fail-fast: it re-connects
+ the already-connected default (every open-core driver's `connect()` is
+ idempotent), which is exactly the boot verification #3741 wants.
+- `DriverPlugin` remains the escape hatch for tests and pre-built/proxy drivers
+ (e.g. the CLI's `telemetry` datasource) — no longer how the standalone
+ default boots. The CLI serve config-load fallback (`createStorageDriver`,
+ incl. mysql/turso) still constructs directly; tracked in #3826.
+
+**Migration.** Boots through `createStandaloneStack` (CLI `serve`/`dev`
+artifact path, quickstarts, embedders using the stack factory) change shape but
+not behavior: same driver kinds, same URLs, same fail-fast semantics, same
+escape hatch. Embedders that composed `DriverPlugin` manually are unaffected.
+An app that declared a datasource literally named `default` now fails to load
+with a rename instruction — that name never routed correctly to begin with.

content/docs/data-modeling/drivers.mdx

@@ -95,6 +95,14 @@ The same guard covers **declared datasources** whose objects have no fallback 
 see [When auto-connect fails](/docs/data-modeling/external-datasources#when-auto-connect-fails)
 ([#3758](https://github.com/objectstack-ai/objectstack/issues/3758)).
 
+The standalone `default` datasource itself is now a **declared definition**
+([#3826](https://github.com/objectstack-ai/objectstack/issues/3826)): the stack
+translates `OS_DATABASE_URL` into `{ driver, config }` and connects it at boot
+through the same datasource connection path — one failure verdict, one escape
+hatch, and a real `status` for the primary DB in **Setup → Datasources**. The
+name `default` is host-reserved: an app bundle declaring a datasource with that
+name is rejected at load.
+
 <Callout type="warn">
 `OS_ALLOW_DRIVER_CONNECT_FAILURE=1` boots anyway, in an explicitly degraded
 state announced by a `DEGRADED BOOT` banner. Queries to a failed driver fail

docs/adr/0062-external-datasource-runtime.md

@@ -64,9 +64,9 @@ Introduce a single service that, given a datasource definition, builds a driver
 > | pool teardown | kernel shutdown via `DriverPlugin` | `DatasourceConnectionService.disconnect()` |
 > | connect policy | not consulted | `DatasourceConnectPolicy` |
 >
-> **What actually blocks the merge is an input-shape mismatch, not ordering.** The kernel's init-all-then-start-all means the connection service *does* exist by `ObjectQLPlugin.start()`, so timing is available. The obstacles are: (1) `DatasourceConnectionService.connect()` takes a datasource *definition* and **builds** the driver, while `default` arrives as an already-constructed driver instance published as a `driver.*` kernel service — there is no "adopt this driver" entry point; and (2) routing `default` through the service would make `ObjectQLPlugin`'s boot depend on an **optional service from a higher layer** (`service-datasource`), inverting the layering for the one driver every app needs. Closing this means either adding an `adoptDriver()` seam to the connection service, or making the standalone `default` a real declared datasource definition — a design decision, not a mechanical move, and still the riskiest single step per §Risk.
+> **Resolution (#3826, second pass) — the standalone `default` is now a declared definition.** The input-shape mismatch was resolved by making the definition the input: `createStandaloneStack` translates the database URL into a `{ driver, config }` definition (URL→config translation and `mkdir` stay host concerns) and the runtime's **`DefaultDatasourcePlugin`** — registered before `ObjectQLPlugin`, so the driver exists before boot schema-sync — connects it through `DatasourceConnectionService.connect(record, { asDefault: true })`. The definition is marked **`bootCritical`**, which adds a third fail-fast cause to D5 (the platform cannot run without it; every unbound object routes to it), sharing `OS_ALLOW_DRIVER_CONNECT_FAILURE` and the `DEGRADED BOOT` banner with the engine guard. `asDefault` keeps the driver's **natural name** (routing to `default` uses the engine's default-driver fallback, never `drivers.get('default')`) and registers with `isDefault: true`. The presumed layering inversion did not materialize: the *runtime host* orchestrates (runtime already depends on `service-datasource`); `ObjectQLPlugin` learned nothing. When the datasource-admin plugin is present its shared connection service is used (so `default` shows a real `status` in Setup → Datasources, #3827); a lite kernel instantiates the same class locally — one implementation either way. `sqlite-wasm` joined the shared factory (the last bespoke construction site), `default` became a host-reserved name (rejected in app bundles at load and in runtime-admin create), and `ObjectQLEngine.init()` keeps its #3741 fail-fast unchanged — it re-connects the already-connected default (all open-core drivers' `connect()` is idempotent), which is precisely the boot *verification* role D1 leaves it.
 >
-> Until then the divergence is guarded rather than assumed: `packages/runtime/src/degraded-boot-parity.test.ts` pins both paths to the same operator-visible contract (fail-fast by default, identical `OS_ALLOW_DRIVER_CONNECT_FAILURE` parsing, `DEGRADED BOOT` on stderr), so a change to one that forgets the other fails CI instead of shipping. #3741 → #3758 was exactly that miss, and it cost three months and a second bug report.
+> **Remaining second sites, tracked in #3826:** the CLI serve **config-load fallback** (`createStorageDriver` + `DriverPlugin`, used when a host `objectstack.config.ts` supplies no driver — it also carries mysql/turso kinds the shared factory does not build, and the `telemetry` sibling-datasource provisioning is coupled to its resolution result), and the cloud stack's own composition. Until those converge, `packages/runtime/src/degraded-boot-parity.test.ts` remains load-bearing: it pins both connect paths to the same operator-visible contract (fail-fast by default, identical `OS_ALLOW_DRIVER_CONNECT_FAILURE` parsing, `DEGRADED BOOT` on stderr), so a change to one that forgets the other fails CI instead of shipping. #3741 → #3758 was exactly that miss.
 
 ### D2 — Connect is opt-in-safe: existing managed apps are byte-for-byte unchanged
 

packages/cli/src/commands/serve.ts

@@ -872,7 +872,16 @@ export default class Serve extends Command {
  // whole dispatch is unit-testable (storage-driver.test.ts). #3276: the
  // `memory` kind now maps to the mingo InMemoryDriver instead of silently
  // falling through to the dev SQLite `:memory:` default.
- const hasDriver = plugins.some((p: any) => p.name?.includes('driver') || p.constructor?.name?.includes('Driver'));
+ // A DefaultDatasourcePlugin counts as a driver provider (#3826): the
+ // standalone stack now DECLARES its `default` datasource and connects it
+ // at boot through the datasource connection service, so building a
+ // storage driver here would construct a duplicate pool the engine then
+ // discards as already-registered.
+ const hasDriver = plugins.some((p: any) =>
+ p.name?.includes('driver') ||
+ p.constructor?.name?.includes('Driver') ||
+ p.name === 'com.objectstack.runtime.default-datasource' ||
+ p.constructor?.name === 'DefaultDatasourcePlugin');
  if (!hasDriver && config.objects) {
  const databaseUrl = process.env.OS_DATABASE_URL;
  const driverType = resolveDriverType(process.env.OS_DATABASE_DRIVER, databaseUrl);

packages/objectql/src/engine.ts

@@ -1519,6 +1519,16 @@ export class ObjectQL implements IDataEngine {
  this.unavailableDatasources.delete(name);
  }
 
+ /**
+ * Name of the DEFAULT driver, when one is registered (#3826). The default
+ * driver keeps its natural name (`registerDriver(driver, true)` — nothing
+ * routes by `drivers.get('default')`), so the datasource connection layer's
+ * `asDefault` idempotency guard needs this rather than a name lookup.
+ */
+ getDefaultDriverName(): string | undefined {
+ return this.defaultDriver ?? undefined;
+ }
+
  /**
  * Datasources that were declared but are NOT usable, with the reason class.
  *

packages/runtime/src/app-plugin.ts

@@ -372,6 +372,31 @@ export class AppPlugin implements Plugin {
  // ONLY — never persisted to the runtime DB store — and stamped
  // `origin:'code'` so the admin service enforces them as read-only.
  // The engine already indexed them for the write gate via registerApp().
+ //
+ // `default` is a HOST-owned reserved name (#3826): the runtime declares
+ // and connects it (DefaultDatasourcePlugin). An app declaring it would
+ // shadow the host's metadata row and — if it passed the D2 gate —
+ // divert every unbound object to a fresh connection. Contract-first:
+ // reject at load, loudly (outside the lenient catch below), instead of
+ // letting the collision produce undefined routing.
+ {
+ const dsDefs = this.bundle.datasources;
+ const declared = Array.isArray(dsDefs)
+ ? dsDefs
+ : dsDefs && typeof dsDefs === 'object'
+ ? Object.values(dsDefs as Record<string, unknown>)
+ : [];
+ const names = Array.isArray(dsDefs)
+ ? declared.map((d: any) => d?.name)
+ : Object.keys((dsDefs as Record<string, unknown>) ?? {});
+ if (declared.some((d: any) => d?.name === 'default') || names.includes('default')) {
+ throw new Error(
+ `[AppPlugin] app '${appId}' declares a datasource named 'default' — that name is ` +
+ `reserved for the host's primary datasource. Rename it (e.g. '${appId.split('.').pop()}_primary') ` +
+ `and route objects to it explicitly, or omit it to use the host default.`,
+ );
+ }
+ }
  try {
  const dsDefs = this.bundle.datasources;
  const dsList = Array.isArray(dsDefs)

packages/runtime/src/default-datasource-plugin.test.ts

@@ -0,0 +1,153 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+//
+// ADR-0062 D1 (#3826): the standalone `default` datasource is a DECLARATION,
+// connected at boot by DefaultDatasourcePlugin through the same
+// DatasourceConnectionService as every declared/runtime datasource — one
+// connect path, one failure verdict, one escape hatch. These boots exercise
+// the real kernel (init-all → start-all) with the real driver factory.
+
+import { describe, it, expect, beforeEach, afterEach } from 'vitest';
+import { Runtime } from './runtime.js';
+import { DefaultDatasourcePlugin } from './default-datasource-plugin.js';
+import { AppPlugin } from './app-plugin.js';
+
+const BOOT_TIMEOUT = 60_000;
+const ENV = 'OS_ALLOW_DRIVER_CONNECT_FAILURE';
+
+async function assemble(opts: {
+ driver?: string;
+ withAdminPlugin?: boolean;
+ connectPolicy?: any;
+ bundle?: any;
+} = {}) {
+ const { ObjectQLPlugin } = await import('@objectstack/objectql');
+ const runtime = new Runtime({ cluster: false });
+ const kernel = runtime.getKernel();
+ // Order matters for START: the default datasource must connect before
+ // ObjectQLPlugin.start() runs boot schema-sync.
+ await kernel.use(new DefaultDatasourcePlugin({ driver: opts.driver ?? 'memory' }));
+ await kernel.use(new ObjectQLPlugin());
+ if (opts.bundle) await kernel.use(new AppPlugin(opts.bundle));
+ if (opts.withAdminPlugin !== false) {
+ const { DatasourceAdminServicePlugin, createDefaultDatasourceDriverFactory } = await import(
+ '@objectstack/service-datasource'
+ );
+ await kernel.use(
+ new DatasourceAdminServicePlugin({
+ driverFactory: createDefaultDatasourceDriverFactory(),
+ connectPolicy: opts.connectPolicy,
+ }),
+ );
+ }
+ return kernel;
+}
+
+describe('DefaultDatasourcePlugin — the default datasource as a declaration (#3826)', () => {
+ let saved: string | undefined;
+ beforeEach(() => { saved = process.env[ENV]; delete process.env[ENV]; });
+ afterEach(() => {
+ if (saved === undefined) delete process.env[ENV];
+ else process.env[ENV] = saved;
+ });
+
+ it('boots, registers the driver as DEFAULT, and serves reads/writes end to end', async () => {
+ const kernel = await assemble({
+ bundle: {
+ manifest: { id: 'com.test.default-ds', name: 'Default DS', version: '1.0.0' },
+ objects: [{ name: 'note', label: 'Note', fields: { title: { type: 'text' } } }],
+ },
+ });
+ try {
+ await kernel.bootstrap();
+ const engine = kernel.getService<any>('data');
+ // The driver keeps its NATURAL name (no 'default' stamping) — routing to
+ // `default` goes through the engine's default-driver fallback.
+ expect(engine.getDriverByName('default')).toBeUndefined();
+ await engine.insert('note', { title: 'through-the-default' });
+ const rows = await engine.find('note');
+ expect(rows.map((r: any) => r.title)).toContain('through-the-default');
+ } finally {
+ try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+ }
+ }, BOOT_TIMEOUT);
+
+ it('shows the primary DB in the datasource-admin list with a REAL status (#3827)', async () => {
+ const kernel = await assemble({});
+ try {
+ await kernel.bootstrap();
+ const admin = kernel.getService<{ listDatasources(): Promise<any[]> }>('datasource-admin');
+ const def = (await admin.listDatasources()).find((d) => d.name === 'default');
+ expect(def).toBeDefined();
+ expect(def!.status).toBe('ok');
+ } finally {
+ try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+ }
+ }, BOOT_TIMEOUT);
+
+ it('works without the datasource-admin plugin — same class, locally instantiated', async () => {
+ const kernel = await assemble({ withAdminPlugin: false });
+ try {
+ await kernel.bootstrap();
+ const engine = kernel.getService<any>('data');
+ await engine.insert('sys_metadata', undefined as never).catch(() => { /* shape probe only */ });
+ // The default driver exists and the engine can answer a trivial query path.
+ expect(typeof engine.find).toBe('function');
+ } finally {
+ try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+ }
+ }, BOOT_TIMEOUT);
+
+ it('refuses the boot when the default cannot be built/connected (bootCritical ⇒ fail-fast)', async () => {
+ const kernel = await assemble({ driver: 'not-a-real-driver' });
+ const err = await kernel.bootstrap().then(
+ () => { throw new Error('bootstrap() resolved but should have thrown'); },
+ (e: unknown) => e as Error,
+ );
+ expect(err.message).toMatch(/default/);
+ expect(err.message).toMatch(/boot-critical/);
+ expect(err.message).toContain('OS_ALLOW_DRIVER_CONNECT_FAILURE');
+ try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+ }, BOOT_TIMEOUT);
+
+ it('boots degraded under OS_ALLOW_DRIVER_CONNECT_FAILURE — same escape hatch as the engine guard', async () => {
+ process.env[ENV] = '1';
+ const kernel = await assemble({ driver: 'not-a-real-driver' });
+ try {
+ await expect(kernel.bootstrap()).resolves.not.toThrow();
+ } finally {
+ try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+ }
+ }, BOOT_TIMEOUT);
+
+ it('is NOT gated by the host connect policy — a deny-all policy cannot block the primary DB', async () => {
+ // Byte-for-byte with the pre-#3826 boot: the default never consulted a
+ // DatasourceConnectPolicy (that gate exists for optional/external
+ // datasources). A multi-tenant host's deny-all must not brick every boot.
+ const kernel = await assemble({
+ connectPolicy: { canConnect: () => ({ allow: false, reason: 'egress blocked' }) },
+ });
+ try {
+ await expect(kernel.bootstrap()).resolves.not.toThrow();
+ const engine = kernel.getService<any>('data');
+ expect(engine.getDefaultDriverName()).toBeDefined();
+ } finally {
+ try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+ }
+ }, BOOT_TIMEOUT);
+
+ it("rejects an app bundle that declares a datasource named 'default' (host-reserved name)", async () => {
+ const kernel = await assemble({
+ bundle: {
+ manifest: { id: 'com.test.reserved', name: 'Reserved', version: '1.0.0' },
+ objects: [{ name: 'note', label: 'Note', fields: { title: { type: 'text' } } }],
+ datasources: [{ name: 'default', driver: 'memory', config: {} }],
+ },
+ });
+ const err = await kernel.bootstrap().then(
+ () => { throw new Error('bootstrap() resolved but should have thrown'); },
+ (e: unknown) => e as Error,
+ );
+ expect(err.message).toMatch(/reserved for the host's primary datasource/);
+ try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+ }, BOOT_TIMEOUT);
+});