diff --git a/.changeset/serve-fallback-declared-default.md b/.changeset/serve-fallback-declared-default.md new file mode 100644 index 0000000000..3c31f57976 --- /dev/null +++ b/.changeset/serve-fallback-declared-default.md @@ -0,0 +1,41 @@ +--- +"@objectstack/cli": minor +"@objectstack/service-datasource": minor +--- + +feat(cli): the serve storage fallback declares the default datasource instead of constructing a driver (#3826) + +The last open-core second site of "definition → live driver": when a host +`objectstack.config.ts` supplies objects but no driver plugin, `serve` built a +driver via `createStorageDriver` and registered it through `DriverPlugin`, with +its connect and failure verdict landing in `ObjectQLEngine.init()` — the same +split #3869 removed from the standalone stack. + +- **`createStorageDriver` is gone.** `resolveStorageDefinition` translates the + driver kind + URL into `{ driverId, config }` (a pure host-side translation, + like `standalone-stack`'s), and serve hands it to the runtime's + `DefaultDatasourcePlugin` — same shared factory, same `bootCritical` failure + verdict, same `OS_ALLOW_DRIVER_CONNECT_FAILURE` escape hatch, and the primary + DB's real status in Setup → Datasources. +- **`mysql`/`mysql2` joined the shared driver factory** (SqlDriver over + `mysql2`; DSN or discrete fields, secret as password). +- **Host-composition passthroughs**: the factory honours `config.autoMigrate` + (the #2186 dev loosen-only self-heal, for the SQL kinds) and `config.persist` + (the CLI's wasm `on-disconnect` mode). Connection builders ignore both keys. +- **`turso`/libSQL fails loud at resolution**, same typed + `UnsupportedDriverError`, same actionable message — nothing is constructed to + fail later. +- **The `telemetry` sibling datasource stays a pre-built `DriverPlugin`** — the + documented escape hatch for named auxiliary drivers. Its provisioning now + gates on the statically-known sqlite file path; the old coupling to the + primary's *resolved* engine is replaced by the telemetry provision's own + step-down check, which already guarded the ABI-broken case. + +Verified end to end: a host-composed config (plugins + objects, no driver) +boots through the declared fallback with the same banner labels; the artifact +path (`dev:crm --fresh`) is table-for-table unchanged (71 tables, zero +`no such table`). + +**Migration.** None for CLI users — same URLs, same env vars, same banner. The +removed `createStorageDriver` was CLI-internal; `resolveDriverType`, +`inferDriverTypeFromUrl` and `UnsupportedDriverError` are unchanged. diff --git a/docs/adr/0062-external-datasource-runtime.md b/docs/adr/0062-external-datasource-runtime.md index 9dcfe6b80d..5f0e926cdd 100644 --- a/docs/adr/0062-external-datasource-runtime.md +++ b/docs/adr/0062-external-datasource-runtime.md @@ -66,7 +66,9 @@ Introduce a single service that, given a datasource definition, builds a driver > > **Resolution (#3826, second pass) — the standalone `default` is now a declared definition.** The input-shape mismatch was resolved by making the definition the input: `createStandaloneStack` translates the database URL into a `{ driver, config }` definition (URL→config translation and `mkdir` stay host concerns) and the runtime's **`DefaultDatasourcePlugin`** — registered before `ObjectQLPlugin`, so the driver exists before boot schema-sync — connects it through `DatasourceConnectionService.connect(record, { asDefault: true })`. The definition is marked **`bootCritical`**, which adds a third fail-fast cause to D5 (the platform cannot run without it; every unbound object routes to it), sharing `OS_ALLOW_DRIVER_CONNECT_FAILURE` and the `DEGRADED BOOT` banner with the engine guard. `asDefault` keeps the driver's **natural name** (routing to `default` uses the engine's default-driver fallback, never `drivers.get('default')`) and registers with `isDefault: true`. The presumed layering inversion did not materialize: the *runtime host* orchestrates (runtime already depends on `service-datasource`); `ObjectQLPlugin` learned nothing. When the datasource-admin plugin is present its shared connection service is used (so `default` shows a real `status` in Setup → Datasources, #3827); a lite kernel instantiates the same class locally — one implementation either way. `sqlite-wasm` joined the shared factory (the last bespoke construction site), `default` became a host-reserved name (rejected in app bundles at load and in runtime-admin create), and `ObjectQLEngine.init()` keeps its #3741 fail-fast unchanged — it re-connects the already-connected default (all open-core drivers' `connect()` is idempotent), which is precisely the boot *verification* role D1 leaves it. > -> **Remaining second sites, tracked in #3826:** the CLI serve **config-load fallback** (`createStorageDriver` + `DriverPlugin`, used when a host `objectstack.config.ts` supplies no driver — it also carries mysql/turso kinds the shared factory does not build, and the `telemetry` sibling-datasource provisioning is coupled to its resolution result), and the cloud stack's own composition. Until those converge, `packages/runtime/src/degraded-boot-parity.test.ts` remains load-bearing: it pins both connect paths to the same operator-visible contract (fail-fast by default, identical `OS_ALLOW_DRIVER_CONNECT_FAILURE` parsing, `DEGRADED BOOT` on stderr), so a change to one that forgets the other fails CI instead of shipping. #3741 → #3758 was exactly that miss. +> **Config-load fallback converged too (#3826, third pass).** `createStorageDriver` is gone: the CLI's serve fallback (a host `objectstack.config.ts` with objects but no driver plugin) now emits a definition via `resolveStorageDefinition` and hands it to the same `DefaultDatasourcePlugin`. `mysql` joined the shared factory for it; the dev loosen-only self-heal (#2186) rides as `config.autoMigrate` and the CLI's wasm persistence mode as `config.persist` — host-composition passthroughs the factory honours, never part of the app-facing datasource spec. `turso`/libSQL keeps its loud typed failure at *resolution* (nothing is constructed to fail later). The `telemetry` sibling datasource deliberately stays a pre-built `DriverPlugin` (the documented escape hatch for named auxiliary drivers): it is best-effort, dev-oriented, and its own `resolveSqliteDriver` step-down check replaces the old primary-resolution coupling. +> +> **Remaining second site: the cloud stack's own composition** (cloud repo, `cloud-stack.ts`), which also owns the `turso` driver. Until it converges, `packages/runtime/src/degraded-boot-parity.test.ts` remains load-bearing: it pins both connect paths to the same operator-visible contract (fail-fast by default, identical `OS_ALLOW_DRIVER_CONNECT_FAILURE` parsing, `DEGRADED BOOT` on stderr). #3741 → #3758 was exactly the miss it exists to catch. ### D2 — Connect is opt-in-safe: existing managed apps are byte-for-byte unchanged diff --git a/packages/cli/src/commands/serve.ts b/packages/cli/src/commands/serve.ts index 0f30ee11be..2b0fd5c10d 100644 --- a/packages/cli/src/commands/serve.ts +++ b/packages/cli/src/commands/serve.ts @@ -8,7 +8,7 @@ import chalk from 'chalk'; import { bundleRequire } from 'bundle-require'; import { loadConfig, BUNDLE_REQUIRE_EXTERNALS } from '../utils/config.js'; import { isHostConfig, shouldBootWithLibrary } from '../utils/plugin-detection.js'; -import { resolveDriverType, createStorageDriver, UnsupportedDriverError } from '../utils/storage-driver.js'; +import { resolveDriverType, resolveStorageDefinition, UnsupportedDriverError } from '../utils/storage-driver.js'; import { readEnvWithDeprecation, resolveMultiOrgEnabled, resolveTenancyPosture, resolveAllowDegradedTenancy, isMcpServerEnabled, resolveSearchPinyinEnabled, isModuleNotFoundError } from '@objectstack/types'; import { PLATFORM_CAPABILITY_TOKENS } from '@objectstack/spec/kernel'; import { missingProviderMessage } from '../utils/capability-preflight.js'; @@ -887,14 +887,22 @@ export default class Serve extends Command { const driverType = resolveDriverType(process.env.OS_DATABASE_DRIVER, databaseUrl); try { - const { DriverPlugin } = await import('@objectstack/runtime'); - const resolution = await createStorageDriver(driverType, { - databaseUrl, - isDev, - warn: (m) => console.warn(chalk.yellow(m)), - }); + // #3826: the fallback no longer constructs a driver — it declares + // the `default` datasource and lets the runtime's + // DefaultDatasourcePlugin connect it at boot through the shared + // DatasourceConnectionService (one connect path, one failure + // verdict incl. OS_ALLOW_DRIVER_CONNECT_FAILURE, retained status in + // Setup → Datasources). URL→config translation stays host-side in + // resolveStorageDefinition. The dev sqlite step-down (#2229) and + // the loosen-only self-heal (#2186, via config.autoMigrate) now run + // inside the factory at connect. + const { DriverPlugin, DefaultDatasourcePlugin } = await import('@objectstack/runtime'); + const resolution = resolveStorageDefinition(driverType, { databaseUrl, isDev }); if (resolution) { - await kernel.use(new DriverPlugin(resolution.driver as any)); + await kernel.use(new DefaultDatasourcePlugin( + { driver: resolution.driverId, config: resolution.config }, + { dev: isDev }, + )); trackPlugin(resolution.trackName); resolvedDriverLabel = resolution.label; resolvedDatabaseUrl = resolution.displayUrl; @@ -906,9 +914,14 @@ export default class Serve extends Command { // for file-backed primaries; `OS_TELEMETRY_DB=0` opts out, // `OS_TELEMETRY_DB=` opts in anywhere (incl. serve). Gated on // an explicit SQLite primary (`sqliteFilePath`, unset for the mingo - // memory driver AND the dev-default `:memory:` store) whose resolved - // engine is real SQLite — never mingo in-memory. - if (resolution.sqliteFilePath && resolution.engine !== 'memory') { + // memory driver AND the dev-default `:memory:` store). The old + // `resolution.engine !== 'memory'` refinement is unknowable now + // that the primary connects later (#3826); the telemetry + // provision's own `telemetry.engine !== 'memory'` check below + // still guards the ABI-broken step-down case. The telemetry + // driver itself stays a pre-built DriverPlugin — the documented + // escape hatch for named auxiliary drivers. + if (resolution.sqliteFilePath) { const { resolveTelemetryDbPath } = await import('../utils/telemetry-datasource.js'); const telemetryPath = resolveTelemetryDbPath({ primaryPath: resolution.sqliteFilePath, env: process.env, dev: isDev }); if (telemetryPath) { diff --git a/packages/cli/src/utils/storage-driver.test.ts b/packages/cli/src/utils/storage-driver.test.ts index 91f297d041..b1fd51ec81 100644 --- a/packages/cli/src/utils/storage-driver.test.ts +++ b/packages/cli/src/utils/storage-driver.test.ts @@ -1,11 +1,10 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. import { describe, it, expect } from 'vitest'; -import { InMemoryDriver } from '@objectstack/driver-memory'; import { inferDriverTypeFromUrl, resolveDriverType, - createStorageDriver, + resolveStorageDefinition, UnsupportedDriverError, } from './storage-driver.js'; @@ -60,114 +59,119 @@ describe('resolveDriverType', () => { }); }); -describe('createStorageDriver', () => { - // ── #3276: the regression this whole change exists to fix ────────────────── - // `memory` must construct the mingo InMemoryDriver — NOT fall through to the - // dev SQLite `:memory:` default (SQLite-in-memory, a different engine). Remove - // the `memory` branch in storage-driver.ts and this assertion goes red: in dev - // it resolves to a SqlDriver/SQLite engine, in prod it resolves to null. - it('constructs the mingo InMemoryDriver for `memory` in DEV', async () => { - const r = await createStorageDriver('memory', { isDev: true }); - expect(r).not.toBeNull(); - expect(r!.driver).toBeInstanceOf(InMemoryDriver); - expect(r!.engine).toBe('memory'); - expect(r!.label).toBe('InMemoryDriver'); - expect(r!.trackName).toBe('MemoryDriver'); - expect(r!.displayUrl).toBe('(in-memory)'); - // Never provisions a telemetry sibling. - expect(r!.sqliteFilePath).toBeUndefined(); - }); - - // The explicit choice is honored in PRODUCTION too — declared === enforced. - it('constructs the mingo InMemoryDriver for `memory` in PROD', async () => { - const r = await createStorageDriver('memory', { isDev: false }); - expect(r!.driver).toBeInstanceOf(InMemoryDriver); - expect(r!.engine).toBe('memory'); +describe('resolveStorageDefinition (#3826 — a definition, not a driver)', () => { + // ── #3276: the regression the memory branch exists to fix ────────────────── + // `memory` must declare the mingo InMemoryDriver — NOT fall through to the + // dev SQLite `:memory:` default. Remove the `memory` branch and this goes + // red: in dev it resolves to the sqlite dev-default, in prod to null. + it('declares the mingo memory driver for `memory` in DEV and PROD', () => { + for (const isDev of [true, false]) { + const r = resolveStorageDefinition('memory', { isDev }); + expect(r).not.toBeNull(); + expect(r!.driverId).toBe('memory'); + expect(r!.label).toBe('InMemoryDriver'); + expect(r!.trackName).toBe('MemoryDriver'); + expect(r!.displayUrl).toBe('(in-memory)'); + // Never provisions a telemetry sibling. + expect(r!.sqliteFilePath).toBeUndefined(); + } + }); + + it('accepts the `mingo` and `in-memory` aliases', () => { + expect(resolveStorageDefinition('mingo', { isDev: false })!.driverId).toBe('memory'); + expect(resolveStorageDefinition('in-memory', { isDev: false })!.driverId).toBe('memory'); + }); + + it('declares mongodb with the default URL when none is supplied', () => { + const r = resolveStorageDefinition('mongodb', { isDev: false }); + expect(r!.driverId).toBe('mongodb'); + expect(r!.config).toEqual({ url: 'mongodb://localhost:27017/objectstack' }); + expect(r!.trackName).toBe('MongoDBDriver'); }); - it('accepts the `mingo` and `in-memory` aliases', async () => { - expect((await createStorageDriver('mingo', { isDev: false }))!.driver).toBeInstanceOf(InMemoryDriver); - expect((await createStorageDriver('in-memory', { isDev: false }))!.driver).toBeInstanceOf(InMemoryDriver); + it('declares postgres / mysql with the DSN in config and their SqlDriver labels', () => { + const pg = resolveStorageDefinition('postgres', { databaseUrl: 'postgres://u:p@h/db', isDev: false }); + expect(pg!.driverId).toBe('postgres'); + expect(pg!.config.url).toBe('postgres://u:p@h/db'); + expect(pg!.label).toBe('SqlDriver(pg)'); + const my = resolveStorageDefinition('mysql', { databaseUrl: 'mysql://u:p@h/db', isDev: false }); + expect(my!.driverId).toBe('mysql'); + expect(my!.config.url).toBe('mysql://u:p@h/db'); + expect(my!.label).toBe('SqlDriver(mysql2)'); }); - // ── Regression guards for the other branches (no connection is opened) ───── - it('constructs mongodb with the default URL when none is supplied', async () => { - const r = await createStorageDriver('mongodb', { isDev: false }); - expect(r!.label).toBe('MongoDBDriver'); - expect(r!.trackName).toBe('MongoDBDriver'); - expect(r!.displayUrl).toBe('mongodb://localhost:27017/objectstack'); + // #2186: the dev loosen-only self-heal rides in config so the shared factory + // applies it at construction. Never present in production configs. + it('carries autoMigrate:safe in DEV configs for the SQL kinds, never in PROD', () => { + expect(resolveStorageDefinition('sqlite', { databaseUrl: 'file:./x.db', isDev: true })!.config.autoMigrate).toBe('safe'); + expect(resolveStorageDefinition('postgres', { databaseUrl: 'postgres://h/db', isDev: true })!.config.autoMigrate).toBe('safe'); + expect(resolveStorageDefinition('mysql', { databaseUrl: 'mysql://h/db', isDev: true })!.config.autoMigrate).toBe('safe'); + expect(resolveStorageDefinition('sqlite', { databaseUrl: 'file:./x.db', isDev: false })!.config.autoMigrate).toBeUndefined(); }); - it('constructs postgres / mysql with their SqlDriver labels', async () => { - const pg = await createStorageDriver('postgres', { databaseUrl: 'postgres://u:p@h/db', isDev: false }); - expect(pg!.label).toBe('SqlDriver(pg)'); - expect(pg!.trackName).toBe('PostgresDriver'); - const my = await createStorageDriver('mysql', { databaseUrl: 'mysql://u:p@h/db', isDev: false }); - expect(my!.label).toBe('SqlDriver(mysql2)'); - expect(my!.trackName).toBe('MySQLDriver'); + it('declares sqlite-wasm with the CLI on-disconnect persistence', () => { + const r = resolveStorageDefinition('sqlite-wasm', { databaseUrl: 'file:./x.db', isDev: false }); + expect(r!.driverId).toBe('sqlite-wasm'); + expect(r!.config).toEqual({ filename: './x.db', persist: 'on-disconnect' }); }); - it('constructs sqlite-wasm without connecting', async () => { - const r = await createStorageDriver('sqlite-wasm', { databaseUrl: 'file:./x.db', isDev: false }); - expect(r!.label).toBe('SqliteWasmDriver'); - expect(r!.trackName).toBe('SqliteWasmDriver'); + // An explicit sqlite primary surfaces `sqliteFilePath` for the telemetry + // sibling — the field the memory/dev-default branches deliberately leave unset. + it('declares explicit sqlite and surfaces sqliteFilePath for telemetry', () => { + const r = resolveStorageDefinition('sqlite', { databaseUrl: 'file:./app.db', isDev: false }); + expect(r!.driverId).toBe('sqlite'); + expect(r!.config.filename).toBe('./app.db'); + expect(r!.sqliteFilePath).toBe('./app.db'); }); - // In PROD, `resolveSqliteDriver` returns the native driver UNPROBED (no - // connect), so this is fast and native-addon-free. It also documents that an - // explicit sqlite primary DOES surface `sqliteFilePath` for the telemetry - // sibling — the field the `memory` driver deliberately leaves unset. - it('constructs explicit sqlite and surfaces sqliteFilePath for telemetry', async () => { - const r = await createStorageDriver('sqlite', { databaseUrl: ':memory:', isDev: false }); - expect(r!.engine).toBe('better-sqlite3'); - expect(r!.label).toBe('SqlDriver(sqlite)'); - expect(r!.trackName).toBe('SqlDriver'); - expect(r!.sqliteFilePath).toBe(':memory:'); + it('dev default (no driver) declares sqlite :memory: with NO telemetry sibling', () => { + const r = resolveStorageDefinition('', { isDev: true }); + expect(r!.driverId).toBe('sqlite'); + expect(r!.config.filename).toBe(':memory:'); + expect(r!.sqliteFilePath).toBeUndefined(); }); // Production with no driver configured registers nothing (loud downstream // failure), rather than silently inventing an engine. - it('returns null for an unknown/absent driver in PROD', async () => { - expect(await createStorageDriver('', { isDev: false })).toBeNull(); - expect(await createStorageDriver('nonsense', { isDev: false })).toBeNull(); + it('returns null for an unknown/absent driver in PROD', () => { + expect(resolveStorageDefinition('', { isDev: false })).toBeNull(); + expect(resolveStorageDefinition('nonsense', { isDev: false })).toBeNull(); }); }); -describe('createStorageDriver: turso / libSQL is recognized but fails loud', () => { +describe('resolveStorageDefinition: turso / libSQL is recognized but fails loud', () => { // turso is a cloud/EE driver (@objectstack/driver-turso) the open-core CLI - // cannot construct. Selecting it must THROW a typed error — NOT fall through - // to the SQLite default. Remove the `turso` branch in storage-driver.ts and - // these go red: in dev it resolves to a SqlDriver, in prod it returns null — - // both silently ignoring the requested turso engine (the reported bug). - it('throws UnsupportedDriverError for `turso` in DEV and PROD', async () => { - await expect(createStorageDriver('turso', { isDev: true })).rejects.toBeInstanceOf(UnsupportedDriverError); - await expect(createStorageDriver('turso', { isDev: false })).rejects.toBeInstanceOf(UnsupportedDriverError); + // cannot build. Selecting it must THROW a typed error — NOT fall through to + // the SQLite default. Remove the `turso` branch and these go red: in dev it + // resolves to the sqlite dev-default, in prod it returns null — both silently + // ignoring the requested turso engine (the reported bug). + it('throws UnsupportedDriverError for `turso` in DEV and PROD', () => { + expect(() => resolveStorageDefinition('turso', { isDev: true })).toThrow(UnsupportedDriverError); + expect(() => resolveStorageDefinition('turso', { isDev: false })).toThrow(UnsupportedDriverError); }); - it('throws for the `libsql` alias too', async () => { - await expect(createStorageDriver('libsql', { isDev: true })).rejects.toBeInstanceOf(UnsupportedDriverError); + it('throws for the `libsql` alias too', () => { + expect(() => resolveStorageDefinition('libsql', { isDev: true })).toThrow(UnsupportedDriverError); }); // The message must be actionable: name the cloud/EE package and the open-core // alternatives, so an operator knows exactly how to proceed. - it('carries an actionable message (cloud/EE package + open-core alternatives)', async () => { - await expect(createStorageDriver('turso', { isDev: false })).rejects.toThrow(/@objectstack\/driver-turso/); - await expect(createStorageDriver('turso', { isDev: false })).rejects.toThrow(/cloud|enterprise/i); - const err = await createStorageDriver('turso', { isDev: false }).catch((e) => e); + it('carries an actionable message (cloud/EE package + open-core alternatives)', () => { + expect(() => resolveStorageDefinition('turso', { isDev: false })).toThrow(/@objectstack\/driver-turso/); + let err: unknown; + try { resolveStorageDefinition('turso', { isDev: false }); } catch (e) { err = e; } expect(err).toBeInstanceOf(UnsupportedDriverError); expect((err as UnsupportedDriverError).driverType).toBe('turso'); + expect((err as Error).message).toMatch(/cloud|enterprise/i); }); // A `libsql://` / Turso URL routes to the same loud failure — it is NOT left // unrecognized (which would silently fall through to SQLite). - it('routes libsql:// and *.turso.* URLs to the turso failure, never SQLite', async () => { + it('routes libsql:// and *.turso.* URLs to the turso failure, never SQLite', () => { expect(resolveDriverType(undefined, 'libsql://my-db.turso.io')).toBe('turso'); expect(resolveDriverType(undefined, 'https://my-db.turso.io')).toBe('turso'); - await expect( - createStorageDriver(resolveDriverType(undefined, 'libsql://my-db.turso.io'), { - databaseUrl: 'libsql://my-db.turso.io', - isDev: true, - }), - ).rejects.toBeInstanceOf(UnsupportedDriverError); + expect(() => + resolveStorageDefinition(resolveDriverType(undefined, 'libsql://my-db.turso.io'), { isDev: true }), + ).toThrow(UnsupportedDriverError); }); }); diff --git a/packages/cli/src/utils/storage-driver.ts b/packages/cli/src/utils/storage-driver.ts index 25f3debb5d..aecd27720a 100644 --- a/packages/cli/src/utils/storage-driver.ts +++ b/packages/cli/src/utils/storage-driver.ts @@ -9,9 +9,13 @@ * * 1. {@link resolveDriverType} — pick a canonical driver KIND from the * explicit `OS_DATABASE_DRIVER` override plus the `OS_DATABASE_URL` scheme. - * 2. {@link createStorageDriver} — construct the concrete driver instance for - * that kind. Driver packages are dynamically imported so the CLI carries no - * static dependency on any one of them. + * 2. {@link resolveStorageDefinition} — translate that kind + URL into the + * `default` datasource DEFINITION (`{ driverId, config }`). Since #3826 + * the CLI constructs no driver here: the runtime's + * `DefaultDatasourcePlugin` connects the definition at boot through the + * shared `DatasourceConnectionService`, so the config-load fallback shares + * the same connect path, failure verdict, and escape hatch as every other + * datasource. * * ## #3276 — the `memory` driver was advertised but had no dispatch branch * @@ -33,7 +37,7 @@ export type SqliteFamilyEngine = 'better-sqlite3' | 'sqlite-wasm' | 'memory'; /** - * Thrown by {@link createStorageDriver} when a driver kind is *recognized* but the + * Thrown by {@link resolveStorageDefinition} when a driver kind is *recognized* but the * open-core CLI cannot construct it — currently `turso`/libSQL, which ships in the * ObjectStack cloud / enterprise distribution (`@objectstack/driver-turso`, an * extension of SqlDriver over `@libsql/client`), composed by the cloud runtime's @@ -67,7 +71,7 @@ export function inferDriverTypeFromUrl(url: string | undefined): string { if (/^mysql2?:\/\//i.test(u)) return 'mysql'; // libSQL / Turso URLs are DELIBERATELY still classified as `turso` (not left // unrecognized). Open-core can't construct that driver, but classifying it - // lets createStorageDriver fail LOUDLY with a clear cloud/EE message — if we + // lets resolveStorageDefinition fail LOUDLY with a clear cloud/EE message — if we // returned '' here instead, a `libsql://` URL would fall through to the SQLite // default and silently ignore the remote connection (the very bug we're fixing). if (/^libsql:\/\//i.test(u)) return 'turso'; @@ -94,66 +98,71 @@ export function resolveDriverType( return explicit || inferDriverTypeFromUrl(databaseUrl); } -export interface CreateStorageDriverOptions { - /** Raw `OS_DATABASE_URL` (scheme stripped per-branch as before). */ +export interface ResolveStorageDefinitionOptions { + /** Raw `OS_DATABASE_URL` (scheme stripped per-branch, as the old constructor did). */ databaseUrl?: string; - /** Dev mode — enables the sqlite native→wasm→in-memory step-down + loosen-only auto-migrate. */ + /** Dev mode — arms the sqlite step-down + the loosen-only auto-migrate (#2186). */ isDev: boolean; - /** Warning sink for the sqlite step-down banners (serve.ts wraps in `chalk.yellow`). */ - warn?: (message: string) => void; } -export interface StorageDriverResolution { - /** The concrete driver instance to wrap in `DriverPlugin` and register. */ - driver: unknown; +/** + * A `default`-datasource DEFINITION for a canonical driver kind (#3826). + * + * This replaced `createStorageDriver`: serve no longer constructs a driver and + * wraps it in `DriverPlugin` — it hands `{ driverId, config }` to the runtime's + * `DefaultDatasourcePlugin`, which connects it at boot through the shared + * `DatasourceConnectionService` (one connect path, one failure verdict, one + * `OS_ALLOW_DRIVER_CONNECT_FAILURE` escape hatch, retained status in + * Setup → Datasources). URL→config translation stays here — a host concern, + * exactly like `standalone-stack`'s. + * + * Two knowingly-static fields, because nothing is constructed yet: + * - `label`/`trackName` name the REQUESTED engine. The dev sqlite step-down + * (#2229) may resolve to wasm/in-memory at connect; its own banner says so. + * - `sqliteFilePath` keys the telemetry-sibling provisioning. The old + * `resolution.engine !== 'memory'` refinement is unknowable pre-connect; + * the telemetry provision's OWN step-down check (`telemetry.engine !== + * 'memory'`) still guards the ABI-broken case. + */ +export interface StorageDefinitionResolution { + /** Driver id the shared datasource factory can build. */ + driverId: string; + /** Factory config, incl. the `autoMigrate`/`persist` host passthroughs. */ + config: Record; /** Short name for the boot banner's plugin list (serve.ts `trackPlugin`). */ trackName: string; - /** Human label for the startup banner's "driver" row. */ + /** Human label for the startup banner's "driver" row (requested engine). */ label: string; /** Display-shaped database URL for the startup banner (e.g. `(in-memory)`). */ displayUrl: string | undefined; - /** sqlite-family resolved engine, else undefined. Keys the telemetry-sibling guard. */ - engine?: SqliteFamilyEngine; - /** - * On-disk sqlite path the telemetry datasource is provisioned next to. Set - * ONLY for the explicit `sqlite`/`sql` driver — never the dev-default - * `:memory:` path — so serve.ts provisions the telemetry sibling exactly where - * it did before this extraction. - */ + /** On-disk sqlite path the telemetry datasource is provisioned next to. */ sqliteFilePath?: string; } /** - * Construct the storage driver for a canonical driver kind. Returns `null` when - * nothing matches and we are NOT in dev (production with an unknown/absent - * driver registers no driver, matching the prior inline behavior). - * - * Throws {@link UnsupportedDriverError} for `turso`/libSQL — a cloud/EE driver the - * open-core CLI cannot construct. serve.ts surfaces that as a fatal, actionable - * boot error so the selection never silently degrades to SQLite. + * Resolve the `default` datasource definition for a canonical driver kind. + * Returns `null` when nothing matches and we are NOT in dev (production with an + * unknown/absent driver registers no datasource, so the missing driver surfaces + * loudly downstream — the pre-#3826 behavior). * - * @see {@link resolveDriverType} + * Throws {@link UnsupportedDriverError} for `turso`/libSQL — a cloud/EE driver + * open-core cannot build. serve.ts surfaces that as a fatal, actionable boot + * error so the selection never silently degrades to SQLite. */ -export async function createStorageDriver( +export function resolveStorageDefinition( driverType: string, - opts: CreateStorageDriverOptions, -): Promise { + opts: ResolveStorageDefinitionOptions, +): StorageDefinitionResolution | null { const { databaseUrl, isDev } = opts; - const warn = - opts.warn ?? - ((message: string) => { - try { - console.warn(message); - } catch { - /* ignore */ - } - }); + // #2186: dev-only loosen-only self-heal, honored by the factory for the SQL + // kinds. Never in production, never destructive. + const autoMigrate = isDev ? ({ autoMigrate: 'safe' } as const) : {}; if (driverType === 'mongodb' || driverType === 'mongo') { - const { MongoDBDriver } = await import('@objectstack/driver-mongodb'); const url = databaseUrl ?? 'mongodb://localhost:27017/objectstack'; return { - driver: new MongoDBDriver({ url }) as any, + driverId: 'mongodb', + config: { url }, trackName: 'MongoDBDriver', label: 'MongoDBDriver', displayUrl: url, @@ -165,43 +174,29 @@ export async function createStorageDriver( .replace(/^file:/, '') .replace(/^sqlite:/, '') .replace(/^sql:\/\//, ''); - // Probe-by-connect with a dev-only native → wasm → in-memory step-down - // (#2229). better-sqlite3 loads its native addon lazily (first query), so an - // ABI mismatch is invisible here and would otherwise surface much later as a - // runtime crash. resolveSqliteDriver forces the load and degrades gracefully - // in dev / fails loudly in prod. - const { resolveSqliteDriver } = await import('@objectstack/service-datasource'); - const resolved = await resolveSqliteDriver({ - filename: filePath, - dev: isDev, - // #2186: in dev, self-heal a persisted DB when a metadata change relaxes a - // constraint (loosen-only; never destructive / never in prod). - autoMigrate: isDev ? 'safe' : undefined, - warn, - }); return { - driver: resolved.driver, - trackName: - resolved.engine === 'memory' - ? 'MemoryDriver' - : resolved.engine === 'sqlite-wasm' - ? 'SqliteWasmDriver' - : 'SqlDriver', - label: resolved.label, - displayUrl: resolved.engine === 'memory' ? '(in-memory)' : (databaseUrl ?? ':memory:'), - engine: resolved.engine, + driverId: 'sqlite', + config: { filename: filePath, ...autoMigrate }, + trackName: 'SqlDriver', + label: 'SqlDriver(better-sqlite3)', + displayUrl: databaseUrl ?? ':memory:', + // Only the explicit sqlite kind gets a telemetry sibling — never the + // dev-default `:memory:` branch below (resolveTelemetryDbPath also + // returns undefined for `:memory:`; both gates preserved). sqliteFilePath: filePath, }; } if (driverType === 'sqlite-wasm' || driverType === 'wasm-sqlite' || driverType === 'wasm') { - const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm'); const filePath = (databaseUrl ?? ':memory:') .replace(/^file:/, '') .replace(/^wasm-sqlite:\/\//, '') .replace(/^sqlite:/, ''); return { - driver: new SqliteWasmDriver({ filename: filePath, persist: 'on-disconnect' }) as any, + driverId: 'sqlite-wasm', + // `persist` passthrough: the CLI kept `on-disconnect` semantics here + // (the factory's default for file-backed wasm is `on-write`). + config: { filename: filePath, persist: 'on-disconnect' }, trackName: 'SqliteWasmDriver', label: 'SqliteWasmDriver', displayUrl: databaseUrl ?? ':memory:', @@ -209,14 +204,9 @@ export async function createStorageDriver( } if (driverType === 'postgres' || driverType === 'postgresql' || driverType === 'pg') { - const { SqlDriver } = await import('@objectstack/driver-sql'); return { - driver: new SqlDriver({ - client: 'pg', - connection: databaseUrl, - pool: { min: 0, max: 5 }, - autoMigrate: isDev ? 'safe' : undefined, // #2186 dev loosen-only self-heal - }) as any, + driverId: 'postgres', + config: { url: databaseUrl, ...autoMigrate }, trackName: 'PostgresDriver', label: 'SqlDriver(pg)', displayUrl: databaseUrl, @@ -224,14 +214,9 @@ export async function createStorageDriver( } if (driverType === 'mysql' || driverType === 'mysql2') { - const { SqlDriver } = await import('@objectstack/driver-sql'); return { - driver: new SqlDriver({ - client: 'mysql2', - connection: databaseUrl, - pool: { min: 0, max: 5 }, - autoMigrate: isDev ? 'safe' : undefined, // #2186 dev loosen-only self-heal - }) as any, + driverId: 'mysql', + config: { url: databaseUrl, ...autoMigrate }, trackName: 'MySQLDriver', label: 'SqlDriver(mysql2)', displayUrl: databaseUrl, @@ -240,12 +225,10 @@ export async function createStorageDriver( // turso / libSQL: recognized but NOT constructible by the open-core CLI. The // driver (`@objectstack/driver-turso`) ships in the cloud / enterprise - // distribution and is composed by the cloud runtime's own kernel factory — - // runtime/standalone-stack.ts explicitly stopped consuming its auth token, and - // its config schema lives in the cloud package so it never pollutes open-core - // `@objectstack/spec`. Fail LOUDLY here rather than let the selection fall - // through to the SQLite default (the reported "declared ≠ enforced" bug): - // serve.ts turns this typed error into a fatal, actionable boot message. + // distribution and is composed by the cloud runtime's own kernel factory. + // Fail LOUDLY here rather than let the selection fall through to the SQLite + // default (the reported "declared ≠ enforced" bug): serve.ts turns this typed + // error into a fatal, actionable boot message. if (driverType === 'turso' || driverType === 'libsql') { throw new UnsupportedDriverError( 'turso', @@ -261,46 +244,30 @@ export async function createStorageDriver( // #3276: explicit in-memory (mingo) driver. Honored in dev AND production — an // operator asking for `memory` gets the mingo InMemoryDriver (ephemeral, not - // real SQL), never the SQLite `:memory:` default. This is the branch whose - // absence caused the reported "declared ≠ enforced" fall-through to SQLite. + // real SQL), never the SQLite `:memory:` default. if (driverType === 'memory' || driverType === 'mingo' || driverType === 'in-memory') { - const { InMemoryDriver } = await import('@objectstack/driver-memory'); return { - driver: new InMemoryDriver(), + driverId: 'memory', + config: {}, trackName: 'MemoryDriver', label: 'InMemoryDriver', displayUrl: '(in-memory)', - engine: 'memory', }; } // Default (no driver configured): dev prefers native SQLite for production-like - // SQL at native speed, with a graceful step-down to wasm SQLite then in-memory - // when the native better-sqlite3 binary is unavailable (#2229). Production - // registers nothing here so a missing driver surfaces loudly downstream. + // SQL at native speed; the factory's step-down (#2229) degrades to wasm then + // in-memory when the native binary is unavailable. Production returns null so + // a missing driver surfaces loudly downstream. if (isDev) { - const { resolveSqliteDriver } = await import('@objectstack/service-datasource'); - const resolved = await resolveSqliteDriver({ - filename: ':memory:', - dev: true, - autoMigrate: 'safe', // #2186 dev loosen-only self-heal - warn, - }); return { - driver: resolved.driver, - trackName: - resolved.engine === 'memory' - ? 'MemoryDriver' - : resolved.engine === 'sqlite-wasm' - ? 'SqliteWasmDriver' - : 'SqlDriver', - label: resolved.label, - displayUrl: resolved.engine === 'memory' ? '(in-memory)' : ':memory:', - engine: resolved.engine, + driverId: 'sqlite', + config: { filename: ':memory:', ...autoMigrate }, + trackName: 'SqlDriver', + label: 'SqlDriver(better-sqlite3)', + displayUrl: ':memory:', // No sqliteFilePath: the dev-default `:memory:` store never gets a - // telemetry sibling (resolveTelemetryDbPath returns undefined for it), so - // leaving this unset keeps serve.ts from provisioning one — matching the - // pre-extraction behavior where this branch never touched telemetry. + // telemetry sibling. }; } diff --git a/packages/services/service-datasource/src/__tests__/default-datasource-driver-factory.test.ts b/packages/services/service-datasource/src/__tests__/default-datasource-driver-factory.test.ts index c7bdf55d7b..ad7cc7e6b0 100644 --- a/packages/services/service-datasource/src/__tests__/default-datasource-driver-factory.test.ts +++ b/packages/services/service-datasource/src/__tests__/default-datasource-driver-factory.test.ts @@ -55,3 +55,24 @@ describe('createDefaultDatasourceDriverFactory — sqlite-wasm construction (#38 expect(await roundTrip({})).toContain('wasm-hello'); }, 30_000); }); + +// #3826 (config-load convergence): mysql joined the factory so the CLI's +// serve fallback can DECLARE a mysql default instead of constructing one. +describe('createDefaultDatasourceDriverFactory — mysql construction', () => { + it('supports the mysql id and its mysql2 alias', () => { + expect(factory().supports('mysql')).toBe(true); + expect(factory().supports('mysql2')).toBe(true); + }); + + it('builds a SqlDriver(mysql2) from a DSN without connecting', async () => { + const handle: any = await factory().create({ + driver: 'mysql', + config: { url: 'mysql://user:pw@localhost:3306/db' }, + }); + const driver = handle.driver ?? handle; + expect(driver?.constructor?.name).toMatch(/SqlDriver$/); + // Construction must not open a socket — no connect() was called. + expect(typeof handle.connect).toBe('function'); + try { await handle.disconnect?.(); } catch { /* pool never opened */ } + }); +}); diff --git a/packages/services/service-datasource/src/default-datasource-driver-factory.ts b/packages/services/service-datasource/src/default-datasource-driver-factory.ts index 6d7e11acaa..d7cbc3e619 100644 --- a/packages/services/service-datasource/src/default-datasource-driver-factory.ts +++ b/packages/services/service-datasource/src/default-datasource-driver-factory.ts @@ -14,6 +14,7 @@ * - `postgres` / `pg` / `postgresql` → `@objectstack/driver-sql` (client `pg`) * - `sqlite` / `sqlite3` → `@objectstack/driver-sql` (better-sqlite3) * - `sqlite-wasm` / `wasm-sqlite` → `@objectstack/driver-sqlite-wasm` (pure-JS) + * - `mysql` / `mysql2` → `@objectstack/driver-sql` (client `mysql2`) * - `mongodb` / `mongo` → `@objectstack/driver-mongodb` (peer dep) * - `memory` / `inmemory` → `@objectstack/driver-memory` * @@ -35,7 +36,7 @@ import type { DatasourceDriverHandle, } from './contracts/index.js'; -type ResolvedKind = 'postgres' | 'sqlite' | 'sqlite-wasm' | 'mongodb' | 'memory'; +type ResolvedKind = 'postgres' | 'sqlite' | 'sqlite-wasm' | 'mysql' | 'mongodb' | 'memory'; const DRIVER_ID_ALIASES: Record = { postgres: 'postgres', @@ -46,6 +47,8 @@ const DRIVER_ID_ALIASES: Record = { 'better-sqlite3': 'sqlite', 'sqlite-wasm': 'sqlite-wasm', 'wasm-sqlite': 'sqlite-wasm', + mysql: 'mysql', + mysql2: 'mysql', mongodb: 'mongodb', mongo: 'mongodb', memory: 'memory', @@ -103,6 +106,26 @@ function buildSqlConnection(spec: DatasourceConnectionSpec, client: 'pg' | 'bett }; } +/** + * Build the Knex `connection` for mysql2 from a spec's config + secret. A DSN + * (`url`/`connectionString`) passes through as-is — knex's mysql2 dialect + * accepts a connection string; otherwise discrete fields, with the secret as + * the password (never part of `config`). + */ +function buildMysqlConnection(spec: DatasourceConnectionSpec): unknown { + const cfg = (spec.config ?? {}) as Record; + const url = (cfg.url as string | undefined) ?? (cfg.connectionString as string | undefined); + if (url) return url; + return { + host: cfg.host, + port: cfg.port, + database: cfg.database, + user: cfg.user ?? cfg.username, + ...(spec.secret ? { password: spec.secret } : cfg.password ? { password: cfg.password } : {}), + ...(cfg.ssl != null ? { ssl: cfg.ssl } : {}), + }; +} + /** Build a mongodb connection URL from a spec's config + secret. */ function buildMongoUrl(spec: DatasourceConnectionSpec): string { const cfg = (spec.config ?? {}) as Record; @@ -147,6 +170,13 @@ export function createDefaultDatasourceDriverFactory( const schemaMode = (spec.external as { schemaMode?: string } | undefined)?.schemaMode ?? ((spec.config as Record | undefined)?.schemaMode as string | undefined); + // Host-composition passthroughs (#3826): the CLI's declared `default` + // definition carries the dev loosen-only self-heal (#2186) and the wasm + // persistence mode in `config`. Connection builders ignore both keys, so + // they never leak into a DSN. + const cfg = (spec.config ?? {}) as Record; + const autoMigrate = cfg.autoMigrate as 'safe' | undefined; + const persistOverride = cfg.persist as string | undefined; if (kind === 'postgres') { const { SqlDriver } = await import('@objectstack/driver-sql'); @@ -155,6 +185,7 @@ export function createDefaultDatasourceDriverFactory( connection: buildSqlConnection(spec, 'pg') as any, pool: { min: 0, max: 5 }, ...(schemaMode ? { schemaMode: schemaMode as any } : {}), + ...(autoMigrate ? { autoMigrate } : {}), } as any); return toHandle(driver, () => sqlServerVersion(driver, 'pg')); } @@ -171,6 +202,7 @@ export function createDefaultDatasourceDriverFactory( filename: conn.filename ?? ':memory:', dev: options.dev, ...(schemaMode ? { schemaMode } : {}), + ...(autoMigrate ? { autoMigrate } : {}), }); return toHandle(resolved.driver, () => sqlServerVersion(resolved.driver, 'sqlite')); } @@ -193,12 +225,24 @@ export function createDefaultDatasourceDriverFactory( const filename = conn.filename ?? ':memory:'; const driver = new SqliteWasmDriver({ filename, - persist: filename !== ':memory:' ? 'on-write' : undefined, + persist: persistOverride ?? (filename !== ':memory:' ? 'on-write' : undefined), ...(schemaMode ? { schemaMode } : {}), }); return toHandle(driver, () => sqlServerVersion(driver, 'sqlite')); } + if (kind === 'mysql') { + const { SqlDriver } = await import('@objectstack/driver-sql'); + const driver = new SqlDriver({ + client: 'mysql2', + connection: buildMysqlConnection(spec) as any, + pool: { min: 0, max: 5 }, + ...(schemaMode ? { schemaMode: schemaMode as any } : {}), + ...(autoMigrate ? { autoMigrate } : {}), + } as any); + return toHandle(driver); + } + if (kind === 'mongodb') { let MongoDBDriver: any; try {