From 87cdb7cf84fad503b0b6789415c82606dc1812ea Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 09:58:43 +0000 Subject: [PATCH 1/3] fix(service-datasource,spec): a declared `memory` datasource is ephemeral, and its config reaches the driver (#4083) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "memory" did not mean memory. The memory branch of `createDefaultDatasourceDriverFactory` was the one kind that dropped `spec.config` on the floor and constructed a bare `new InMemoryDriver()`. The driver's own default is `persistence: 'auto'`, which under Node means file persistence at a CWD-relative `.objectstack/data/memory-driver.json` — so a datasource declared `driver: 'memory'` silently became a file-backed store that reloaded its rows on the next boot, with nothing disclosing it, and two memory datasources in one process wrote through that same default path. It also contradicted the platform's own contract: `objectstack dev` reads an explicit memory driver as the user opting OUT of persistence (`resolveDefaultDevDbUrl`) and persists through a sqlite file instead. Persistence now follows an EXPLICIT request, matching the `sqlite-wasm` branch directly above it (`:memory:` ephemeral, a filename persists). An author who wants a file says so: `config: { persistence: 'file' }`. Constructing `new InMemoryDriver()` yourself is untouched — that path still defaults to `'auto'`, and the spec's TSDoc now states the difference instead of leaving two de-facto contracts. How it surfaced — the reason this is filed as a bug and not a preference: `datasource-autoconnect.test.ts` (the ADR-0062 D1 acceptance test) seeds two rows and asserts a federated read returns them. On a virgin checkout it passed, so CI — always a fresh checkout — was permanently green. On every later local run it failed, the seeded rows accumulating two at a time (2 → 4 → 6 …). The test that is supposed to protect declared-datasource auto-connect was only ever green by accident of a clean working directory. - The factory forwards the declared `config` and defaults `persistence: false`. - Three regression tests in the factory's own suite: ephemeral by default, nothing written to a scratch CWD after a write, and an explicit persistence request still honoured. The first two fail if the fix is reverted. - The acceptance test constructs its default driver with an explicit `persistence: false` — a test must neither depend on nor leave behind state in the checkout. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UvcnzFr5XBuvvAd7mosG54 --- .changeset/memory-datasource-is-ephemeral.md | 14 ++++ .../src/datasource-autoconnect.test.ts | 10 ++- .../default-datasource-driver-factory.test.ts | 64 ++++++++++++++++++- .../src/default-datasource-driver-factory.ts | 24 ++++++- packages/spec/src/data/driver/memory.zod.ts | 11 +++- 5 files changed, 119 insertions(+), 4 deletions(-) create mode 100644 .changeset/memory-datasource-is-ephemeral.md diff --git a/.changeset/memory-datasource-is-ephemeral.md b/.changeset/memory-datasource-is-ephemeral.md new file mode 100644 index 0000000000..cd90530764 --- /dev/null +++ b/.changeset/memory-datasource-is-ephemeral.md @@ -0,0 +1,14 @@ +--- +'@objectstack/service-datasource': patch +'@objectstack/spec': patch +--- + +A datasource declared with `driver: 'memory'` is now ephemeral, and its `config` reaches the driver (#4083). + +The memory branch of `createDefaultDatasourceDriverFactory` was the one kind that dropped `spec.config` and constructed a bare `new InMemoryDriver()`. The driver's own default is `persistence: 'auto'`, which under Node means file persistence at a CWD-relative `.objectstack/data/memory-driver.json` — so a declared memory datasource silently became a file-backed store that reloaded its rows on the next boot, nothing disclosed it, and two memory datasources in one process wrote through the same default path. + +It also contradicted the platform's own contract: `objectstack dev` treats an explicit memory driver as the user opting *out* of persistence and persists through a sqlite file instead. + +How it surfaced: the ADR-0062 auto-connect acceptance test seeds two rows and asserts a federated read returns them. On a virgin checkout it passed — so CI, always fresh, stayed green — and on every later local run it failed, the rows accumulating two at a time. + +FROM → TO: if you relied on a declared memory datasource keeping its data across restarts, say so explicitly — `{ driver: 'memory', config: { persistence: 'file' } }` (or any shape `MemoryConfigSchema.persistence` accepts, including a custom adapter). Constructing `new InMemoryDriver()` yourself is unchanged: that path still defaults to `'auto'`. diff --git a/packages/runtime/src/datasource-autoconnect.test.ts b/packages/runtime/src/datasource-autoconnect.test.ts index 99505205f8..2ae2dab8cf 100644 --- a/packages/runtime/src/datasource-autoconnect.test.ts +++ b/packages/runtime/src/datasource-autoconnect.test.ts @@ -71,7 +71,15 @@ async function boot(opts: { connectPolicy?: DatasourceConnectPolicy } = {}) { const runtime = new Runtime({ cluster: false }); const kernel = runtime.getKernel(); - await kernel.use(new DriverPlugin(new InMemoryDriver())); // default driver + // [#4083] `persistence: false` explicitly: InMemoryDriver's own default is + // `'auto'`, which under Node writes a CWD-relative + // `.objectstack/data/memory-driver.json`. A test must neither depend on nor + // leave behind state in the checkout — and this file learned that the hard + // way: its federated-read assertion passed on a virgin clone (so CI, always + // fresh, stayed green) and failed on every later local run, the seeded rows + // accumulating two per run. The factory-built datasource driver is ephemeral + // by default now; this one is constructed directly, so it says so itself. + await kernel.use(new DriverPlugin(new InMemoryDriver({ persistence: false }))); // default driver await kernel.use(new ObjectQLPlugin()); await kernel.use(new AppPlugin(artifact())); await kernel.use( diff --git a/packages/services/service-datasource/src/__tests__/default-datasource-driver-factory.test.ts b/packages/services/service-datasource/src/__tests__/default-datasource-driver-factory.test.ts index ad7cc7e6b0..c2bc3d3879 100644 --- a/packages/services/service-datasource/src/__tests__/default-datasource-driver-factory.test.ts +++ b/packages/services/service-datasource/src/__tests__/default-datasource-driver-factory.test.ts @@ -6,7 +6,7 @@ // kind. These are the first direct tests of the factory's id → driver mapping. import { describe, it, expect, beforeAll, afterAll } from 'vitest'; -import { mkdtempSync, rmSync } from 'node:fs'; +import { mkdtempSync, rmSync, existsSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { createDefaultDatasourceDriverFactory } from '../default-datasource-driver-factory.js'; @@ -76,3 +76,65 @@ describe('createDefaultDatasourceDriverFactory — mysql construction', () => { try { await handle.disconnect?.(); } catch { /* pool never opened */ } }); }); + +// [#4083] "memory" must mean memory. This branch used to construct a bare +// `new InMemoryDriver()`, dropping the declared `config` — and the driver's own +// `persistence: 'auto'` default writes a CWD-relative +// `.objectstack/data/memory-driver.json` under Node. A declared memory +// datasource was therefore file-backed and reloaded its rows on the next boot, +// with nothing disclosing it: the ADR-0062 acceptance test passed on a virgin +// checkout (and in CI, always a fresh one) then failed on every subsequent +// local run, its seeded rows accumulating two at a time. +describe('createDefaultDatasourceDriverFactory — memory construction (#4083)', () => { + const memoryDriver = async (config?: Record) => { + const handle: any = await factory().create({ driver: 'memory', ...(config ? { config } : {}) } as any); + return handle.driver ?? handle; + }; + + it('is ephemeral by default — no persistence adapter is installed', async () => { + const driver = await memoryDriver(); + expect(driver?.constructor?.name).toMatch(/InMemoryDriver$/); + // The adapter is built during connect(), not construction, so connect + // first — asserting on a fresh instance would pass vacuously. + await driver.connect?.(); + try { + // The adapter is what writes to disk; no adapter ⇒ nothing to write. + expect((driver as any).persistenceAdapter ?? null).toBeNull(); + } finally { + await driver.disconnect?.(); + } + }); + + it('leaves nothing on disk after a write', async () => { + const cwd = mkdtempSync(join(tmpdir(), 'os-mem-ds-')); + const prev = process.cwd(); + process.chdir(cwd); + try { + const driver = await memoryDriver(); + await driver.connect?.(); + await driver.bulkCreate?.('note', [{ id: 'n1', title: 'first' }]); + await driver.disconnect?.(); + // A CWD-relative `.objectstack` is exactly what the old default wrote. + expect(existsSync(join(cwd, '.objectstack'))).toBe(false); + } finally { + process.chdir(prev); + rmSync(cwd, { recursive: true, force: true }); + } + }); + + it('forwards an explicit persistence request from the declared config', async () => { + // The declared `config` reaching the driver at all is the other half of + // this fix — an author who wants a file can still say so. + const dir = mkdtempSync(join(tmpdir(), 'os-mem-explicit-')); + const persisted = await memoryDriver({ + persistence: { type: 'file', path: join(dir, 'store.json') }, + }); + await persisted.connect?.(); + try { + expect((persisted as any).persistenceAdapter ?? null).not.toBeNull(); + } finally { + await persisted.disconnect?.(); + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/services/service-datasource/src/default-datasource-driver-factory.ts b/packages/services/service-datasource/src/default-datasource-driver-factory.ts index d7cbc3e619..9a64827434 100644 --- a/packages/services/service-datasource/src/default-datasource-driver-factory.ts +++ b/packages/services/service-datasource/src/default-datasource-driver-factory.ts @@ -257,8 +257,30 @@ export function createDefaultDatasourceDriverFactory( } // memory + // + // [#4083] Ephemeral by default, and the declared `config` is forwarded — + // this was the one branch that dropped `spec.config` on the floor and + // constructed a bare `new InMemoryDriver()`. + // + // The driver's OWN default is `persistence: 'auto'`, which under Node + // means file persistence at a CWD-relative + // `.objectstack/data/memory-driver.json`. So a datasource declared + // `driver: 'memory'` silently became file-backed and reloaded its rows on + // the next boot: "memory" did not mean memory, nothing disclosed it, and + // two memory datasources in one process wrote through the same default + // path. It also contradicted the platform's own contract — `objectstack + // dev` reads an explicit memory driver as the user opting OUT of + // persistence (`resolveDefaultDevDbUrl`, cli/dev.ts) and persists via a + // sqlite file instead. + // + // The rule now matches the `sqlite-wasm` branch above: persistence + // follows an EXPLICIT request, never an implicit default. An author who + // wants a file says so — `config: { persistence: 'file' }`, or any other + // shape the driver accepts. const { InMemoryDriver } = await import('@objectstack/driver-memory'); - return toHandle(new InMemoryDriver()); + return toHandle(new InMemoryDriver({ + persistence: ('persistence' in cfg ? cfg.persistence : false) as never, + })); }, }; } diff --git a/packages/spec/src/data/driver/memory.zod.ts b/packages/spec/src/data/driver/memory.zod.ts index cd70ec19e3..b9f6802b82 100644 --- a/packages/spec/src/data/driver/memory.zod.ts +++ b/packages/spec/src/data/driver/memory.zod.ts @@ -216,8 +216,17 @@ export const MemoryConfigSchema = lazySchema(() => z.object({ * new InMemoryDriver({ persistence: false }) * // Custom adapter for serverless * new InMemoryDriver({ persistence: { adapter: upstashAdapter } }) + * + * **A datasource declared with `driver: 'memory'` is ephemeral unless this + * key is set (#4083).** `'auto'` is the default for a driver you construct + * yourself; the shared datasource factory + * (`createDefaultDatasourceDriverFactory`) passes `false` when the + * declaration says nothing, so a declared memory datasource cannot silently + * become a file-backed store in the process's working directory. That + * matches `objectstack dev`, which already reads an explicit memory driver + * as the user opting out of persistence. Set this key to opt back in. */ - persistence: MemoryPersistenceConfigSchema.or(z.literal(false)).default('auto').describe('Persistence configuration (defaults to auto-detect)'), + persistence: MemoryPersistenceConfigSchema.or(z.literal(false)).default('auto').describe('Persistence configuration (auto-detect by default; a declared memory datasource is ephemeral unless set)'), /** * Fields to index for faster lookups. From 0d3f911632ffaac22e8447f23d853a4e05c7aaca Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 10:11:20 +0000 Subject: [PATCH 2/3] docs(drivers): the Memory Driver section states what actually persists (#4083) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Data is lost when the process exits" was false for the snippet printed directly beneath it: `new InMemoryDriver()` defaults to `persistence: 'auto'`, which under Node writes `.objectstack/data/memory-driver.json` relative to the working directory and reloads it on the next boot. The two build paths now differ deliberately, so the section says so: a declared `{ driver: 'memory' }` datasource is ephemeral (this PR), while a directly constructed driver keeps the `'auto'` default. Also notes the shared default path between two `'auto'` drivers in one process, and shows `persistence: false` in the test-usage tip — the pitfall that made this repo's own acceptance test green only on a virgin checkout. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UvcnzFr5XBuvvAd7mosG54 --- content/docs/data-modeling/drivers.mdx | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/content/docs/data-modeling/drivers.mdx b/content/docs/data-modeling/drivers.mdx index 2955712ff6..7cb9b39216 100644 --- a/content/docs/data-modeling/drivers.mdx +++ b/content/docs/data-modeling/drivers.mdx @@ -353,20 +353,34 @@ equivalent age-based reap. ## Memory Driver The in-memory driver keeps records in plain in-process objects (queried via -[`mingo`](https://github.com/kofrasa/mingo)). Data is lost when the process exits. +[`mingo`](https://github.com/kofrasa/mingo)). It is the **last-resort fallback** in dev mode: `objectstack dev` prefers native SQLite (`better-sqlite3`), falls back to the pure-JS WASM SQLite driver if the native binary is unavailable, and only drops to the in-memory driver if WASM also fails to load. Set `OS_DATABASE_DRIVER=memory` to select it explicitly. +**Whether data survives the process depends on how the driver is built** (#4083): + +| How it is built | Persistence | +| :--- | :--- | +| A **declared datasource** — `{ driver: 'memory' }` in a stack/app config | **Ephemeral.** Nothing is written to disk unless the declaration sets `config.persistence`. | +| `new InMemoryDriver()` **constructed directly** | `persistence: 'auto'` — under Node that means a JSON file at `.objectstack/data/memory-driver.json`, relative to the process's working directory, reloaded on the next boot. | + +Pass `persistence: false` for a driver you construct yourself and want purely in +memory — a test, for instance, which should neither depend on nor leave behind +state in the working directory. Note that two `'auto'` drivers in one process +share that one default path. + ```typescript import { InMemoryDriver } from '@objectstack/driver-memory'; -new InMemoryDriver(); +new InMemoryDriver({ persistence: false }); // pure memory +new InMemoryDriver(); // 'auto' — file-backed under Node ``` -Use the memory driver for unit tests. It requires no setup and runs instantly. +Use the memory driver for unit tests. It requires no setup and runs instantly — +with `persistence: false`, so one run cannot see what an earlier run left behind. ## Local Environment Runtime From a3d68506bf8ba6eb1ebc416511482579b02c955e Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 10:30:11 +0000 Subject: [PATCH 3/3] docs(drivers,spec): say which memory-driver path persists, now that only one of them does (#4083) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #4111 made a declared `{ driver: 'memory' }` datasource ephemeral and scoped an explicitly-requested destination per datasource. It left the documentation describing the old world, in the one place a reader looks first: - `drivers.mdx` claimed "Data is lost when the process exits" directly above a `new InMemoryDriver()` snippet — still false for that path, which keeps the `'auto'` default and writes `.objectstack/data/memory-driver.json` relative to the working directory. The section now contrasts the two build paths, notes that two directly-constructed `'auto'` drivers still share that one path, and shows `persistence: false` in the test-usage tip — the exact pitfall that let this repo's own ADR-0062 acceptance test pass on a virgin checkout and fail on every later local run. - `MemoryConfigSchema.persistence` documented `'auto'` as *the* default, with no hint that the datasource factory now overrides it. Its TSDoc and `.describe()` state the split, so the contract says what the runtime does rather than leaving two de-facto answers. Documentation only — this branch's earlier code changes for #4083 were dropped in favour of #4111, which landed the same fix plus per-pool store scoping. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UvcnzFr5XBuvvAd7mosG54 --- content/docs/data-modeling/drivers.mdx | 6 +++--- packages/spec/src/data/driver/memory.zod.ts | 12 +++++++----- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/content/docs/data-modeling/drivers.mdx b/content/docs/data-modeling/drivers.mdx index 7cb9b39216..ffb5eb50ec 100644 --- a/content/docs/data-modeling/drivers.mdx +++ b/content/docs/data-modeling/drivers.mdx @@ -363,13 +363,13 @@ fails to load. Set `OS_DATABASE_DRIVER=memory` to select it explicitly. | How it is built | Persistence | | :--- | :--- | -| A **declared datasource** — `{ driver: 'memory' }` in a stack/app config | **Ephemeral.** Nothing is written to disk unless the declaration sets `config.persistence`. | +| A **declared datasource** — `{ driver: 'memory' }` in a stack/app config | **Ephemeral.** Nothing is written to disk unless the declaration sets `config.persistence` — and when it does, the destination is scoped per datasource, so two memory datasources never share one file. | | `new InMemoryDriver()` **constructed directly** | `persistence: 'auto'` — under Node that means a JSON file at `.objectstack/data/memory-driver.json`, relative to the process's working directory, reloaded on the next boot. | Pass `persistence: false` for a driver you construct yourself and want purely in memory — a test, for instance, which should neither depend on nor leave behind -state in the working directory. Note that two `'auto'` drivers in one process -share that one default path. +state in the working directory. Two directly-constructed `'auto'` drivers in one +process still share that single default path. ```typescript import { InMemoryDriver } from '@objectstack/driver-memory'; diff --git a/packages/spec/src/data/driver/memory.zod.ts b/packages/spec/src/data/driver/memory.zod.ts index b9f6802b82..232c486b5a 100644 --- a/packages/spec/src/data/driver/memory.zod.ts +++ b/packages/spec/src/data/driver/memory.zod.ts @@ -218,13 +218,15 @@ export const MemoryConfigSchema = lazySchema(() => z.object({ * new InMemoryDriver({ persistence: { adapter: upstashAdapter } }) * * **A datasource declared with `driver: 'memory'` is ephemeral unless this - * key is set (#4083).** `'auto'` is the default for a driver you construct - * yourself; the shared datasource factory + * key is set (#4083).** `'auto'` is the default only for a driver you + * construct yourself. The shared datasource factory * (`createDefaultDatasourceDriverFactory`) passes `false` when the * declaration says nothing, so a declared memory datasource cannot silently - * become a file-backed store in the process's working directory. That - * matches `objectstack dev`, which already reads an explicit memory driver - * as the user opting out of persistence. Set this key to opt back in. + * become a file-backed store in the process's working directory — matching + * `objectstack dev`, which already reads an explicit memory driver as the + * user opting out of persistence. Set this key to opt back in; when you do + * without naming a `path`/`key`, the factory scopes the destination per + * datasource, so two memory datasources never share one file. */ persistence: MemoryPersistenceConfigSchema.or(z.literal(false)).default('auto').describe('Persistence configuration (auto-detect by default; a declared memory datasource is ephemeral unless set)'),