diff --git a/.changeset/localization-success-read-cache.md b/.changeset/localization-success-read-cache.md new file mode 100644 index 0000000000..d92750aa7f --- /dev/null +++ b/.changeset/localization-success-read-cache.md @@ -0,0 +1,58 @@ +--- +"@objectstack/core": minor +--- + +feat(core): cache successful `sys_setting` localization reads, invalidated synchronously on write (#11966) + +Leg C (ship-first) of the accepted #11633 cross-request caching design +(maintainer acceptance 2026-08-25, forks 1A / 2B / 3A / TTL-0). +`resolveLocalizationContext` re-read `sys_setting` on **every** authenticated +request to answer the same three keys — `timezone` / `locale` / `currency` — +for a workspace whose values change roughly never. That read is now cached. + +**Grade: `minor`, not `patch`.** It adds a deployment variable +(`OS_LOCALIZATION_CACHE_TTL_MS`) and changes the query pattern of a shipped code +path. Not `major`: the observable contract callers actually depend on — a +settings write is visible to the very next read — is preserved, and pinned. + +Caching this read was tried once before and reverted. #10221's first version +memoized every outcome for 30s and CI went red on +`analytics-timezone.dogfood.test.ts`, which writes a new org timezone and +expects the very next analytics query to bucket under it; the cache was narrowed +to memoize **failures** only. That verdict was on **TTL-only** caching and it +still stands unamended. What changed is that the process now has invalidation +seams it did not have then: + +- **Primary — the settings change seam.** `SettingsService.subscribe(ns, handler)` + dispatches synchronously and in-process from the write path, after the row is + persisted. (⚠️ #11633 calls this a "settings change bus"; no such module + exists — `subscribe()` is the seam. No change was needed in + `@objectstack/service-settings`: the seam was already public and already does + exactly this.) +- **Backstop — the engine write epoch** from #11968's substrate. Needed because + this resolver's own fallback reads `sys_setting` *directly*, so a seeder or + any other direct engine write emits no settings event at all. It is read + structurally rather than imported, because `@objectstack/objectql` depends on + `@objectstack/core` and the substrate declared `WriteEpochLike` separately for + exactly this consumer. A peer node's hint arrives as a local bump, so an + attached `authz.invalidated` bridge narrows cross-node convergence for free. +- **TTL** — the residual bound, for what neither seam can see. Default 30s, + `0` disables the cache on a real path rather than a degenerate one. + +Two rules carry the change and are pinned rather than merely documented: + +1. **A success is cached only when the engine exposes the write epoch.** A `ql` + with no seam is a `ql` whose writes the cache cannot observe, so rather than + degrade to the TTL-only shape that was already reverted here once, the cache + declines. A partial `{ current }` shape is not a seam either — a counter + nothing can bump would read as a live invalidation source and pin the answer + for a whole TTL. +2. **Invalidation retires success entries only.** #10221's failure memo exists + for an environment where `sys_setting` is missing; retiring it on a write + would restart precisely the per-request driver log spam that memo removed, + and no write can create a missing table. It stays TTL-bound and behaves + exactly as #10221/#11877 shipped it. + +`analytics-timezone.dogfood.test.ts` is unchanged and unweakened — it is this +leg's acceptance test, and an ablation that reduces the cache to its TTL turns +it red on the same assertion the original revert was recorded against. diff --git a/content/docs/deployment/environment-variables.mdx b/content/docs/deployment/environment-variables.mdx index d56336f68c..e2216400f5 100644 --- a/content/docs/deployment/environment-variables.mdx +++ b/content/docs/deployment/environment-variables.mdx @@ -355,6 +355,7 @@ the hosted ObjectOS Cloud control plane. | `OS_SANDBOX_HOOK_TIMEOUT_MS` | number | `250` | Default **CPU-time** budget for a sandboxed **hook** body (QuickJS, ADR-0102): how much *VM-active* time a body may burn — idle host-await time and a nested hook's own run are NOT charged. A loaded/slow host rarely needs to raise this now (it is not wall-clock), but the knob remains. Only a positive integer is honored; unset / non-numeric / non-positive keeps the 250ms default. A hook body's own declared `timeoutMs` still wins over this. | | `OS_SANDBOX_ACTION_TIMEOUT_MS` | number | `5000` | Default **CPU-time** budget for a sandboxed **action** body (QuickJS). Same resolution rules as the hook variant above (positive integer only; an action body's own `timeoutMs` still wins). | | `OS_SANDBOX_WALL_CEILING_MS` | number | `30000` | Wall-clock ceiling (ADR-0102) — the backstop that cuts a hook/action body stuck on a host call that never settles (which burns no CPU, so the CPU budget alone would never fire). The effective ceiling is `max(this, cpuBudget)`, so it can never cut a body still inside its CPU budget. Positive integer only; unset keeps 30s. | +| `OS_LOCALIZATION_CACHE_TTL_MS` | number | `30000` | Staleness bound, in milliseconds, for the cross-request cache of a workspace's reference localization (`timezone` / `locale` / `currency`, read from `sys_setting`) — leg C of #11633. `0` means **off**, a real path that restores the uncached query pattern exactly. Unlike `OS_AUTHZ_GRANTS_CACHE_TTL_MS` (which is off by default) this one ships **on**, because its invalidation is synchronous and in-process rather than TTL-bound: a `localization` settings change and any engine write both retire a cached answer immediately, so the TTL only bounds what neither seam can see — a write made on another replica with no `authz.invalidated` bridge attached. ⚠️ A malformed value reads as `0` (off), the opposite arm from the grants variable and deliberately so: there `0` is also the default, whereas here folding `3OOO` (letter O) into the default would hand you a **longer** window than the one you were setting. Deployment config only — never a settings row, because `sys_setting` is the table this cache caches. | | `OS_INLINE_SEED_BUDGET_MS` | number | `8000` | Time budget for synchronous seed execution at boot before deferring to a worker. | | `OS_TENANT_AUDIT` | flag | `1` | Set to `0` to silence the tenant-isolation audit warnings emitted by the SQL driver. | diff --git a/packages/core/src/security/resolve-authz-context.test.ts b/packages/core/src/security/resolve-authz-context.test.ts index 3f1135a743..7f544cd0d2 100644 --- a/packages/core/src/security/resolve-authz-context.test.ts +++ b/packages/core/src/security/resolve-authz-context.test.ts @@ -396,7 +396,10 @@ function makeMissingTableQl() { // The #2409 batching above already collapsed one request down to a single // query; this collapses the FAILING query across requests with a short TTL // cache — but ONLY the failure, never a successful (or legitimately empty) -// read. A first version cached every outcome, mirroring +// read on a `ql` that offers no way to learn a write happened. (#11966 later +// added the success cache behind exactly that seam; the failure memo below is +// untouched by it, and a dedicated pin in `resolve-localization-cache.test.ts` +// holds it untouched — retiring it on a write would restart this very spam.) A first version cached every outcome, mirroring // `packages/plugins/plugin-audit/src/audit-writers.ts` (`resolveWriteLocale`)'s // existing memoization of this same read — that broke // `packages/qa/dogfood/test/analytics-timezone.dogfood.test.ts` (#1982/#2018), @@ -458,7 +461,15 @@ describe('resolveLocalizationContext — failure-only cross-request cache (#1022 // must never be served stale. Simulates the exact shape of the failing CI // scenario — a settings write changes the effective row between two calls — // entirely at this unit level, without booting the dogfood stack. - it('never caches a successful read: a value change between two calls is visible on the very next call', async () => { + // + // [#11966] `makeCountingQl` carries no write-epoch seam, and that is now + // load-bearing rather than incidental: leg C caches a success ONLY behind an + // engine that can tell it a write happened, so this double pins the + // seam-ABSENT arm — where the pre-#11966 multiset must survive byte for byte. + // The seam-PRESENT arm is `resolve-localization-cache.test.ts`, which pins + // the same staleness property through the invalidation instead of through the + // absence of a cache. + it('without an engine write-epoch seam, a successful read is never cached: a value change between two calls is visible on the very next call', async () => { const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]; const ql = makeCountingQl({ sys_setting: rows }); const first = await resolveLocalizationContext({ ql, tenantId: 'o1' }); @@ -472,10 +483,11 @@ describe('resolveLocalizationContext — failure-only cross-request cache (#1022 }); // A legitimate empty result (table exists, no settings configured for this - // tenant yet) is a successful read too — not a failure — so it must not be - // cached either: the first write for a previously-unconfigured tenant must - // be visible on the very next call, same as the value-change case above. - it('never caches a legitimate empty result: the first write for a previously-unconfigured tenant is visible immediately', async () => { + // tenant yet) is a successful read too — not a failure — so on a seam-less + // `ql` it must not be cached either: the first write for a previously- + // unconfigured tenant must be visible on the very next call, same as the + // value-change case above. (#11966: same seam-absent arm as that case.) + it('without a seam, a legitimate empty result is not cached either: the first write for a previously-unconfigured tenant is visible immediately', async () => { const rows: Array<{ namespace: string; key: string; scope: string; value: string }> = []; const ql = makeCountingQl({ sys_setting: rows }); const first = await resolveLocalizationContext({ ql, tenantId: 'o1' }); diff --git a/packages/core/src/security/resolve-authz-context.ts b/packages/core/src/security/resolve-authz-context.ts index 475e6a6166..3423c2b235 100644 --- a/packages/core/src/security/resolve-authz-context.ts +++ b/packages/core/src/security/resolve-authz-context.ts @@ -797,7 +797,205 @@ type LocalizationResult = { timezone: string; locale: string; currency?: string * entirely — there is no query to dedupe in that case. */ const LOCALIZATION_FAILURE_CACHE_TTL_MS = 30_000; -const localizationFailureCache = new WeakMap>(); + +/** + * ── Leg C of #11633: the SUCCESS side of this same cache (#11966) ─────────── + * + * The docblock above records why a successful read was NOT cached: a 30s TTL + * is far longer than the gap between a settings write and the next request, so + * `analytics-timezone.dogfood.test.ts` went red. That verdict was on **TTL-only + * caching** and it still stands, unamended. What changed is that this process + * now has invalidation seams it did not have then, so a successful answer can + * be bounded by a WRITE rather than by a clock: + * + * 1. **Primary — the settings change seam.** `SettingsService.subscribe(ns, + * handler)` dispatches SYNCHRONOUSLY and in-process from the write path, + * and it does so AFTER the row is persisted (`settings-service.ts` calls + * `emitChange` below its `await this.upsertRow(...)`). One subscription per + * settings occupant advances a generation counter; an entry resolved at an + * older generation is dead on arrival. + * ⚠️ There is **no module called a "settings change bus"** — #11633's term + * for this seam maps to nothing in the tree. `subscribe()` is the seam. + * 2. **Backstop — the engine write epoch** (#11968's substrate, declared in + * `objectql/src/write-epoch.ts`). Needed because this resolver's own + * fallback reads `sys_setting` DIRECTLY, so a seeder — or any other direct + * engine write — emits no settings event at all. Read STRUCTURALLY, never + * by import: `@objectstack/objectql` depends on this package, so the edge + * cannot be reversed, and the substrate declared `WriteEpochLike` + * separately for exactly this consumer. + * 3. **TTL** — the residual bound, covering only what neither seam can see: a + * peer node's write, on a deployment with no `authz.invalidated` bridge + * attached. With a bridge, a peer's hint bumps the LOCAL epoch + * (`authz-invalidation-bridge.ts` calls `epoch.bump('remote')`), so the + * backstop narrows cross-node convergence for free. + * + * ⭐ **A success is cached ONLY when the engine exposes the write epoch.** That + * is the load-bearing rule of this change. It is a rule about the CACHE, not + * about the caller: a `ql` with no seam is a `ql` whose writes this cache + * cannot see, and leg C's ruled requirement is that invalidation be + * synchronous and in-process — "a TTL alone does not satisfy it". So instead of + * degrading to the TTL-only shape that was already reverted once here, the + * cache declines. Every existing test double takes that path and keeps its + * exact query multiset; only a real engine caches. + * + * ⛔ **Invalidation retires SUCCESS entries only.** Dropping failure entries on + * a write would hand #10221 straight back: on the environment that memo exists + * for, `sys_setting` is missing, so a write to ANY object would retire the memo + * and the failing query — with the driver's log line behind it — would resume + * repeating once per request. No write can create a missing table, so there is + * nothing there for a write to correct; the failure memo stays purely + * TTL-bound and behaviourally identical to what #10221/#11877 shipped. + */ +const LOCALIZATION_CACHE_TTL_ENV = 'OS_LOCALIZATION_CACHE_TTL_MS'; +const LOCALIZATION_SUCCESS_CACHE_DEFAULT_TTL_MS = 30_000; + +/** + * Staleness bound for the success cache, in ms. `0` disables it — a real path + * that restores the pre-#11966 query multiset exactly, not a degenerate TTL. + * + * Deployment config, never a settings row (#11633 §5): `sys_setting` is the + * table this cache caches, so a knob living there would be served BY the cache + * it governs. + * + * ⚠️ A malformed value resolves to `0` (off), which is the OPPOSITE arm from + * `readAuthzGrantsCacheTtlMs`'s, and deliberately so. There, `0` is also the + * default, so malformed-means-off changes nothing. Here the default is ON, so + * the two candidate readings are "off" and "30s" — and folding `3OOO` (letter + * O) into the default would hand the operator a LONGER staleness window than + * the one they were trying to set. Off is the only arm whose failure mode is a + * missed optimisation rather than an unasked-for window. + */ +function localizationSuccessCacheTtlMs( + env: Record = typeof process !== 'undefined' ? process.env : {}, +): number { + const raw = env[LOCALIZATION_CACHE_TTL_ENV]; + if (raw === undefined || raw.trim() === '') return LOCALIZATION_SUCCESS_CACHE_DEFAULT_TTL_MS; + const parsed = Number(raw.trim()); + if (!Number.isFinite(parsed) || parsed < 0) return 0; + return Math.floor(parsed); +} + +/** + * The engine's current write epoch, or `undefined` when this `ql` carries no + * such seam. Mirrors `isWriteEpochLike` from `@objectstack/objectql` rather + * than importing it — see point 2 of the docblock above for why the import is + * not available in this direction. + * + * ⚠️ The whole surface is checked, not just `current`. A bare + * `{ current: number }` on some unrelated double would otherwise read as a live + * invalidation seam and license caching against a counter that nothing ever + * bumps — precisely the state this guard exists to keep unreachable. + */ +function readWriteEpoch(ql: unknown): number | undefined { + if (!ql || typeof ql !== 'object') return undefined; + const epoch = (ql as { writeEpoch?: unknown }).writeEpoch; + if (!epoch || typeof epoch !== 'object') return undefined; + const seam = epoch as { current?: unknown; bump?: unknown; subscribe?: unknown }; + if ( + typeof seam.current !== 'number' || + typeof seam.bump !== 'function' || + typeof seam.subscribe !== 'function' + ) { + return undefined; + } + return seam.current; +} + +/** Per-settings-occupant invalidation state for the localization bucket. */ +interface LocalizationSettingsState { + /** Advanced by the occupant's change seam. Compared, never interpreted. */ + gen: number; +} + +const localizationSettingsStates = new WeakMap(); + +/** + * The state every call that passes NO settings occupant shares. A distinct + * object rather than `undefined` so entry validity stays one identity + * comparison: an answer resolved through a settings service must not be served + * to a call made without one, or through a different one. + */ +const localizationNoSettingsState: LocalizationSettingsState = { gen: 0 }; + +/** + * Fetch — and, on first sight of an occupant, subscribe to — the invalidation + * state for one settings service. + * + * #11633 §2.3: the change event is `{ namespace, key, scope, action, at }` with + * **no tenant discriminator**, so a handler cannot know whose entry to drop and + * the whole `localization` bucket has to go. A generation counter IS that drop, + * in O(1), without walking a Map from inside a change handler. + * + * The subscription is deliberately never disposed: it holds one integer per + * occupant, the occupant outlives this module's interest in it, and a disposer + * would need a shutdown hook a pure resolver does not have. `subscribe` is + * feature-detected — an occupant without the seam still gets the epoch backstop + * and the TTL, it just loses the precise trigger. + */ +function localizationSettingsState(settings: unknown): LocalizationSettingsState { + if (!settings || typeof settings !== 'object') return localizationNoSettingsState; + const existing = localizationSettingsStates.get(settings as object); + if (existing) return existing; + const state: LocalizationSettingsState = { gen: 0 }; + localizationSettingsStates.set(settings as object, state); + const subscribe = (settings as { subscribe?: unknown }).subscribe; + if (typeof subscribe === 'function') { + try { + (subscribe as (ns: string, handler: () => void) => unknown).call( + settings, + 'localization', + () => { + state.gen += 1; + }, + ); + } catch { + // An occupant whose seam refuses leaves the epoch backstop and the TTL. + } + } + return state; +} + +/** + * One entry per `ql` → `tenantId|userId`. ONE table, two lifecycles — the two + * docblocks above are why they must not collapse into a single rule: + * + * `failure` — #10221/#11877. TTL only; no invalidation ever retires it. + * `success` — #11966 / #11633 leg C. Retired by the settings seam, by the + * engine write epoch, or by the TTL — whichever comes first. + */ +interface LocalizationCacheEntry { + value: LocalizationResult; + expiresAt: number; + kind: 'failure' | 'success'; + /** `success` only: the engine write epoch this value was read at. */ + epoch?: number; + /** `success` only: the settings occupant this value was read through. */ + settings?: LocalizationSettingsState; + /** `success` only: that occupant's generation at read time. */ + settingsGen?: number; +} + +const localizationCache = new WeakMap>(); + +/** + * The invalidation half of "is this entry still the answer?" — the caller + * checks `expiresAt` separately, because the TTL applies to both kinds and + * these rules apply to one. + */ +function localizationEntryIsLive( + entry: LocalizationCacheEntry, + epoch: number | undefined, + settings: LocalizationSettingsState, +): boolean { + if (entry.kind === 'failure') return true; + return entry.epoch === epoch && entry.settings === settings && entry.settingsGen === settings.gen; +} + +function putLocalizationEntry(ql: object, key: string, entry: LocalizationCacheEntry): void { + const bucket = localizationCache.get(ql) ?? new Map(); + bucket.set(key, entry); + localizationCache.set(ql, bucket); +} /** * Resolve workspace localization defaults (reference `timezone` / `locale` / @@ -809,26 +1007,68 @@ const localizationFailureCache = new WeakMap { - const { ql, tenantId, userId } = input; + const { ql, settings, tenantId, userId } = input; const cacheKey = `${tenantId ?? ''}|${userId ?? ''}`; - if (ql && typeof ql === 'object') { - const hit = localizationFailureCache.get(ql)?.get(cacheKey); - if (hit && hit.expiresAt > Date.now()) return hit.value; + const cacheable = Boolean(ql) && typeof ql === 'object'; + + // ⭐ Both invalidation readings are taken BEFORE the resolve, and it is these + // pre-read values that get stored with the answer. A write landing WHILE this + // read is in flight therefore moves the epoch (or the generation) past what + // the entry records, so the entry is already dead when it is written — the + // safe direction. Reading them afterwards would stamp a pre-write value with + // a post-write epoch and make that staleness permanent: the + // clear-then-repopulate-from-a-stale-read failure #11633 §7 pin 2 names. + const epoch = cacheable ? readWriteEpoch(ql) : undefined; + const settingsState = localizationSettingsState(settings); + + if (cacheable) { + const hit = localizationCache.get(ql)?.get(cacheKey); + if (hit && hit.expiresAt > Date.now() && localizationEntryIsLive(hit, epoch, settingsState)) { + return hit.value; + } } const { value, backendFailed } = await resolveLocalizationContextUncached(input); - if (backendFailed && ql && typeof ql === 'object') { - const bucket = localizationFailureCache.get(ql) ?? new Map(); - bucket.set(cacheKey, { value, expiresAt: Date.now() + LOCALIZATION_FAILURE_CACHE_TTL_MS }); - localizationFailureCache.set(ql, bucket); + if (!cacheable) return value; + + if (backendFailed) { + putLocalizationEntry(ql, cacheKey, { + value, + expiresAt: Date.now() + LOCALIZATION_FAILURE_CACHE_TTL_MS, + kind: 'failure', + }); + return value; + } + + const ttlMs = localizationSuccessCacheTtlMs(); + if (epoch !== undefined && ttlMs > 0) { + putLocalizationEntry(ql, cacheKey, { + value, + expiresAt: Date.now() + ttlMs, + kind: 'success', + epoch, + settings: settingsState, + settingsGen: settingsState.gen, + }); + } else { + // Nothing to store — but the entry this read just superseded must not be + // left behind either. (Reaching here means no LIVE entry was found above, + // so this only ever drops a dead one.) + localizationCache.get(ql)?.delete(cacheKey); } return value; } diff --git a/packages/core/src/security/resolve-localization-cache.test.ts b/packages/core/src/security/resolve-localization-cache.test.ts new file mode 100644 index 0000000000..6549c8629a --- /dev/null +++ b/packages/core/src/security/resolve-localization-cache.test.ts @@ -0,0 +1,443 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// ── Leg C of #11633 (#11966): the SUCCESS side of the localization cache ──── +// +// The sibling pins in `resolve-authz-context.test.ts` cover #10221/#11877 — the +// FAILURE memo — and they still hold unchanged. This file covers the success +// cache and, above everything else, the one property the ruling made hard: +// **invalidation is synchronous and in-process**, so a read after a write +// observes the write. A TTL alone does not satisfy that, and a TTL alone is the +// exact shape that was already reverted once on this function. +// +// Two seams do the invalidating, and both are pinned here in BOTH directions — +// that it fires when it should, and that the cache actually caches when nothing +// fired (an "invalidation works" pin passes trivially on a cache that never +// caches, which is why every staleness pin below is paired with a hit pin): +// +// 1. `SettingsService.subscribe('localization', ...)` — the primary. ⚠️ There +// is no module called a "settings change bus"; `subscribe()` IS the seam. +// 2. The engine write epoch (#11968) — the backstop, for writes that reach +// `sys_setting` without passing through the settings service at all. + +import { describe, it, expect, vi, afterEach } from 'vitest'; +import { resolveLocalizationContext } from './resolve-authz-context.js'; + +const TTL_ENV = 'OS_LOCALIZATION_CACHE_TTL_MS'; + +/** Rows the direct `$in` fallback reads, plus a per-object query counter. */ +function makeQl(rows: Array>, opts: { epoch?: boolean } = {}) { + const counts = { sys_setting: 0 }; + const listeners = new Set<(epoch: number, reason: string) => void>(); + let epoch = 0; + const writeEpoch = { + get current() { + return epoch; + }, + bump(reason: string) { + epoch += 1; + for (const l of [...listeners]) l(epoch, reason); + return epoch; + }, + subscribe(l: (e: number, r: string) => void) { + listeners.add(l); + return () => listeners.delete(l); + }, + }; + const ql: Record = { + counts, + async find(_object: string, o: any) { + counts.sys_setting += 1; + const where = o?.where ?? {}; + const matched = rows.filter((r) => + Object.entries(where).every(([k, v]) => { + // REFUSE a combinator rather than matching it as a field name: a + // hand-written matcher that reads `$and` as a column silently answers + // the wrong question instead of failing (`check:where-matcher`). + if (k.startsWith('$')) throw new Error(`fake driver: unsupported operator ${k}`); + if (v && typeof v === 'object' && '$in' in (v as any)) return (v as any).$in.includes(r[k]); + return r[k] === v; + }), + ); + // [#10978] Hold the caller's bound, AFTER the filter and by PRESENCE — a + // double that hands back everything it matched cannot tell this read's + // `limit: 10` from no bound at all, so folding or dropping that bound + // would stay green here by construction. + return typeof o?.limit === 'number' ? matched.slice(0, o.limit) : matched; + }, + }; + // The seam is opt-in per double on purpose: the guard under test is "no seam + // ⇒ no success cache", so a double without one is a fixture, not an oversight. + if (opts.epoch !== false) ql.writeEpoch = writeEpoch; + return ql as typeof ql & { counts: typeof counts; writeEpoch: typeof writeEpoch }; +} + +/** A settings occupant carrying the real `subscribe(ns, handler)` seam. */ +function makeSettings(values: Record) { + const subs = new Set<{ ns?: string; handler: (e: unknown) => void }>(); + const counts = { reads: 0 }; + return { + counts, + subscribe(ns: string | undefined, handler: (e: unknown) => void) { + const entry = { ns, handler }; + subs.add(entry); + return () => subs.delete(entry); + }, + async get(_ns: string, key: string) { + counts.reads += 1; + return values[key] === undefined ? undefined : { value: values[key] }; + }, + async getMany(_ns: string, keys: string[]) { + counts.reads += 1; + const out: Record = {}; + for (const k of keys) out[k] = values[k] === undefined ? undefined : { value: values[k] }; + return out; + }, + /** Mirrors the real service: persist FIRST, then emit synchronously. */ + write(key: string, value: string) { + values[key] = value; + for (const s of [...subs]) { + if (s.ns && s.ns !== 'localization') continue; + s.handler({ namespace: 'localization', key, scope: 'tenant', action: 'set', at: '' }); + } + }, + }; +} + +describe('resolveLocalizationContext — success cache identity (#11633 §7 pin 1)', () => { + it('a second resolve returns a DEEP-EQUAL answer and issues ZERO reads', async () => { + const ql = makeQl([ + { namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'Asia/Tokyo' }, + { namespace: 'localization', key: 'locale', scope: 'tenant', value: 'ja-JP' }, + { namespace: 'localization', key: 'currency', scope: 'tenant', value: 'JPY' }, + ]); + const first = await resolveLocalizationContext({ ql, tenantId: 'o1' }); + const second = await resolveLocalizationContext({ ql, tenantId: 'o1' }); + // Both halves are load-bearing: equality alone passes on a cache that never + // caches, and a zero-read count alone passes on a cache that returns junk. + expect(second).toEqual(first); + expect(first).toEqual({ timezone: 'Asia/Tokyo', locale: 'ja-JP', currency: 'JPY' }); + expect(ql.counts.sys_setting).toBe(1); + }); + + it('keys per tenant and per `ql`, so no environment or tenant reads another\'s answer', async () => { + const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'Asia/Tokyo' }]; + const qlA = makeQl(rows); + const qlB = makeQl(rows); + await resolveLocalizationContext({ ql: qlA, tenantId: 't1' }); + await resolveLocalizationContext({ ql: qlA, tenantId: 't2' }); + await resolveLocalizationContext({ ql: qlA, tenantId: 't1' }); + expect(qlA.counts.sys_setting).toBe(2); + await resolveLocalizationContext({ ql: qlB, tenantId: 't1' }); + expect(qlB.counts.sys_setting).toBe(1); + }); +}); + +describe('resolveLocalizationContext — the settings seam invalidates synchronously', () => { + it('a `localization` write is observed by the very next resolve, with NO clock advance', async () => { + const ql = makeQl([]); + const settings = makeSettings({ timezone: 'UTC' }); + const first = await resolveLocalizationContext({ ql, settings, tenantId: 'o1' }); + expect(first.timezone).toBe('UTC'); + // Cached: a repeat issues no read at all. + await resolveLocalizationContext({ ql, settings, tenantId: 'o1' }); + expect(settings.counts.reads).toBe(1); + + settings.write('timezone', 'America/Los_Angeles'); + + // ⭐ Assert the END of the chain — the new VALUE — never "the cache was + // cleared". A clear-then-repopulate-from-a-stale-read implementation + // passes the second and fails this one (#11633 §7 pin 2's discipline). + const after = await resolveLocalizationContext({ ql, settings, tenantId: 'o1' }); + expect(after.timezone).toBe('America/Los_Angeles'); + expect(settings.counts.reads).toBe(2); + }); + + it('an occupant with no `subscribe` still resolves — it just loses the precise trigger', async () => { + const ql = makeQl([]); + const full = makeSettings({ timezone: 'UTC' }); + const { subscribe: _drop, ...seamless } = full; + const settings = seamless as unknown as typeof full; + const first = await resolveLocalizationContext({ ql, settings, tenantId: 'o1' }); + expect(first.timezone).toBe('UTC'); + // Still cached — the epoch backstop and the TTL are the remaining bounds. + await resolveLocalizationContext({ ql, settings, tenantId: 'o1' }); + expect(settings.counts.reads).toBe(1); + // ...and the backstop still retires it. + ql.writeEpoch.bump('write'); + await resolveLocalizationContext({ ql, settings, tenantId: 'o1' }); + expect(settings.counts.reads).toBe(2); + }); + + it('never serves an answer resolved through a DIFFERENT settings occupant', async () => { + const ql = makeQl([]); + const a = makeSettings({ timezone: 'Asia/Tokyo' }); + const b = makeSettings({ timezone: 'Europe/Paris' }); + expect((await resolveLocalizationContext({ ql, settings: a, tenantId: 'o1' })).timezone).toBe('Asia/Tokyo'); + expect((await resolveLocalizationContext({ ql, settings: b, tenantId: 'o1' })).timezone).toBe('Europe/Paris'); + }); +}); + +describe('resolveLocalizationContext — the engine write epoch is the backstop', () => { + it('a DIRECT `sys_setting` write, bypassing the settings service entirely, is observed at once', async () => { + // The case the backstop exists for: a seeder writes the row through the + // engine, so no settings event is ever emitted. + const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]; + const ql = makeQl(rows); + expect((await resolveLocalizationContext({ ql, tenantId: 'o1' })).timezone).toBe('UTC'); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(ql.counts.sys_setting).toBe(1); + + rows[0].value = 'America/Los_Angeles'; + ql.writeEpoch.bump('write'); + + expect((await resolveLocalizationContext({ ql, tenantId: 'o1' })).timezone).toBe('America/Los_Angeles'); + expect(ql.counts.sys_setting).toBe(2); + }); + + it('a peer node\'s hint (`remote`) retires the entry the same way a local write does', async () => { + // With the `authz.invalidated` bridge attached, a peer's write arrives as a + // local bump — so cross-node convergence rides the same backstop. + const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]; + const ql = makeQl(rows); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + rows[0].value = 'Asia/Tokyo'; + ql.writeEpoch.bump('remote'); + expect((await resolveLocalizationContext({ ql, tenantId: 'o1' })).timezone).toBe('Asia/Tokyo'); + }); +}); + +describe('resolveLocalizationContext — no seam, no success cache', () => { + it('a `ql` without a write epoch never caches a success (the pre-#11966 multiset, exactly)', async () => { + const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]; + const ql = makeQl(rows, { epoch: false }); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + rows[0].value = 'Asia/Tokyo'; + const second = await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(second.timezone).toBe('Asia/Tokyo'); + expect(ql.counts.sys_setting).toBe(2); + }); + + it('a PARTIAL epoch shape is not a seam — `{ current }` alone must not license caching', async () => { + // A counter nothing can bump is worse than no counter: it would read as a + // live invalidation source and pin the answer for the whole TTL. + const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]; + const ql = makeQl(rows, { epoch: false }); + (ql as Record).writeEpoch = { current: 0 }; + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + rows[0].value = 'Asia/Tokyo'; + expect((await resolveLocalizationContext({ ql, tenantId: 'o1' })).timezone).toBe('Asia/Tokyo'); + expect(ql.counts.sys_setting).toBe(2); + }); +}); + +describe('resolveLocalizationContext — the TTL is the residual bound (#11633 §7 pin 7)', () => { + const saved = process.env[TTL_ENV]; + afterEach(() => { + if (saved === undefined) delete process.env[TTL_ENV]; + else process.env[TTL_ENV] = saved; + }); + + it('`0` means OFF — a real path, and the query multiset returns to the uncached golden', async () => { + process.env[TTL_ENV] = '0'; + const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]; + const ql = makeQl(rows); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + rows[0].value = 'Asia/Tokyo'; + const second = await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(second.timezone).toBe('Asia/Tokyo'); + expect(ql.counts.sys_setting).toBe(2); + }); + + it('a MALFORMED value reads as off, never as the default — it must not widen the window', async () => { + // `3OOO` with letter O. Folding it into the 30s default would hand the + // operator a LONGER window than the one they were trying to set. + process.env[TTL_ENV] = '3OOO'; + const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]; + const ql = makeQl(rows); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + rows[0].value = 'Asia/Tokyo'; + expect((await resolveLocalizationContext({ ql, tenantId: 'o1' })).timezone).toBe('Asia/Tokyo'); + expect(ql.counts.sys_setting).toBe(2); + }); + + it('an entry expires on the configured bound even when no write ever happens', async () => { + vi.useFakeTimers(); + try { + const ql = makeQl([{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(ql.counts.sys_setting).toBe(1); + await vi.advanceTimersByTimeAsync(30_001); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(ql.counts.sys_setting).toBe(2); + } finally { + vi.useRealTimers(); + } + }); +}); + +// ── The card's own hard constraint: the two memos MUST NOT FIGHT ──────────── +// +// #10221's memo exists for an environment where `sys_setting` does not exist, +// so the read throws on every request and the driver logs a line every time. +// Invalidation must not touch it: a write to ANY object retiring that memo +// would restart exactly the log spam #10221 removed — and no write can create a +// missing table, so there is nothing there for a write to correct. +describe('resolveLocalizationContext — the failure memo survives what retires a success', () => { + function makeFailingQl() { + const counts = { sys_setting: 0 }; + let epoch = 0; + return { + counts, + writeEpoch: { + get current() { + return epoch; + }, + bump() { + epoch += 1; + return epoch; + }, + subscribe() { + return () => {}; + }, + }, + async find() { + counts.sys_setting += 1; + throw new Error('no such table: sys_setting'); + }, + }; + } + + it('an engine write does NOT retire a failure entry (#10221 log spam stays fixed)', async () => { + const ql = makeFailingQl(); + const settings = makeSettings({}); + expect(await resolveLocalizationContext({ ql, settings, tenantId: 'o1' })).toEqual({ + timezone: 'UTC', + locale: 'en-US', + currency: undefined, + }); + expect(ql.counts.sys_setting).toBe(1); + + ql.writeEpoch.bump(); + settings.write('timezone', 'Asia/Tokyo'); + + await resolveLocalizationContext({ ql, settings, tenantId: 'o1' }); + // Still ONE failing query: the memo is TTL-bound only, exactly as shipped. + expect(ql.counts.sys_setting).toBe(1); + }); + + it('the failure memo still self-heals on its own TTL once the migration lands', async () => { + vi.useFakeTimers(); + try { + const ql = makeFailingQl(); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(ql.counts.sys_setting).toBe(1); + await vi.advanceTimersByTimeAsync(30_001); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(ql.counts.sys_setting).toBe(2); + } finally { + vi.useRealTimers(); + } + }); + + it('a success recorded AFTER a failure replaces it, and is itself invalidatable', async () => { + // The two kinds share one key, so the later outcome must win cleanly in + // both directions — a stuck failure entry would pin the fallback forever. + const rows: Array> = []; + let broken = true; + const counts = { sys_setting: 0 }; + let epoch = 0; + const ql = { + counts, + writeEpoch: { + get current() { + return epoch; + }, + bump() { + epoch += 1; + return epoch; + }, + subscribe() { + return () => {}; + }, + }, + async find() { + counts.sys_setting += 1; + if (broken) throw new Error('no such table: sys_setting'); + return rows; + }, + }; + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(counts.sys_setting).toBe(1); + + // Migration lands; the memo is still standing, so nothing changes yet. + broken = false; + rows.push({ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'Asia/Tokyo' }); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(counts.sys_setting).toBe(1); + + // The memo expires, a SUCCESS is recorded in its place... + vi.useFakeTimers(); + try { + await vi.advanceTimersByTimeAsync(30_001); + expect((await resolveLocalizationContext({ ql, tenantId: 'o1' })).timezone).toBe('Asia/Tokyo'); + expect(counts.sys_setting).toBe(2); + await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(counts.sys_setting).toBe(2); + } finally { + vi.useRealTimers(); + } + + // ...and that success is retired by a write, like any other. + rows[0].value = 'Europe/Paris'; + ql.writeEpoch.bump(); + expect((await resolveLocalizationContext({ ql, tenantId: 'o1' })).timezone).toBe('Europe/Paris'); + }); +}); + +describe('resolveLocalizationContext — a write DURING the read must not be swallowed', () => { + it('an entry is stamped with the PRE-read epoch, so an in-flight write kills it on arrival', async () => { + // The clear-then-repopulate-from-a-stale-read failure, at its sharpest: the + // write lands after the query was issued and before the answer is stored. + // Stamping the entry with the post-read epoch would make that staleness + // permanent, invisible, and unbounded by any further event. + const rows = [{ namespace: 'localization', key: 'timezone', scope: 'tenant', value: 'UTC' }]; + const counts = { sys_setting: 0 }; + let epoch = 0; + const writeEpoch = { + get current() { + return epoch; + }, + bump() { + epoch += 1; + return epoch; + }, + subscribe() { + return () => {}; + }, + }; + let raceOnce = true; + const ql = { + counts, + writeEpoch, + async find() { + counts.sys_setting += 1; + const snapshot = rows.map((r) => ({ ...r })); + if (raceOnce) { + raceOnce = false; + // The concurrent write, mid-flight: the row changes and the seam + // advances AFTER this read's result was already determined. + rows[0].value = 'Asia/Tokyo'; + writeEpoch.bump(); + } + return snapshot; + }, + }; + const first = await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(first.timezone).toBe('UTC'); + // The stale answer must NOT be reusable — the next call re-reads. + const second = await resolveLocalizationContext({ ql, tenantId: 'o1' }); + expect(second.timezone).toBe('Asia/Tokyo'); + expect(counts.sys_setting).toBe(2); + }); +});