fix(service-automation): a durable PAUSED run is visible to listRuns and run-detail after a cold restart (#8050)

[huangyiirene]

Closes#8050.

What was wrong

sys_automation_run holds two disjoint row families:

family	id	status	written by	lifetime
terminal history	run_ + runId	completed / failed	recordTerminal	tombstone, capped + age-swept
live suspension	the raw runId	paused	save	deleted on completion, exempt from the sweep

AutomationEngine.listRuns merged the in-memory ring buffer with the first
family only, and getRun fell back to the first family only. So after a
process restart, a run parked at an approval / screen / wait node answered:

read	before	after
GET /automation/:name/runs	200, zero rows	the parked run
GET /automation/:name/runs?status=paused	200, zero rows	the parked run
GET /automation/:name/runs/:runId	404RESOURCE_NOT_FOUND	200, status: 'paused'

…while the same run served …/runs/:runId/screen and resumed cleanly. Before a
restart the gap is invisible, because a paused run is still in the ring — which
is why every existing in-process listRuns test passes on main.

The sharp edge is the filter. #7359 had just made ?status=paused a real filter,
and with no post-restart producer of a paused entry it could never match a
row — so the one query an operator reaches for when asking "what is in flight?"
was structurally guaranteed to answer "nothing pending".

What changed

Both reads consult the suspension rows, through one rehydration
(suspendedRunToLogEntry) that reproduces the entry the two status: 'paused'
recordLog sites write:

• trigger is rebuilt via the buildRunTrigger chokepoint on the persisted
  context_json, not from the flattened trigger_* columns — those exist
  for filtering (trigger.recordId is never populated on record_change runs, and the persisted sys_automation_run row carries no trigger block at all — runs cannot be correlated to their triggering record, and trigger kinds are lost across restart #7533) and spell an absent type null where the log entry says
  'manual'.
• variables is carried, because finding: a suspended flow run's variables are unreadable on run-detail — the paused recordLog omits them while the continuation keeps them #7639 made it part of what a paused run
  discloses on run-detail. Dropping it would have re-opened that card for exactly
  the runs that need it most.
• durationMs / completedAt are absent: a suspension row records when the run
  started, not when it parked.

getRun gains the paused fallback after the terminal probe, and listRuns
merges weakest-source-first — durable paused → durable history → in-memory
ring. That order is a claim, not an accident: a paused row is the only source
that can be stale (its delete on completion is best-effort), so a terminal row or
ring entry for the same id is later evidence and wins. Letting paused win would
re-introduce #3456's "paused forever". There is no symmetric hazard — within a
process the ring is written in the same breath as the paused row, and across a
restart the ring is empty.

What did not change

• No persistence change. Suspension rows keep their id space, lifecycle and
  retention exemption, and are not reshaped into history rows.
• Durability was never the defect. Independently re-measured: a parked run
  survives a cold boot over the same sqlite file, still resumes to completion,
  and both surfaces then report it terminal.
• The refusal envelope.store.load answers null for an unknown id exactly
  as loadTerminal does, so a genuinely nonexistent run id still reaches
  deps.error('Execution not found', 404).

Reverse verification

Every case was run against origin/main with only the engine change reverted
(test file unchanged): 8 red / 7 green.

	case	reading on main
🔴	cold restart: ?status=paused returns the parked run	[]
🔴	cold restart: bare enumeration returns the parked run	[]
🔴	cold restart: run-detail answers	null
🔴	cold restart: trigger attribution + #7639 variables	null
🔴	cold restart: no cross-flow leak	[]
🔴	cold restart: ?status= narrows without widening	[] (len 0, want 1)
🔴	an unreadable paused store degrades rather than throws	no such warning
🔴	full stack, cold boot over the same sqlite FILE	[] / null
🟢	pre-restart: the run is in both sources and lists once	guards dedupe
🟢	pre-restart: another flow's paused run does not leak in	guards the flow filter
🟢	a stale paused row cannot mask a finished run (ring side)	guards precedence
🟢	a stale paused row cannot mask a finished run (durable side, cold)	guards precedence
🟢	a parked run still resumes to completion after a restart	guards durability
🟢	an unknown run id is still not found	guards the 404 envelope
🟢	a flow with no paused rows behaves exactly as before	guards the pre-#8050 path

Three cases drafted as green measured red, and the labels — not the tests —
were wrong: each asserts the row is there before asserting anything about its
scoping, so none can pass on a tree where it never appears. They are recorded as
red with the extra invariant each carries named on the case, and a genuinely
green pre-restart scoping twin was added alongside.

The trap this avoids: parking a run and enumerating it in the same process is
green on main too. The gate is park → cold restart → read.

Test edited (called out deliberately)

suspended-run-store.test.ts — one assertion inverted, none deleted.

-expect(await cold.getRun(paused.runId!)).toBeNull();+expect((await cold.getRun(paused.runId!))?.status).toBe('paused');

It sat inside a hasSuspendedRun case as a contrast ("the case getRun
cannot answer"). That contrast is the defect — it is what made run-detail 404
for a healthy parked run. Inverted rather than removed, so the pair stays pinned
together, plus a new case for the distinction that survives: hasSuspendedRun
rejects on an unreadable store (it backs a write decision) where getRun degrades
to null (it is an observability read).

Two now-false doc comments in plugin-approvals that asserted the old contrast
were corrected. No behaviour changes there: inspectStrandedRequests gates on
hasSuspendedRun === false first, and releaseDeadRunRequests treats null and
paused through the same !TERMINAL_RUN_STATUSES.has(status) branch.

Gates

gate	reading
service-automation suite	955 passed / 80 files
plugin-approvals suite	458 passed / 21 files
runtime domain suite	481 passed / 24 files
check-test-source-alias	OK — 72 packages scanned, registry unchanged (no new entry; the new test adds no workspace dep the package's tests did not already import)
check-type-check-coverage	OK — 64/77 type-checked, 13 ledgered
check-type-check-coverage --re-measure	OK — 33 entries, none above its ceiling
tsc --noEmit (service-automation)	3 errors, all pre-existing in nested-region-parity.test.ts; the new file adds 0 (ceiling 5)
eslint on changed files	clean

The re-measure reports a pre-existing 270-error surplus across 9 entries
(metadata, service-storage, plugin-auth, mcp, lint, … — mostly packages
untouched here). Not lowered: out of scope, and the gate states lowering is
optional.

content/docs/releases/**, docs/adr/** and the skills trees are untouched.
Changeset: .changeset/paused-run-visibility-after-restart.md.

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
objectstack	Ignored		Aug 12, 2026 6:03pm

[github-actions]

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/plugin-approvals, @objectstack/service-automation.

4 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

• content/docs/automation/approvals.mdx(via @objectstack/plugin-approvals)
• content/docs/automation/flows.mdx(via @objectstack/service-automation)
• content/docs/kernel/services-checklist.mdx(via @objectstack/plugin-approvals, @objectstack/service-automation)
• content/docs/plugins/packages.mdx(via @objectstack/plugin-approvals, @objectstack/service-automation)

⛔ 2 release-owned page(s) also reference the affected code. These are read-only:

• content/docs/releases/implementation-status.mdx(via @objectstack/plugin-approvals, @objectstack/service-automation)
• content/docs/releases/v9.mdx(via @objectstack/plugin-approvals, @objectstack/service-automation)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.