feat(runtime): add durable safe-boundary resume phases 0-2 - #1223

Merged
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2
Jul 19, 2026
Merged

feat(runtime): add durable safe-boundary resume phases 0-2#1223
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2

Conversation

@zhiiw

Copy link
Copy Markdown
Contributor

Summary

Implements the Runtime Resume foundation from Phase 0 through Phase 2, with RuntimeEvent as the single canonical recovery fact source.

  • Phase 0: fail-closed RuntimeEvent replay projection, unmatched tool-call parking, terminal repair, and real process-crash coverage.
  • Phase 1: safe-boundary continuation into fresh Run/Turn/Invocation identities, durable continuation claims, plan/execute separation, and execution-time revalidation.
  • Phase 2: sticky SQLite canonical RuntimeEvent storage, transactional T1/T2 tool boundaries, operation identity/CAS, import/export, and one canonical writer.
  • Phase 2.5 convergence: RecoveryResolver is the sole decision authority; tool journal tables are rebuildable projections, not a parallel truth source.

The Desktop manual recovery entry is attached to the interrupted-turn banner as Safe resume. CLI/TUI retain /resume. Park reasons are translated for users while stable reason codes remain available to diagnostics.

Related to #186.

Safety and rollout

Both capabilities remain disabled by default:

FlagBehavior
MAKA_RUNTIME_SQLITE_CANONICAL=1Triggers a sticky, one-way workspace migration to runtime.sqlite; disabling the flag does not return that workspace to JSONL.
MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1Enables safe-boundary recovery surfaces and Desktop startup auto-resume; these paths may call the provider and consume tokens.

Ambiguous tool outcomes are parked, never blindly retried. Phase 3 reconciliation is intentionally outside this PR.

Dogfooding and regression fixes

Real restart/resume testing found and fixed:

  1. second-generation continuation envelopes dropping source-only replay context;
  2. continuation replay/history reconstruction gaps across ancestor runs;
  3. continuation runs being misclassified as child-agent runs in model, UI, compaction, and repair projections;
  4. canonical continuation-start facts being treated as unsupported chat events;
  5. empty materialized continuation replay reaching the provider.

The final recovery gates also reject duplicate dispatch/response facts and unknown runtime protocol markers as corruption.

Known limitations

  • Automatic pre-migration backup is not implemented yet.
  • Populated SQLite v2-to-v4 upgrade coverage remains to be added.
  • Crash harness support targets Linux/macOS; Windows remains limited support.
  • definitely_not_dispatched is classified but its Phase 3 automated recovery policy is not implemented.
  • Legacy tool-projection rebuilding remains compatibility-only and must not become a second recovery truth source.
  • Manual and startup auto-resume still share one feature flag; they should be split before broader rollout.

Verification

  • npm run typecheck (all workspaces)
  • RecoveryResolver suite: 11 passed
  • empty continuation replay provider guard: passed
  • Desktop interrupted-banner routing and eligibility: 6 passed
  • user-facing recovery copy: 3 passed
  • changed-file Biome lint and git diff --check

@likun666661

likun666661 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Review conclusion

This is a novel and technically strong approach to runtime resume. The PR does not try to revive an old JavaScript stack, provider stream, Promise, or operating-system process. Instead, it reconstructs durable execution facts, proves that a continuation boundary is safe, and then starts a new Run, Invocation, and Turn.

The most important design choice is the distinction between replaying tool history and re-executing tool side effects:

  • completed function_call and function_response pairs are replayed to the provider as history;
  • the previous tool implementation is not executed again;
  • an unmatched or ambiguous tool call is parked rather than retried;
  • contradictory recovery facts fail closed.

This gives Maka a useful resume primitive without claiming exactly-once execution for arbitrary Bash commands, filesystem mutations, or external APIs.

Problem being solved

The existing RuntimeEvent ledger could answer what had been recorded, but it could not reliably answer whether a tool might already have crossed the side-effect boundary.

Two crash windows are particularly dangerous:

  1. The tool implementation starts before the canonical call fact is durably committed.
  2. The tool finishes its side effect, but the process crashes before the canonical response is durably committed.

In either case, treating a missing result as proof that the tool did not execute can duplicate writes, deletes, deployments, payments, or remote requests. The previous startup repair path could converge a dangling Run to a failed terminal state, but it could not safely continue execution.

This PR solves that by adding a durable tool-dispatch boundary, deterministic recovery projection, and safe-boundary continuation.

Module view

flowchart LR
subgraph Entry["Product entry points"]
Desktop["Desktop Safe resume and startup recovery"]
CLI["CLI and TUI /resume"]
end
subgraph Control["Recovery control plane"]
SessionManager["SessionManager<br/>candidate discovery and lifecycle"]
Inspector["ContinuationSafetyInspector<br/>cwd, workspace, tools, background work"]
Planner["RuntimeContinuationPlanner<br/>continue or park"]
Resolver["RecoveryResolver<br/>single decision authority"]
end
subgraph Execution["Execution plane"]
Kernel["RuntimeKernel<br/>revalidation and claim"]
AgentRun["AgentRun<br/>fresh execution identity"]
Runner["RuntimeRunner<br/>continuation-start fact"]
Backend["AiSdkFlow and AiSdkBackend<br/>provider replay"]
ToolRuntime["ToolRuntime<br/>preflight, T1, implementation, T2"]
end
subgraph Persistence["Persistence"]
Bootstrap["openRuntimeEventPersistence"]
SQLite[("runtime.sqlite<br/>canonical RuntimeEvent store")]
Projection["tool_journal_events and tool_operations<br/>rebuildable projections"]
JSONL["Legacy RuntimeEvent JSONL<br/>import and explicit export"]
end
Desktop --> SessionManager
CLI --> SessionManager
SessionManager --> Inspector
SessionManager --> Planner
Inspector --> Planner
SQLite --> Planner
Planner --> Resolver
Resolver --> Planner
Planner -->|"approved continuation"| Kernel
Planner -->|"park with stable reason codes"| Desktop
Kernel --> AgentRun
AgentRun --> Runner
Runner --> Backend
Backend --> ToolRuntime
Runner -->|"continuation-start"| SQLite
ToolRuntime -->|"T1 and T2"| SQLite
SQLite -.->|"synchronous projection"| Projection
Bootstrap --> SQLite
Bootstrap -.->|"unmigrated workspace and flag disabled"| JSONL
JSONL -->|"sticky, idempotent import"| SQLite
Loading

The architectural boundary is clear:

  • RuntimeEvent is the canonical recovery fact source.
  • RecoveryResolver is the only component that interprets tool recovery state.
  • Tool journal and operation tables are query projections and can be rebuilt from RuntimeEvents.
  • Product surfaces consume stable plans and reason codes instead of implementing their own recovery logic.

How the tool boundary closes the crash windows

sequenceDiagram
autonumber
participant Provider as Model Provider
participant Backend as AiSdkBackend
participant Runtime as ToolRuntime
participant Store as SqliteRuntimeStore
participant Tool as Tool Implementation
Provider->>Backend: function_call
Backend->>Runtime: execute tool
Runtime->>Runtime: validate args, availability, loop, permissions, and runtime guards
Runtime->>Store: T1 commitToolPrepared
Note over Store: Insert or verify function_call<br/>append toolDispatch RuntimeEvent<br/>update journal and operation projection
alt T1 fails
Store--xRuntime: rollback
Note right of Runtime: Tool implementation is never called
else T1 commits
Store-->>Runtime: durable dispatch boundary
Runtime->>Tool: execute implementation
Tool-->>Runtime: result or error
Runtime->>Store: T2 commitToolOutcome
Note over Store: Append function_response<br/>append outcome journal<br/>CAS operation state
alt T2 fails
Store--xRuntime: rollback
Note right of Runtime: Result is not returned to the next model step
else T2 commits
Store-->>Runtime: durable outcome
Runtime-->>Backend: tool_result
Backend-->>Provider: next provider step
end
end
Loading

This establishes two important invariants:

  1. If T1 fails, the tool implementation runs zero times.
  2. If T2 fails, the tool result cannot reach the next model step.

The external side effect still occurs between two short database transactions, so SQLite cannot make an arbitrary external operation atomic. Instead, a crash between T1 and T2 becomes an explicit indeterminate state that recovery must park or reconcile later.

How safe continuation works

sequenceDiagram
autonumber
participant Entry as User or Startup Recovery
participant Manager as SessionManager
participant Inspector as Safety Inspector
participant Planner as Continuation Planner
participant Resolver as RecoveryResolver
participant Store as Durable Stores
participant Kernel as RuntimeKernel
participant Provider as Model Provider
Entry->>Manager: resumeLatest
Manager->>Store: find latest failed or cancelled session-inline Run
Manager->>Inspector: inspect cwd, workspace identity, tool catalog, and background work
Inspector-->>Manager: authoritative safety observation
Manager->>Planner: plan source boundary
Planner->>Store: read Run header, RuntimeEvents, and continuation ancestors
Planner->>Resolver: resolve tool recovery facts
Resolver-->>Planner: completed, indeterminate, not dispatched, or corruption
alt Any safety condition fails
Planner-->>Manager: park with stable reason codes
Manager-->>Entry: explain why resume is unavailable
Note over Entry,Provider: Provider is not called
else Boundary is safe
Planner-->>Manager: continuation plan with fresh identities and safety snapshot
Manager->>Kernel: execute continuation
Kernel->>Store: re-read source terminal, high-water, identity, and replay
Kernel->>Inspector: revalidate external safety facts
Kernel->>Store: create target Run with continuationSource claim
Kernel->>Store: commit continuation-start RuntimeEvent
Kernel->>Provider: replay committed history without a duplicate user message
Provider-->>Kernel: continue from the validated boundary
end
Loading

Planning and execution are intentionally separate. A valid plan is not treated as an execution lease. Immediately before execution, the Runtime re-reads durable state and rechecks workspace identity, active operations, tool availability, source high-water, and replay equality.

The continuation also creates new execution identity:

source Session / Invocation / Run / Turn
-> validated RuntimeEvent high-water
-> new Invocation / Run / Turn
-> durable continuation-start
-> provider replay without a duplicate user message

The source ledger is not mutated by the continuation.

Recovery decisions

Durable factsResolver decisionCurrent behavior
call + matching responsecompletedReplay call and response as provider history
call + dispatch + no responseindeterminatePark; never blindly retry
call + no dispatch + new protocol markerdefinitely not dispatchedClassified, but currently still parked until Phase 3 policy exists
call + no dispatch + legacy or unknown protocolindeterminatePark
orphan, duplicate, or identity-conflicting factscorruptionFail closed

This is effectively tool-log replay: the provider receives the completed tool interaction as historical context, while the Runtime avoids replaying the old implementation or side effect.

What this PR introduces

AreaIntroduced capability
Recovery semanticsRecoveryResolver, deterministic tool-operation projection, stable diagnostics and park reasons
ContinuationRuntimeContinuationPlanner, fresh execution identities, durable continuation claim, execution-time revalidation
Tool protocoltoolDispatch RuntimeEvent, protocol marker, deterministic operation ID, canonical argument hash, recovery mode
Transaction boundaryRuntimeCommitSink, T1 commitToolPrepared, T2 commitToolOutcome, CAS updates
Canonical storageSQLite RuntimeEvent store, WAL, synchronous=FULL, foreign keys, schema migrations
CompatibilitySticky JSONL-to-SQLite import, source fingerprints, explicit JSONL export
Product surfacesDesktop Safe resume, startup continuation, CLI and TUI /resume, user-facing park copy
ValidationProcess-crash harnesses, replay tests, continuation tests, SQLite crash tests, and Linux/macOS CI coverage

Current boundaries

The conservative scope is appropriate, but it is important to preserve these limitations:

  • this PR does not re-execute historical tool implementations;
  • Phase 3 reconciliation and restricted verification are not implemented;
  • definitely_not_dispatched is classified but does not yet trigger automatic recovery;
  • general Bash and unknown external side effects remain parked;
  • replay-safe, idempotent, reconcile, and reattach tool contracts are future work;
  • SQLite migration is sticky and does not yet provide automatic pre-migration backup;
  • manual resume and startup auto-resume currently share one feature flag;
  • multi-worker leases and fencing are intentionally out of scope.

Future value of checkpoints and Git snapshots

The next major step that could make this resume system broadly practical is coupling the RuntimeEvent high-water with a workspace checkpoint, especially a Git-backed snapshot.

That would let recovery validate or restore both sides of the continuation boundary:

  1. the model and tool interaction history; and
  2. the exact agent-visible workspace state associated with that history.

A durable boundary containing:

RuntimeEvent high-water
+ workspace checkpoint reference
+ Git commit or tree snapshot
+ workspace identity

would allow Maka to resume after a crash without assuming that the current filesystem still matches the interrupted Run. It would also make planning and execution revalidation stronger because the continuation could prove that conversational state and workspace state refer to the same execution boundary.

Overall assessment

This is an excellent foundation and a genuinely interesting resume design.

The durable tool-call protocol makes safe history replay possible without conflating replay with side-effect re-execution. The fail-closed behavior is correct, the new execution identity keeps lineage understandable, and the single canonical recovery authority avoids parallel state machines.

With checkpoint and Git snapshot integration, this design could evolve from safe conversational continuation into a practical end-to-end runtime resume system.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

Conflict-resolution verification update

The branch was reconciled with current main through local merge commits 81edf2ba (Phase 1) and f0653e02 (Phase 2), then pushed without including local dogfooding files.

Additional compatibility fixes made during the merge:

  • aligned SQLite canonical storage with the current terminal durability barrier;
  • taught the exact-shape RuntimeEvent decoder about toolDispatch, runtimeProtocol, and operationId;
  • kept legacy JSONL import independent of a surviving run.json header;
  • preserved continuation invocation identity and session-inline projection semantics;
  • retained current steering, strict-recovery, child-agent, and Desktop IPC contracts.

Verification after reconciliation:

  • npm run build:test — passed across all workspaces;
  • npm run typecheck — passed across all workspaces;
  • Runtime resume/continuation/runner/tool-boundary focused suites — 62 passed;
  • SQLite runtime store — 10 passed;
  • legacy JSONL-to-SQLite transfer — 5 passed;
  • real-process Runtime resume crash harness — passed;
  • Desktop interrupted-banner routing — 4 passed;
  • focused SessionManager continuation/canonical/UI projection cases — 3 passed.

Windows-only file fsync/signal harness limitations remain documented; Linux/macOS are the supported crash-durability targets.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

CI follow-up is complete on head \

@zhiiw
zhiiw marked this pull request as ready for review July 19, 2026 10:40
@Astro-Han
Astro-Han merged commit b5e015a into apache:mainJul 19, 2026
5 checks passed
@zhiiw
zhiiw deleted the codex/runtime-resume-phase2 branch July 19, 2026 11:03
jackwener added a commit that referenced this pull request Jul 19, 2026
…nip zero-hint, resume hook, QR fixture (#1241)
* feat(core): add settings-bots-onboarding visual-smoke scenario + botOnboardingProvider
* feat(desktop): hold-in-waiting bot onboarding adapter + fixture state for QR capture
* feat(desktop): auto-open bot scan-login modal under settings-bots-onboarding fixture
* chore(scripts): register settings-bots-onboarding in capture + audit scenario lists
* fix(desktop): wait for QR paint before auto-capturing bot-onboarding fixture
* docs(notes): frontend architecture map — measured baseline + 8 staged rounds (2026-07-19)
Re-ran the census (wc over non-test TS/TSX for desktop main/preload/renderer +
packages/ui; CSS totals; top-10 hotspot table incl. visual-smoke-fixture.ts marked
OUT-OF-SCOPE other-sessions' territory). Staged R1 knip de-rot, R2 app-shell
resume-cluster extraction, R3 visual-smoke split (BLOCKED on concurrent fixture branch),
R4-R6 main.ts extractions (require maintainer-approved contract re-pins — registerIpc /
startup / tool-assembly / modelSupportsVision direct-pinned), R7 provider-connection-detail
decomposition (needs its own blade plan), R8 CSS raw-hex residue (verified clean on this
tip — #1085 already converted it).
* chore(knip): de-rot config — clear all 13 stale hints (R1)
knip reported 13 configuration hints (7 desktop + 6 ui), all redundant/stale config,
zero code impact:
- Drop 'overlayscrollbars' from desktop ignoreDependencies — now redundant (knip no
longer flags it; the package is owned by packages/ui and only referenced there). The
overlay-scrollbars contract forbids declaring it as a *desktop dependency*, not as an
ignore — removing a redundant ignore is compatible and coverage is unchanged.
- Delete the dead 'src/renderer/**/*.test.ts' entry glob (no matches — renderer has zero
test files; all 338 desktop tests live under src/main/__tests__/*.test.ts).
- Drop 11 redundant explicit entries knip already auto-detects: desktop
preload.ts / renderer/main.tsx / playwright.config.ts / .storybook/main.ts / dev.mjs
(vite / playwright / storybook / npm-scripts plugins), and all 6 ui entries
(index.ts, icons.tsx, artifact-preview-registry.ts, assistant-stream.ts, maka-uri.ts,
smooth-stream.ts) resolved from packages/ui package.json#exports.
Non-redundant entries kept verbatim (main.ts, overlay/*, main test glob, e2e specs,
storybook preview, stories, browser-observe-act-smoke.mjs, both ui test globs). No broad
ignores added — coverage is not weakened. knip --workspace apps/desktop and packages/ui
now both exit 0 AND report zero config hints (was: exit 0 with 13 hints).
* refactor(renderer): extract app-shell resume cluster into use-shell-resume (R2)
The #1223 safe-boundary resume cluster moves out of app-shell.tsx into a new
use-shell-resume.ts hook, following the use-shell-connections / use-shell-chat-model
house style (options object; state + handler returned; stable identities preserved).
Pure move, zero behavior change: the two useState declarations (resumePendingSessionId,
resumeParkDescriptionBySession) and the resumeInterruptedSession handler move verbatim;
activeId/toastApi/shellCopy/uiLocale are injected as options. app-shell keeps the banner
JSX wiring (safeResumeAction=) and the sendWithAttachments guard. resumeParkToastCopy is
no longer imported into app-shell (it now lives in the hook). app-shell.tsx 1686 -> 1654
(-32 lines).
Contract re-pin (never deletes coverage): runtime-resume-routing-contract.test.ts read
the handler-shape assertions (resumeInterruptedSession / resumeLatest(sessionId) /
resumeParkToastCopy / no rejectionReasons.join) from app-shell.tsx single-file; they now
read use-shell-resume.ts where that logic lives, while the app-shell-specific assertions
(sendWithAttachments '/resume' guard, safeResumeAction= wiring) still read app-shell.tsx.
use-shell-resume.ts added to renderer-shell-source-helpers sourcePaths (Round B/E
precedent) so combined-source contracts see it.
Gates: desktop 2740/2740, ui 196/196, typecheck 0, check-dead-css clean, knip
desktop+ui exit 0, AUDIT_PORT_BASE=23900 alignment auditor exit 0 (all 10 fixtures
clean, real renderer). CDP turn-narrative branch-vs-baseline (real Electron, light+dark
1280) byte-identical (sha256 match) — proven render no-op.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@zhiiw@likun666661@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat(runtime): add durable safe-boundary resume phases 0-2 - #1223

Merged
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2
Jul 19, 2026
Merged

feat(runtime): add durable safe-boundary resume phases 0-2#1223
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2

Conversation

@zhiiw

Copy link
Copy Markdown
Contributor

Summary

Implements the Runtime Resume foundation from Phase 0 through Phase 2, with RuntimeEvent as the single canonical recovery fact source.

  • Phase 0: fail-closed RuntimeEvent replay projection, unmatched tool-call parking, terminal repair, and real process-crash coverage.
  • Phase 1: safe-boundary continuation into fresh Run/Turn/Invocation identities, durable continuation claims, plan/execute separation, and execution-time revalidation.
  • Phase 2: sticky SQLite canonical RuntimeEvent storage, transactional T1/T2 tool boundaries, operation identity/CAS, import/export, and one canonical writer.
  • Phase 2.5 convergence: RecoveryResolver is the sole decision authority; tool journal tables are rebuildable projections, not a parallel truth source.

The Desktop manual recovery entry is attached to the interrupted-turn banner as Safe resume. CLI/TUI retain /resume. Park reasons are translated for users while stable reason codes remain available to diagnostics.

Related to #186.

Safety and rollout

Both capabilities remain disabled by default:

FlagBehavior
MAKA_RUNTIME_SQLITE_CANONICAL=1Triggers a sticky, one-way workspace migration to runtime.sqlite; disabling the flag does not return that workspace to JSONL.
MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1Enables safe-boundary recovery surfaces and Desktop startup auto-resume; these paths may call the provider and consume tokens.

Ambiguous tool outcomes are parked, never blindly retried. Phase 3 reconciliation is intentionally outside this PR.

Dogfooding and regression fixes

Real restart/resume testing found and fixed:

  1. second-generation continuation envelopes dropping source-only replay context;
  2. continuation replay/history reconstruction gaps across ancestor runs;
  3. continuation runs being misclassified as child-agent runs in model, UI, compaction, and repair projections;
  4. canonical continuation-start facts being treated as unsupported chat events;
  5. empty materialized continuation replay reaching the provider.

The final recovery gates also reject duplicate dispatch/response facts and unknown runtime protocol markers as corruption.

Known limitations

  • Automatic pre-migration backup is not implemented yet.
  • Populated SQLite v2-to-v4 upgrade coverage remains to be added.
  • Crash harness support targets Linux/macOS; Windows remains limited support.
  • definitely_not_dispatched is classified but its Phase 3 automated recovery policy is not implemented.
  • Legacy tool-projection rebuilding remains compatibility-only and must not become a second recovery truth source.
  • Manual and startup auto-resume still share one feature flag; they should be split before broader rollout.

Verification

  • npm run typecheck (all workspaces)
  • RecoveryResolver suite: 11 passed
  • empty continuation replay provider guard: passed
  • Desktop interrupted-banner routing and eligibility: 6 passed
  • user-facing recovery copy: 3 passed
  • changed-file Biome lint and git diff --check

@likun666661

likun666661 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Review conclusion

This is a novel and technically strong approach to runtime resume. The PR does not try to revive an old JavaScript stack, provider stream, Promise, or operating-system process. Instead, it reconstructs durable execution facts, proves that a continuation boundary is safe, and then starts a new Run, Invocation, and Turn.

The most important design choice is the distinction between replaying tool history and re-executing tool side effects:

  • completed function_call and function_response pairs are replayed to the provider as history;
  • the previous tool implementation is not executed again;
  • an unmatched or ambiguous tool call is parked rather than retried;
  • contradictory recovery facts fail closed.

This gives Maka a useful resume primitive without claiming exactly-once execution for arbitrary Bash commands, filesystem mutations, or external APIs.

Problem being solved

The existing RuntimeEvent ledger could answer what had been recorded, but it could not reliably answer whether a tool might already have crossed the side-effect boundary.

Two crash windows are particularly dangerous:

  1. The tool implementation starts before the canonical call fact is durably committed.
  2. The tool finishes its side effect, but the process crashes before the canonical response is durably committed.

In either case, treating a missing result as proof that the tool did not execute can duplicate writes, deletes, deployments, payments, or remote requests. The previous startup repair path could converge a dangling Run to a failed terminal state, but it could not safely continue execution.

This PR solves that by adding a durable tool-dispatch boundary, deterministic recovery projection, and safe-boundary continuation.

Module view

flowchart LR
subgraph Entry["Product entry points"]
Desktop["Desktop Safe resume and startup recovery"]
CLI["CLI and TUI /resume"]
end
subgraph Control["Recovery control plane"]
SessionManager["SessionManager<br/>candidate discovery and lifecycle"]
Inspector["ContinuationSafetyInspector<br/>cwd, workspace, tools, background work"]
Planner["RuntimeContinuationPlanner<br/>continue or park"]
Resolver["RecoveryResolver<br/>single decision authority"]
end
subgraph Execution["Execution plane"]
Kernel["RuntimeKernel<br/>revalidation and claim"]
AgentRun["AgentRun<br/>fresh execution identity"]
Runner["RuntimeRunner<br/>continuation-start fact"]
Backend["AiSdkFlow and AiSdkBackend<br/>provider replay"]
ToolRuntime["ToolRuntime<br/>preflight, T1, implementation, T2"]
end
subgraph Persistence["Persistence"]
Bootstrap["openRuntimeEventPersistence"]
SQLite[("runtime.sqlite<br/>canonical RuntimeEvent store")]
Projection["tool_journal_events and tool_operations<br/>rebuildable projections"]
JSONL["Legacy RuntimeEvent JSONL<br/>import and explicit export"]
end
Desktop --> SessionManager
CLI --> SessionManager
SessionManager --> Inspector
SessionManager --> Planner
Inspector --> Planner
SQLite --> Planner
Planner --> Resolver
Resolver --> Planner
Planner -->|"approved continuation"| Kernel
Planner -->|"park with stable reason codes"| Desktop
Kernel --> AgentRun
AgentRun --> Runner
Runner --> Backend
Backend --> ToolRuntime
Runner -->|"continuation-start"| SQLite
ToolRuntime -->|"T1 and T2"| SQLite
SQLite -.->|"synchronous projection"| Projection
Bootstrap --> SQLite
Bootstrap -.->|"unmigrated workspace and flag disabled"| JSONL
JSONL -->|"sticky, idempotent import"| SQLite
Loading

The architectural boundary is clear:

  • RuntimeEvent is the canonical recovery fact source.
  • RecoveryResolver is the only component that interprets tool recovery state.
  • Tool journal and operation tables are query projections and can be rebuilt from RuntimeEvents.
  • Product surfaces consume stable plans and reason codes instead of implementing their own recovery logic.

How the tool boundary closes the crash windows

sequenceDiagram
autonumber
participant Provider as Model Provider
participant Backend as AiSdkBackend
participant Runtime as ToolRuntime
participant Store as SqliteRuntimeStore
participant Tool as Tool Implementation
Provider->>Backend: function_call
Backend->>Runtime: execute tool
Runtime->>Runtime: validate args, availability, loop, permissions, and runtime guards
Runtime->>Store: T1 commitToolPrepared
Note over Store: Insert or verify function_call<br/>append toolDispatch RuntimeEvent<br/>update journal and operation projection
alt T1 fails
Store--xRuntime: rollback
Note right of Runtime: Tool implementation is never called
else T1 commits
Store-->>Runtime: durable dispatch boundary
Runtime->>Tool: execute implementation
Tool-->>Runtime: result or error
Runtime->>Store: T2 commitToolOutcome
Note over Store: Append function_response<br/>append outcome journal<br/>CAS operation state
alt T2 fails
Store--xRuntime: rollback
Note right of Runtime: Result is not returned to the next model step
else T2 commits
Store-->>Runtime: durable outcome
Runtime-->>Backend: tool_result
Backend-->>Provider: next provider step
end
end
Loading

This establishes two important invariants:

  1. If T1 fails, the tool implementation runs zero times.
  2. If T2 fails, the tool result cannot reach the next model step.

The external side effect still occurs between two short database transactions, so SQLite cannot make an arbitrary external operation atomic. Instead, a crash between T1 and T2 becomes an explicit indeterminate state that recovery must park or reconcile later.

How safe continuation works

sequenceDiagram
autonumber
participant Entry as User or Startup Recovery
participant Manager as SessionManager
participant Inspector as Safety Inspector
participant Planner as Continuation Planner
participant Resolver as RecoveryResolver
participant Store as Durable Stores
participant Kernel as RuntimeKernel
participant Provider as Model Provider
Entry->>Manager: resumeLatest
Manager->>Store: find latest failed or cancelled session-inline Run
Manager->>Inspector: inspect cwd, workspace identity, tool catalog, and background work
Inspector-->>Manager: authoritative safety observation
Manager->>Planner: plan source boundary
Planner->>Store: read Run header, RuntimeEvents, and continuation ancestors
Planner->>Resolver: resolve tool recovery facts
Resolver-->>Planner: completed, indeterminate, not dispatched, or corruption
alt Any safety condition fails
Planner-->>Manager: park with stable reason codes
Manager-->>Entry: explain why resume is unavailable
Note over Entry,Provider: Provider is not called
else Boundary is safe
Planner-->>Manager: continuation plan with fresh identities and safety snapshot
Manager->>Kernel: execute continuation
Kernel->>Store: re-read source terminal, high-water, identity, and replay
Kernel->>Inspector: revalidate external safety facts
Kernel->>Store: create target Run with continuationSource claim
Kernel->>Store: commit continuation-start RuntimeEvent
Kernel->>Provider: replay committed history without a duplicate user message
Provider-->>Kernel: continue from the validated boundary
end
Loading

Planning and execution are intentionally separate. A valid plan is not treated as an execution lease. Immediately before execution, the Runtime re-reads durable state and rechecks workspace identity, active operations, tool availability, source high-water, and replay equality.

The continuation also creates new execution identity:

source Session / Invocation / Run / Turn
-> validated RuntimeEvent high-water
-> new Invocation / Run / Turn
-> durable continuation-start
-> provider replay without a duplicate user message

The source ledger is not mutated by the continuation.

Recovery decisions

Durable factsResolver decisionCurrent behavior
call + matching responsecompletedReplay call and response as provider history
call + dispatch + no responseindeterminatePark; never blindly retry
call + no dispatch + new protocol markerdefinitely not dispatchedClassified, but currently still parked until Phase 3 policy exists
call + no dispatch + legacy or unknown protocolindeterminatePark
orphan, duplicate, or identity-conflicting factscorruptionFail closed

This is effectively tool-log replay: the provider receives the completed tool interaction as historical context, while the Runtime avoids replaying the old implementation or side effect.

What this PR introduces

AreaIntroduced capability
Recovery semanticsRecoveryResolver, deterministic tool-operation projection, stable diagnostics and park reasons
ContinuationRuntimeContinuationPlanner, fresh execution identities, durable continuation claim, execution-time revalidation
Tool protocoltoolDispatch RuntimeEvent, protocol marker, deterministic operation ID, canonical argument hash, recovery mode
Transaction boundaryRuntimeCommitSink, T1 commitToolPrepared, T2 commitToolOutcome, CAS updates
Canonical storageSQLite RuntimeEvent store, WAL, synchronous=FULL, foreign keys, schema migrations
CompatibilitySticky JSONL-to-SQLite import, source fingerprints, explicit JSONL export
Product surfacesDesktop Safe resume, startup continuation, CLI and TUI /resume, user-facing park copy
ValidationProcess-crash harnesses, replay tests, continuation tests, SQLite crash tests, and Linux/macOS CI coverage

Current boundaries

The conservative scope is appropriate, but it is important to preserve these limitations:

  • this PR does not re-execute historical tool implementations;
  • Phase 3 reconciliation and restricted verification are not implemented;
  • definitely_not_dispatched is classified but does not yet trigger automatic recovery;
  • general Bash and unknown external side effects remain parked;
  • replay-safe, idempotent, reconcile, and reattach tool contracts are future work;
  • SQLite migration is sticky and does not yet provide automatic pre-migration backup;
  • manual resume and startup auto-resume currently share one feature flag;
  • multi-worker leases and fencing are intentionally out of scope.

Future value of checkpoints and Git snapshots

The next major step that could make this resume system broadly practical is coupling the RuntimeEvent high-water with a workspace checkpoint, especially a Git-backed snapshot.

That would let recovery validate or restore both sides of the continuation boundary:

  1. the model and tool interaction history; and
  2. the exact agent-visible workspace state associated with that history.

A durable boundary containing:

RuntimeEvent high-water
+ workspace checkpoint reference
+ Git commit or tree snapshot
+ workspace identity

would allow Maka to resume after a crash without assuming that the current filesystem still matches the interrupted Run. It would also make planning and execution revalidation stronger because the continuation could prove that conversational state and workspace state refer to the same execution boundary.

Overall assessment

This is an excellent foundation and a genuinely interesting resume design.

The durable tool-call protocol makes safe history replay possible without conflating replay with side-effect re-execution. The fail-closed behavior is correct, the new execution identity keeps lineage understandable, and the single canonical recovery authority avoids parallel state machines.

With checkpoint and Git snapshot integration, this design could evolve from safe conversational continuation into a practical end-to-end runtime resume system.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

Conflict-resolution verification update

The branch was reconciled with current main through local merge commits 81edf2ba (Phase 1) and f0653e02 (Phase 2), then pushed without including local dogfooding files.

Additional compatibility fixes made during the merge:

  • aligned SQLite canonical storage with the current terminal durability barrier;
  • taught the exact-shape RuntimeEvent decoder about toolDispatch, runtimeProtocol, and operationId;
  • kept legacy JSONL import independent of a surviving run.json header;
  • preserved continuation invocation identity and session-inline projection semantics;
  • retained current steering, strict-recovery, child-agent, and Desktop IPC contracts.

Verification after reconciliation:

  • npm run build:test — passed across all workspaces;
  • npm run typecheck — passed across all workspaces;
  • Runtime resume/continuation/runner/tool-boundary focused suites — 62 passed;
  • SQLite runtime store — 10 passed;
  • legacy JSONL-to-SQLite transfer — 5 passed;
  • real-process Runtime resume crash harness — passed;
  • Desktop interrupted-banner routing — 4 passed;
  • focused SessionManager continuation/canonical/UI projection cases — 3 passed.

Windows-only file fsync/signal harness limitations remain documented; Linux/macOS are the supported crash-durability targets.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

CI follow-up is complete on head \

@zhiiw
zhiiw marked this pull request as ready for review July 19, 2026 10:40
@Astro-Han
Astro-Han merged commit b5e015a into apache:mainJul 19, 2026
5 checks passed
@zhiiw
zhiiw deleted the codex/runtime-resume-phase2 branch July 19, 2026 11:03
jackwener added a commit that referenced this pull request Jul 19, 2026
…nip zero-hint, resume hook, QR fixture (#1241)
* feat(core): add settings-bots-onboarding visual-smoke scenario + botOnboardingProvider
* feat(desktop): hold-in-waiting bot onboarding adapter + fixture state for QR capture
* feat(desktop): auto-open bot scan-login modal under settings-bots-onboarding fixture
* chore(scripts): register settings-bots-onboarding in capture + audit scenario lists
* fix(desktop): wait for QR paint before auto-capturing bot-onboarding fixture
* docs(notes): frontend architecture map — measured baseline + 8 staged rounds (2026-07-19)
Re-ran the census (wc over non-test TS/TSX for desktop main/preload/renderer +
packages/ui; CSS totals; top-10 hotspot table incl. visual-smoke-fixture.ts marked
OUT-OF-SCOPE other-sessions' territory). Staged R1 knip de-rot, R2 app-shell
resume-cluster extraction, R3 visual-smoke split (BLOCKED on concurrent fixture branch),
R4-R6 main.ts extractions (require maintainer-approved contract re-pins — registerIpc /
startup / tool-assembly / modelSupportsVision direct-pinned), R7 provider-connection-detail
decomposition (needs its own blade plan), R8 CSS raw-hex residue (verified clean on this
tip — #1085 already converted it).
* chore(knip): de-rot config — clear all 13 stale hints (R1)
knip reported 13 configuration hints (7 desktop + 6 ui), all redundant/stale config,
zero code impact:
- Drop 'overlayscrollbars' from desktop ignoreDependencies — now redundant (knip no
longer flags it; the package is owned by packages/ui and only referenced there). The
overlay-scrollbars contract forbids declaring it as a *desktop dependency*, not as an
ignore — removing a redundant ignore is compatible and coverage is unchanged.
- Delete the dead 'src/renderer/**/*.test.ts' entry glob (no matches — renderer has zero
test files; all 338 desktop tests live under src/main/__tests__/*.test.ts).
- Drop 11 redundant explicit entries knip already auto-detects: desktop
preload.ts / renderer/main.tsx / playwright.config.ts / .storybook/main.ts / dev.mjs
(vite / playwright / storybook / npm-scripts plugins), and all 6 ui entries
(index.ts, icons.tsx, artifact-preview-registry.ts, assistant-stream.ts, maka-uri.ts,
smooth-stream.ts) resolved from packages/ui package.json#exports.
Non-redundant entries kept verbatim (main.ts, overlay/*, main test glob, e2e specs,
storybook preview, stories, browser-observe-act-smoke.mjs, both ui test globs). No broad
ignores added — coverage is not weakened. knip --workspace apps/desktop and packages/ui
now both exit 0 AND report zero config hints (was: exit 0 with 13 hints).
* refactor(renderer): extract app-shell resume cluster into use-shell-resume (R2)
The #1223 safe-boundary resume cluster moves out of app-shell.tsx into a new
use-shell-resume.ts hook, following the use-shell-connections / use-shell-chat-model
house style (options object; state + handler returned; stable identities preserved).
Pure move, zero behavior change: the two useState declarations (resumePendingSessionId,
resumeParkDescriptionBySession) and the resumeInterruptedSession handler move verbatim;
activeId/toastApi/shellCopy/uiLocale are injected as options. app-shell keeps the banner
JSX wiring (safeResumeAction=) and the sendWithAttachments guard. resumeParkToastCopy is
no longer imported into app-shell (it now lives in the hook). app-shell.tsx 1686 -> 1654
(-32 lines).
Contract re-pin (never deletes coverage): runtime-resume-routing-contract.test.ts read
the handler-shape assertions (resumeInterruptedSession / resumeLatest(sessionId) /
resumeParkToastCopy / no rejectionReasons.join) from app-shell.tsx single-file; they now
read use-shell-resume.ts where that logic lives, while the app-shell-specific assertions
(sendWithAttachments '/resume' guard, safeResumeAction= wiring) still read app-shell.tsx.
use-shell-resume.ts added to renderer-shell-source-helpers sourcePaths (Round B/E
precedent) so combined-source contracts see it.
Gates: desktop 2740/2740, ui 196/196, typecheck 0, check-dead-css clean, knip
desktop+ui exit 0, AUDIT_PORT_BASE=23900 alignment auditor exit 0 (all 10 fixtures
clean, real renderer). CDP turn-narrative branch-vs-baseline (real Electron, light+dark
1280) byte-identical (sha256 match) — proven render no-op.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@zhiiw@likun666661@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): add durable safe-boundary resume phases 0-2 - #1223

Merged
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2
Jul 19, 2026
Merged

feat(runtime): add durable safe-boundary resume phases 0-2#1223
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2

Conversation

@zhiiw

Copy link
Copy Markdown
Contributor

Summary

Implements the Runtime Resume foundation from Phase 0 through Phase 2, with RuntimeEvent as the single canonical recovery fact source.

  • Phase 0: fail-closed RuntimeEvent replay projection, unmatched tool-call parking, terminal repair, and real process-crash coverage.
  • Phase 1: safe-boundary continuation into fresh Run/Turn/Invocation identities, durable continuation claims, plan/execute separation, and execution-time revalidation.
  • Phase 2: sticky SQLite canonical RuntimeEvent storage, transactional T1/T2 tool boundaries, operation identity/CAS, import/export, and one canonical writer.
  • Phase 2.5 convergence: RecoveryResolver is the sole decision authority; tool journal tables are rebuildable projections, not a parallel truth source.

The Desktop manual recovery entry is attached to the interrupted-turn banner as Safe resume. CLI/TUI retain /resume. Park reasons are translated for users while stable reason codes remain available to diagnostics.

Related to #186.

Safety and rollout

Both capabilities remain disabled by default:

FlagBehavior
MAKA_RUNTIME_SQLITE_CANONICAL=1Triggers a sticky, one-way workspace migration to runtime.sqlite; disabling the flag does not return that workspace to JSONL.
MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1Enables safe-boundary recovery surfaces and Desktop startup auto-resume; these paths may call the provider and consume tokens.

Ambiguous tool outcomes are parked, never blindly retried. Phase 3 reconciliation is intentionally outside this PR.

Dogfooding and regression fixes

Real restart/resume testing found and fixed:

  1. second-generation continuation envelopes dropping source-only replay context;
  2. continuation replay/history reconstruction gaps across ancestor runs;
  3. continuation runs being misclassified as child-agent runs in model, UI, compaction, and repair projections;
  4. canonical continuation-start facts being treated as unsupported chat events;
  5. empty materialized continuation replay reaching the provider.

The final recovery gates also reject duplicate dispatch/response facts and unknown runtime protocol markers as corruption.

Known limitations

  • Automatic pre-migration backup is not implemented yet.
  • Populated SQLite v2-to-v4 upgrade coverage remains to be added.
  • Crash harness support targets Linux/macOS; Windows remains limited support.
  • definitely_not_dispatched is classified but its Phase 3 automated recovery policy is not implemented.
  • Legacy tool-projection rebuilding remains compatibility-only and must not become a second recovery truth source.
  • Manual and startup auto-resume still share one feature flag; they should be split before broader rollout.

Verification

  • npm run typecheck (all workspaces)
  • RecoveryResolver suite: 11 passed
  • empty continuation replay provider guard: passed
  • Desktop interrupted-banner routing and eligibility: 6 passed
  • user-facing recovery copy: 3 passed
  • changed-file Biome lint and git diff --check

@likun666661

likun666661 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Review conclusion

This is a novel and technically strong approach to runtime resume. The PR does not try to revive an old JavaScript stack, provider stream, Promise, or operating-system process. Instead, it reconstructs durable execution facts, proves that a continuation boundary is safe, and then starts a new Run, Invocation, and Turn.

The most important design choice is the distinction between replaying tool history and re-executing tool side effects:

  • completed function_call and function_response pairs are replayed to the provider as history;
  • the previous tool implementation is not executed again;
  • an unmatched or ambiguous tool call is parked rather than retried;
  • contradictory recovery facts fail closed.

This gives Maka a useful resume primitive without claiming exactly-once execution for arbitrary Bash commands, filesystem mutations, or external APIs.

Problem being solved

The existing RuntimeEvent ledger could answer what had been recorded, but it could not reliably answer whether a tool might already have crossed the side-effect boundary.

Two crash windows are particularly dangerous:

  1. The tool implementation starts before the canonical call fact is durably committed.
  2. The tool finishes its side effect, but the process crashes before the canonical response is durably committed.

In either case, treating a missing result as proof that the tool did not execute can duplicate writes, deletes, deployments, payments, or remote requests. The previous startup repair path could converge a dangling Run to a failed terminal state, but it could not safely continue execution.

This PR solves that by adding a durable tool-dispatch boundary, deterministic recovery projection, and safe-boundary continuation.

Module view

flowchart LR
subgraph Entry["Product entry points"]
Desktop["Desktop Safe resume and startup recovery"]
CLI["CLI and TUI /resume"]
end
subgraph Control["Recovery control plane"]
SessionManager["SessionManager<br/>candidate discovery and lifecycle"]
Inspector["ContinuationSafetyInspector<br/>cwd, workspace, tools, background work"]
Planner["RuntimeContinuationPlanner<br/>continue or park"]
Resolver["RecoveryResolver<br/>single decision authority"]
end
subgraph Execution["Execution plane"]
Kernel["RuntimeKernel<br/>revalidation and claim"]
AgentRun["AgentRun<br/>fresh execution identity"]
Runner["RuntimeRunner<br/>continuation-start fact"]
Backend["AiSdkFlow and AiSdkBackend<br/>provider replay"]
ToolRuntime["ToolRuntime<br/>preflight, T1, implementation, T2"]
end
subgraph Persistence["Persistence"]
Bootstrap["openRuntimeEventPersistence"]
SQLite[("runtime.sqlite<br/>canonical RuntimeEvent store")]
Projection["tool_journal_events and tool_operations<br/>rebuildable projections"]
JSONL["Legacy RuntimeEvent JSONL<br/>import and explicit export"]
end
Desktop --> SessionManager
CLI --> SessionManager
SessionManager --> Inspector
SessionManager --> Planner
Inspector --> Planner
SQLite --> Planner
Planner --> Resolver
Resolver --> Planner
Planner -->|"approved continuation"| Kernel
Planner -->|"park with stable reason codes"| Desktop
Kernel --> AgentRun
AgentRun --> Runner
Runner --> Backend
Backend --> ToolRuntime
Runner -->|"continuation-start"| SQLite
ToolRuntime -->|"T1 and T2"| SQLite
SQLite -.->|"synchronous projection"| Projection
Bootstrap --> SQLite
Bootstrap -.->|"unmigrated workspace and flag disabled"| JSONL
JSONL -->|"sticky, idempotent import"| SQLite
Loading

The architectural boundary is clear:

  • RuntimeEvent is the canonical recovery fact source.
  • RecoveryResolver is the only component that interprets tool recovery state.
  • Tool journal and operation tables are query projections and can be rebuilt from RuntimeEvents.
  • Product surfaces consume stable plans and reason codes instead of implementing their own recovery logic.

How the tool boundary closes the crash windows

sequenceDiagram
autonumber
participant Provider as Model Provider
participant Backend as AiSdkBackend
participant Runtime as ToolRuntime
participant Store as SqliteRuntimeStore
participant Tool as Tool Implementation
Provider->>Backend: function_call
Backend->>Runtime: execute tool
Runtime->>Runtime: validate args, availability, loop, permissions, and runtime guards
Runtime->>Store: T1 commitToolPrepared
Note over Store: Insert or verify function_call<br/>append toolDispatch RuntimeEvent<br/>update journal and operation projection
alt T1 fails
Store--xRuntime: rollback
Note right of Runtime: Tool implementation is never called
else T1 commits
Store-->>Runtime: durable dispatch boundary
Runtime->>Tool: execute implementation
Tool-->>Runtime: result or error
Runtime->>Store: T2 commitToolOutcome
Note over Store: Append function_response<br/>append outcome journal<br/>CAS operation state
alt T2 fails
Store--xRuntime: rollback
Note right of Runtime: Result is not returned to the next model step
else T2 commits
Store-->>Runtime: durable outcome
Runtime-->>Backend: tool_result
Backend-->>Provider: next provider step
end
end
Loading

This establishes two important invariants:

  1. If T1 fails, the tool implementation runs zero times.
  2. If T2 fails, the tool result cannot reach the next model step.

The external side effect still occurs between two short database transactions, so SQLite cannot make an arbitrary external operation atomic. Instead, a crash between T1 and T2 becomes an explicit indeterminate state that recovery must park or reconcile later.

How safe continuation works

sequenceDiagram
autonumber
participant Entry as User or Startup Recovery
participant Manager as SessionManager
participant Inspector as Safety Inspector
participant Planner as Continuation Planner
participant Resolver as RecoveryResolver
participant Store as Durable Stores
participant Kernel as RuntimeKernel
participant Provider as Model Provider
Entry->>Manager: resumeLatest
Manager->>Store: find latest failed or cancelled session-inline Run
Manager->>Inspector: inspect cwd, workspace identity, tool catalog, and background work
Inspector-->>Manager: authoritative safety observation
Manager->>Planner: plan source boundary
Planner->>Store: read Run header, RuntimeEvents, and continuation ancestors
Planner->>Resolver: resolve tool recovery facts
Resolver-->>Planner: completed, indeterminate, not dispatched, or corruption
alt Any safety condition fails
Planner-->>Manager: park with stable reason codes
Manager-->>Entry: explain why resume is unavailable
Note over Entry,Provider: Provider is not called
else Boundary is safe
Planner-->>Manager: continuation plan with fresh identities and safety snapshot
Manager->>Kernel: execute continuation
Kernel->>Store: re-read source terminal, high-water, identity, and replay
Kernel->>Inspector: revalidate external safety facts
Kernel->>Store: create target Run with continuationSource claim
Kernel->>Store: commit continuation-start RuntimeEvent
Kernel->>Provider: replay committed history without a duplicate user message
Provider-->>Kernel: continue from the validated boundary
end
Loading

Planning and execution are intentionally separate. A valid plan is not treated as an execution lease. Immediately before execution, the Runtime re-reads durable state and rechecks workspace identity, active operations, tool availability, source high-water, and replay equality.

The continuation also creates new execution identity:

source Session / Invocation / Run / Turn
-> validated RuntimeEvent high-water
-> new Invocation / Run / Turn
-> durable continuation-start
-> provider replay without a duplicate user message

The source ledger is not mutated by the continuation.

Recovery decisions

Durable factsResolver decisionCurrent behavior
call + matching responsecompletedReplay call and response as provider history
call + dispatch + no responseindeterminatePark; never blindly retry
call + no dispatch + new protocol markerdefinitely not dispatchedClassified, but currently still parked until Phase 3 policy exists
call + no dispatch + legacy or unknown protocolindeterminatePark
orphan, duplicate, or identity-conflicting factscorruptionFail closed

This is effectively tool-log replay: the provider receives the completed tool interaction as historical context, while the Runtime avoids replaying the old implementation or side effect.

What this PR introduces

AreaIntroduced capability
Recovery semanticsRecoveryResolver, deterministic tool-operation projection, stable diagnostics and park reasons
ContinuationRuntimeContinuationPlanner, fresh execution identities, durable continuation claim, execution-time revalidation
Tool protocoltoolDispatch RuntimeEvent, protocol marker, deterministic operation ID, canonical argument hash, recovery mode
Transaction boundaryRuntimeCommitSink, T1 commitToolPrepared, T2 commitToolOutcome, CAS updates
Canonical storageSQLite RuntimeEvent store, WAL, synchronous=FULL, foreign keys, schema migrations
CompatibilitySticky JSONL-to-SQLite import, source fingerprints, explicit JSONL export
Product surfacesDesktop Safe resume, startup continuation, CLI and TUI /resume, user-facing park copy
ValidationProcess-crash harnesses, replay tests, continuation tests, SQLite crash tests, and Linux/macOS CI coverage

Current boundaries

The conservative scope is appropriate, but it is important to preserve these limitations:

  • this PR does not re-execute historical tool implementations;
  • Phase 3 reconciliation and restricted verification are not implemented;
  • definitely_not_dispatched is classified but does not yet trigger automatic recovery;
  • general Bash and unknown external side effects remain parked;
  • replay-safe, idempotent, reconcile, and reattach tool contracts are future work;
  • SQLite migration is sticky and does not yet provide automatic pre-migration backup;
  • manual resume and startup auto-resume currently share one feature flag;
  • multi-worker leases and fencing are intentionally out of scope.

Future value of checkpoints and Git snapshots

The next major step that could make this resume system broadly practical is coupling the RuntimeEvent high-water with a workspace checkpoint, especially a Git-backed snapshot.

That would let recovery validate or restore both sides of the continuation boundary:

  1. the model and tool interaction history; and
  2. the exact agent-visible workspace state associated with that history.

A durable boundary containing:

RuntimeEvent high-water
+ workspace checkpoint reference
+ Git commit or tree snapshot
+ workspace identity

would allow Maka to resume after a crash without assuming that the current filesystem still matches the interrupted Run. It would also make planning and execution revalidation stronger because the continuation could prove that conversational state and workspace state refer to the same execution boundary.

Overall assessment

This is an excellent foundation and a genuinely interesting resume design.

The durable tool-call protocol makes safe history replay possible without conflating replay with side-effect re-execution. The fail-closed behavior is correct, the new execution identity keeps lineage understandable, and the single canonical recovery authority avoids parallel state machines.

With checkpoint and Git snapshot integration, this design could evolve from safe conversational continuation into a practical end-to-end runtime resume system.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

Conflict-resolution verification update

The branch was reconciled with current main through local merge commits 81edf2ba (Phase 1) and f0653e02 (Phase 2), then pushed without including local dogfooding files.

Additional compatibility fixes made during the merge:

  • aligned SQLite canonical storage with the current terminal durability barrier;
  • taught the exact-shape RuntimeEvent decoder about toolDispatch, runtimeProtocol, and operationId;
  • kept legacy JSONL import independent of a surviving run.json header;
  • preserved continuation invocation identity and session-inline projection semantics;
  • retained current steering, strict-recovery, child-agent, and Desktop IPC contracts.

Verification after reconciliation:

  • npm run build:test — passed across all workspaces;
  • npm run typecheck — passed across all workspaces;
  • Runtime resume/continuation/runner/tool-boundary focused suites — 62 passed;
  • SQLite runtime store — 10 passed;
  • legacy JSONL-to-SQLite transfer — 5 passed;
  • real-process Runtime resume crash harness — passed;
  • Desktop interrupted-banner routing — 4 passed;
  • focused SessionManager continuation/canonical/UI projection cases — 3 passed.

Windows-only file fsync/signal harness limitations remain documented; Linux/macOS are the supported crash-durability targets.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

CI follow-up is complete on head \

@zhiiw
zhiiw marked this pull request as ready for review July 19, 2026 10:40
@Astro-Han
Astro-Han merged commit b5e015a into apache:mainJul 19, 2026
5 checks passed
@zhiiw
zhiiw deleted the codex/runtime-resume-phase2 branch July 19, 2026 11:03
jackwener added a commit that referenced this pull request Jul 19, 2026
…nip zero-hint, resume hook, QR fixture (#1241)
* feat(core): add settings-bots-onboarding visual-smoke scenario + botOnboardingProvider
* feat(desktop): hold-in-waiting bot onboarding adapter + fixture state for QR capture
* feat(desktop): auto-open bot scan-login modal under settings-bots-onboarding fixture
* chore(scripts): register settings-bots-onboarding in capture + audit scenario lists
* fix(desktop): wait for QR paint before auto-capturing bot-onboarding fixture
* docs(notes): frontend architecture map — measured baseline + 8 staged rounds (2026-07-19)
Re-ran the census (wc over non-test TS/TSX for desktop main/preload/renderer +
packages/ui; CSS totals; top-10 hotspot table incl. visual-smoke-fixture.ts marked
OUT-OF-SCOPE other-sessions' territory). Staged R1 knip de-rot, R2 app-shell
resume-cluster extraction, R3 visual-smoke split (BLOCKED on concurrent fixture branch),
R4-R6 main.ts extractions (require maintainer-approved contract re-pins — registerIpc /
startup / tool-assembly / modelSupportsVision direct-pinned), R7 provider-connection-detail
decomposition (needs its own blade plan), R8 CSS raw-hex residue (verified clean on this
tip — #1085 already converted it).
* chore(knip): de-rot config — clear all 13 stale hints (R1)
knip reported 13 configuration hints (7 desktop + 6 ui), all redundant/stale config,
zero code impact:
- Drop 'overlayscrollbars' from desktop ignoreDependencies — now redundant (knip no
longer flags it; the package is owned by packages/ui and only referenced there). The
overlay-scrollbars contract forbids declaring it as a *desktop dependency*, not as an
ignore — removing a redundant ignore is compatible and coverage is unchanged.
- Delete the dead 'src/renderer/**/*.test.ts' entry glob (no matches — renderer has zero
test files; all 338 desktop tests live under src/main/__tests__/*.test.ts).
- Drop 11 redundant explicit entries knip already auto-detects: desktop
preload.ts / renderer/main.tsx / playwright.config.ts / .storybook/main.ts / dev.mjs
(vite / playwright / storybook / npm-scripts plugins), and all 6 ui entries
(index.ts, icons.tsx, artifact-preview-registry.ts, assistant-stream.ts, maka-uri.ts,
smooth-stream.ts) resolved from packages/ui package.json#exports.
Non-redundant entries kept verbatim (main.ts, overlay/*, main test glob, e2e specs,
storybook preview, stories, browser-observe-act-smoke.mjs, both ui test globs). No broad
ignores added — coverage is not weakened. knip --workspace apps/desktop and packages/ui
now both exit 0 AND report zero config hints (was: exit 0 with 13 hints).
* refactor(renderer): extract app-shell resume cluster into use-shell-resume (R2)
The #1223 safe-boundary resume cluster moves out of app-shell.tsx into a new
use-shell-resume.ts hook, following the use-shell-connections / use-shell-chat-model
house style (options object; state + handler returned; stable identities preserved).
Pure move, zero behavior change: the two useState declarations (resumePendingSessionId,
resumeParkDescriptionBySession) and the resumeInterruptedSession handler move verbatim;
activeId/toastApi/shellCopy/uiLocale are injected as options. app-shell keeps the banner
JSX wiring (safeResumeAction=) and the sendWithAttachments guard. resumeParkToastCopy is
no longer imported into app-shell (it now lives in the hook). app-shell.tsx 1686 -> 1654
(-32 lines).
Contract re-pin (never deletes coverage): runtime-resume-routing-contract.test.ts read
the handler-shape assertions (resumeInterruptedSession / resumeLatest(sessionId) /
resumeParkToastCopy / no rejectionReasons.join) from app-shell.tsx single-file; they now
read use-shell-resume.ts where that logic lives, while the app-shell-specific assertions
(sendWithAttachments '/resume' guard, safeResumeAction= wiring) still read app-shell.tsx.
use-shell-resume.ts added to renderer-shell-source-helpers sourcePaths (Round B/E
precedent) so combined-source contracts see it.
Gates: desktop 2740/2740, ui 196/196, typecheck 0, check-dead-css clean, knip
desktop+ui exit 0, AUDIT_PORT_BASE=23900 alignment auditor exit 0 (all 10 fixtures
clean, real renderer). CDP turn-narrative branch-vs-baseline (real Electron, light+dark
1280) byte-identical (sha256 match) — proven render no-op.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@zhiiw@likun666661@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): add durable safe-boundary resume phases 0-2 - #1223

Merged
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2
Jul 19, 2026
Merged

feat(runtime): add durable safe-boundary resume phases 0-2#1223
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2

Conversation

@zhiiw

Copy link
Copy Markdown
Contributor

Summary

Implements the Runtime Resume foundation from Phase 0 through Phase 2, with RuntimeEvent as the single canonical recovery fact source.

  • Phase 0: fail-closed RuntimeEvent replay projection, unmatched tool-call parking, terminal repair, and real process-crash coverage.
  • Phase 1: safe-boundary continuation into fresh Run/Turn/Invocation identities, durable continuation claims, plan/execute separation, and execution-time revalidation.
  • Phase 2: sticky SQLite canonical RuntimeEvent storage, transactional T1/T2 tool boundaries, operation identity/CAS, import/export, and one canonical writer.
  • Phase 2.5 convergence: RecoveryResolver is the sole decision authority; tool journal tables are rebuildable projections, not a parallel truth source.

The Desktop manual recovery entry is attached to the interrupted-turn banner as Safe resume. CLI/TUI retain /resume. Park reasons are translated for users while stable reason codes remain available to diagnostics.

Related to #186.

Safety and rollout

Both capabilities remain disabled by default:

FlagBehavior
MAKA_RUNTIME_SQLITE_CANONICAL=1Triggers a sticky, one-way workspace migration to runtime.sqlite; disabling the flag does not return that workspace to JSONL.
MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1Enables safe-boundary recovery surfaces and Desktop startup auto-resume; these paths may call the provider and consume tokens.

Ambiguous tool outcomes are parked, never blindly retried. Phase 3 reconciliation is intentionally outside this PR.

Dogfooding and regression fixes

Real restart/resume testing found and fixed:

  1. second-generation continuation envelopes dropping source-only replay context;
  2. continuation replay/history reconstruction gaps across ancestor runs;
  3. continuation runs being misclassified as child-agent runs in model, UI, compaction, and repair projections;
  4. canonical continuation-start facts being treated as unsupported chat events;
  5. empty materialized continuation replay reaching the provider.

The final recovery gates also reject duplicate dispatch/response facts and unknown runtime protocol markers as corruption.

Known limitations

  • Automatic pre-migration backup is not implemented yet.
  • Populated SQLite v2-to-v4 upgrade coverage remains to be added.
  • Crash harness support targets Linux/macOS; Windows remains limited support.
  • definitely_not_dispatched is classified but its Phase 3 automated recovery policy is not implemented.
  • Legacy tool-projection rebuilding remains compatibility-only and must not become a second recovery truth source.
  • Manual and startup auto-resume still share one feature flag; they should be split before broader rollout.

Verification

  • npm run typecheck (all workspaces)
  • RecoveryResolver suite: 11 passed
  • empty continuation replay provider guard: passed
  • Desktop interrupted-banner routing and eligibility: 6 passed
  • user-facing recovery copy: 3 passed
  • changed-file Biome lint and git diff --check

@likun666661

likun666661 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Review conclusion

This is a novel and technically strong approach to runtime resume. The PR does not try to revive an old JavaScript stack, provider stream, Promise, or operating-system process. Instead, it reconstructs durable execution facts, proves that a continuation boundary is safe, and then starts a new Run, Invocation, and Turn.

The most important design choice is the distinction between replaying tool history and re-executing tool side effects:

  • completed function_call and function_response pairs are replayed to the provider as history;
  • the previous tool implementation is not executed again;
  • an unmatched or ambiguous tool call is parked rather than retried;
  • contradictory recovery facts fail closed.

This gives Maka a useful resume primitive without claiming exactly-once execution for arbitrary Bash commands, filesystem mutations, or external APIs.

Problem being solved

The existing RuntimeEvent ledger could answer what had been recorded, but it could not reliably answer whether a tool might already have crossed the side-effect boundary.

Two crash windows are particularly dangerous:

  1. The tool implementation starts before the canonical call fact is durably committed.
  2. The tool finishes its side effect, but the process crashes before the canonical response is durably committed.

In either case, treating a missing result as proof that the tool did not execute can duplicate writes, deletes, deployments, payments, or remote requests. The previous startup repair path could converge a dangling Run to a failed terminal state, but it could not safely continue execution.

This PR solves that by adding a durable tool-dispatch boundary, deterministic recovery projection, and safe-boundary continuation.

Module view

flowchart LR
subgraph Entry["Product entry points"]
Desktop["Desktop Safe resume and startup recovery"]
CLI["CLI and TUI /resume"]
end
subgraph Control["Recovery control plane"]
SessionManager["SessionManager<br/>candidate discovery and lifecycle"]
Inspector["ContinuationSafetyInspector<br/>cwd, workspace, tools, background work"]
Planner["RuntimeContinuationPlanner<br/>continue or park"]
Resolver["RecoveryResolver<br/>single decision authority"]
end
subgraph Execution["Execution plane"]
Kernel["RuntimeKernel<br/>revalidation and claim"]
AgentRun["AgentRun<br/>fresh execution identity"]
Runner["RuntimeRunner<br/>continuation-start fact"]
Backend["AiSdkFlow and AiSdkBackend<br/>provider replay"]
ToolRuntime["ToolRuntime<br/>preflight, T1, implementation, T2"]
end
subgraph Persistence["Persistence"]
Bootstrap["openRuntimeEventPersistence"]
SQLite[("runtime.sqlite<br/>canonical RuntimeEvent store")]
Projection["tool_journal_events and tool_operations<br/>rebuildable projections"]
JSONL["Legacy RuntimeEvent JSONL<br/>import and explicit export"]
end
Desktop --> SessionManager
CLI --> SessionManager
SessionManager --> Inspector
SessionManager --> Planner
Inspector --> Planner
SQLite --> Planner
Planner --> Resolver
Resolver --> Planner
Planner -->|"approved continuation"| Kernel
Planner -->|"park with stable reason codes"| Desktop
Kernel --> AgentRun
AgentRun --> Runner
Runner --> Backend
Backend --> ToolRuntime
Runner -->|"continuation-start"| SQLite
ToolRuntime -->|"T1 and T2"| SQLite
SQLite -.->|"synchronous projection"| Projection
Bootstrap --> SQLite
Bootstrap -.->|"unmigrated workspace and flag disabled"| JSONL
JSONL -->|"sticky, idempotent import"| SQLite
Loading

The architectural boundary is clear:

  • RuntimeEvent is the canonical recovery fact source.
  • RecoveryResolver is the only component that interprets tool recovery state.
  • Tool journal and operation tables are query projections and can be rebuilt from RuntimeEvents.
  • Product surfaces consume stable plans and reason codes instead of implementing their own recovery logic.

How the tool boundary closes the crash windows

sequenceDiagram
autonumber
participant Provider as Model Provider
participant Backend as AiSdkBackend
participant Runtime as ToolRuntime
participant Store as SqliteRuntimeStore
participant Tool as Tool Implementation
Provider->>Backend: function_call
Backend->>Runtime: execute tool
Runtime->>Runtime: validate args, availability, loop, permissions, and runtime guards
Runtime->>Store: T1 commitToolPrepared
Note over Store: Insert or verify function_call<br/>append toolDispatch RuntimeEvent<br/>update journal and operation projection
alt T1 fails
Store--xRuntime: rollback
Note right of Runtime: Tool implementation is never called
else T1 commits
Store-->>Runtime: durable dispatch boundary
Runtime->>Tool: execute implementation
Tool-->>Runtime: result or error
Runtime->>Store: T2 commitToolOutcome
Note over Store: Append function_response<br/>append outcome journal<br/>CAS operation state
alt T2 fails
Store--xRuntime: rollback
Note right of Runtime: Result is not returned to the next model step
else T2 commits
Store-->>Runtime: durable outcome
Runtime-->>Backend: tool_result
Backend-->>Provider: next provider step
end
end
Loading

This establishes two important invariants:

  1. If T1 fails, the tool implementation runs zero times.
  2. If T2 fails, the tool result cannot reach the next model step.

The external side effect still occurs between two short database transactions, so SQLite cannot make an arbitrary external operation atomic. Instead, a crash between T1 and T2 becomes an explicit indeterminate state that recovery must park or reconcile later.

How safe continuation works

sequenceDiagram
autonumber
participant Entry as User or Startup Recovery
participant Manager as SessionManager
participant Inspector as Safety Inspector
participant Planner as Continuation Planner
participant Resolver as RecoveryResolver
participant Store as Durable Stores
participant Kernel as RuntimeKernel
participant Provider as Model Provider
Entry->>Manager: resumeLatest
Manager->>Store: find latest failed or cancelled session-inline Run
Manager->>Inspector: inspect cwd, workspace identity, tool catalog, and background work
Inspector-->>Manager: authoritative safety observation
Manager->>Planner: plan source boundary
Planner->>Store: read Run header, RuntimeEvents, and continuation ancestors
Planner->>Resolver: resolve tool recovery facts
Resolver-->>Planner: completed, indeterminate, not dispatched, or corruption
alt Any safety condition fails
Planner-->>Manager: park with stable reason codes
Manager-->>Entry: explain why resume is unavailable
Note over Entry,Provider: Provider is not called
else Boundary is safe
Planner-->>Manager: continuation plan with fresh identities and safety snapshot
Manager->>Kernel: execute continuation
Kernel->>Store: re-read source terminal, high-water, identity, and replay
Kernel->>Inspector: revalidate external safety facts
Kernel->>Store: create target Run with continuationSource claim
Kernel->>Store: commit continuation-start RuntimeEvent
Kernel->>Provider: replay committed history without a duplicate user message
Provider-->>Kernel: continue from the validated boundary
end
Loading

Planning and execution are intentionally separate. A valid plan is not treated as an execution lease. Immediately before execution, the Runtime re-reads durable state and rechecks workspace identity, active operations, tool availability, source high-water, and replay equality.

The continuation also creates new execution identity:

source Session / Invocation / Run / Turn
-> validated RuntimeEvent high-water
-> new Invocation / Run / Turn
-> durable continuation-start
-> provider replay without a duplicate user message

The source ledger is not mutated by the continuation.

Recovery decisions

Durable factsResolver decisionCurrent behavior
call + matching responsecompletedReplay call and response as provider history
call + dispatch + no responseindeterminatePark; never blindly retry
call + no dispatch + new protocol markerdefinitely not dispatchedClassified, but currently still parked until Phase 3 policy exists
call + no dispatch + legacy or unknown protocolindeterminatePark
orphan, duplicate, or identity-conflicting factscorruptionFail closed

This is effectively tool-log replay: the provider receives the completed tool interaction as historical context, while the Runtime avoids replaying the old implementation or side effect.

What this PR introduces

AreaIntroduced capability
Recovery semanticsRecoveryResolver, deterministic tool-operation projection, stable diagnostics and park reasons
ContinuationRuntimeContinuationPlanner, fresh execution identities, durable continuation claim, execution-time revalidation
Tool protocoltoolDispatch RuntimeEvent, protocol marker, deterministic operation ID, canonical argument hash, recovery mode
Transaction boundaryRuntimeCommitSink, T1 commitToolPrepared, T2 commitToolOutcome, CAS updates
Canonical storageSQLite RuntimeEvent store, WAL, synchronous=FULL, foreign keys, schema migrations
CompatibilitySticky JSONL-to-SQLite import, source fingerprints, explicit JSONL export
Product surfacesDesktop Safe resume, startup continuation, CLI and TUI /resume, user-facing park copy
ValidationProcess-crash harnesses, replay tests, continuation tests, SQLite crash tests, and Linux/macOS CI coverage

Current boundaries

The conservative scope is appropriate, but it is important to preserve these limitations:

  • this PR does not re-execute historical tool implementations;
  • Phase 3 reconciliation and restricted verification are not implemented;
  • definitely_not_dispatched is classified but does not yet trigger automatic recovery;
  • general Bash and unknown external side effects remain parked;
  • replay-safe, idempotent, reconcile, and reattach tool contracts are future work;
  • SQLite migration is sticky and does not yet provide automatic pre-migration backup;
  • manual resume and startup auto-resume currently share one feature flag;
  • multi-worker leases and fencing are intentionally out of scope.

Future value of checkpoints and Git snapshots

The next major step that could make this resume system broadly practical is coupling the RuntimeEvent high-water with a workspace checkpoint, especially a Git-backed snapshot.

That would let recovery validate or restore both sides of the continuation boundary:

  1. the model and tool interaction history; and
  2. the exact agent-visible workspace state associated with that history.

A durable boundary containing:

RuntimeEvent high-water
+ workspace checkpoint reference
+ Git commit or tree snapshot
+ workspace identity

would allow Maka to resume after a crash without assuming that the current filesystem still matches the interrupted Run. It would also make planning and execution revalidation stronger because the continuation could prove that conversational state and workspace state refer to the same execution boundary.

Overall assessment

This is an excellent foundation and a genuinely interesting resume design.

The durable tool-call protocol makes safe history replay possible without conflating replay with side-effect re-execution. The fail-closed behavior is correct, the new execution identity keeps lineage understandable, and the single canonical recovery authority avoids parallel state machines.

With checkpoint and Git snapshot integration, this design could evolve from safe conversational continuation into a practical end-to-end runtime resume system.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

Conflict-resolution verification update

The branch was reconciled with current main through local merge commits 81edf2ba (Phase 1) and f0653e02 (Phase 2), then pushed without including local dogfooding files.

Additional compatibility fixes made during the merge:

  • aligned SQLite canonical storage with the current terminal durability barrier;
  • taught the exact-shape RuntimeEvent decoder about toolDispatch, runtimeProtocol, and operationId;
  • kept legacy JSONL import independent of a surviving run.json header;
  • preserved continuation invocation identity and session-inline projection semantics;
  • retained current steering, strict-recovery, child-agent, and Desktop IPC contracts.

Verification after reconciliation:

  • npm run build:test — passed across all workspaces;
  • npm run typecheck — passed across all workspaces;
  • Runtime resume/continuation/runner/tool-boundary focused suites — 62 passed;
  • SQLite runtime store — 10 passed;
  • legacy JSONL-to-SQLite transfer — 5 passed;
  • real-process Runtime resume crash harness — passed;
  • Desktop interrupted-banner routing — 4 passed;
  • focused SessionManager continuation/canonical/UI projection cases — 3 passed.

Windows-only file fsync/signal harness limitations remain documented; Linux/macOS are the supported crash-durability targets.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

CI follow-up is complete on head \

@zhiiw
zhiiw marked this pull request as ready for review July 19, 2026 10:40
@Astro-Han
Astro-Han merged commit b5e015a into apache:mainJul 19, 2026
5 checks passed
@zhiiw
zhiiw deleted the codex/runtime-resume-phase2 branch July 19, 2026 11:03
jackwener added a commit that referenced this pull request Jul 19, 2026
…nip zero-hint, resume hook, QR fixture (#1241)
* feat(core): add settings-bots-onboarding visual-smoke scenario + botOnboardingProvider
* feat(desktop): hold-in-waiting bot onboarding adapter + fixture state for QR capture
* feat(desktop): auto-open bot scan-login modal under settings-bots-onboarding fixture
* chore(scripts): register settings-bots-onboarding in capture + audit scenario lists
* fix(desktop): wait for QR paint before auto-capturing bot-onboarding fixture
* docs(notes): frontend architecture map — measured baseline + 8 staged rounds (2026-07-19)
Re-ran the census (wc over non-test TS/TSX for desktop main/preload/renderer +
packages/ui; CSS totals; top-10 hotspot table incl. visual-smoke-fixture.ts marked
OUT-OF-SCOPE other-sessions' territory). Staged R1 knip de-rot, R2 app-shell
resume-cluster extraction, R3 visual-smoke split (BLOCKED on concurrent fixture branch),
R4-R6 main.ts extractions (require maintainer-approved contract re-pins — registerIpc /
startup / tool-assembly / modelSupportsVision direct-pinned), R7 provider-connection-detail
decomposition (needs its own blade plan), R8 CSS raw-hex residue (verified clean on this
tip — #1085 already converted it).
* chore(knip): de-rot config — clear all 13 stale hints (R1)
knip reported 13 configuration hints (7 desktop + 6 ui), all redundant/stale config,
zero code impact:
- Drop 'overlayscrollbars' from desktop ignoreDependencies — now redundant (knip no
longer flags it; the package is owned by packages/ui and only referenced there). The
overlay-scrollbars contract forbids declaring it as a *desktop dependency*, not as an
ignore — removing a redundant ignore is compatible and coverage is unchanged.
- Delete the dead 'src/renderer/**/*.test.ts' entry glob (no matches — renderer has zero
test files; all 338 desktop tests live under src/main/__tests__/*.test.ts).
- Drop 11 redundant explicit entries knip already auto-detects: desktop
preload.ts / renderer/main.tsx / playwright.config.ts / .storybook/main.ts / dev.mjs
(vite / playwright / storybook / npm-scripts plugins), and all 6 ui entries
(index.ts, icons.tsx, artifact-preview-registry.ts, assistant-stream.ts, maka-uri.ts,
smooth-stream.ts) resolved from packages/ui package.json#exports.
Non-redundant entries kept verbatim (main.ts, overlay/*, main test glob, e2e specs,
storybook preview, stories, browser-observe-act-smoke.mjs, both ui test globs). No broad
ignores added — coverage is not weakened. knip --workspace apps/desktop and packages/ui
now both exit 0 AND report zero config hints (was: exit 0 with 13 hints).
* refactor(renderer): extract app-shell resume cluster into use-shell-resume (R2)
The #1223 safe-boundary resume cluster moves out of app-shell.tsx into a new
use-shell-resume.ts hook, following the use-shell-connections / use-shell-chat-model
house style (options object; state + handler returned; stable identities preserved).
Pure move, zero behavior change: the two useState declarations (resumePendingSessionId,
resumeParkDescriptionBySession) and the resumeInterruptedSession handler move verbatim;
activeId/toastApi/shellCopy/uiLocale are injected as options. app-shell keeps the banner
JSX wiring (safeResumeAction=) and the sendWithAttachments guard. resumeParkToastCopy is
no longer imported into app-shell (it now lives in the hook). app-shell.tsx 1686 -> 1654
(-32 lines).
Contract re-pin (never deletes coverage): runtime-resume-routing-contract.test.ts read
the handler-shape assertions (resumeInterruptedSession / resumeLatest(sessionId) /
resumeParkToastCopy / no rejectionReasons.join) from app-shell.tsx single-file; they now
read use-shell-resume.ts where that logic lives, while the app-shell-specific assertions
(sendWithAttachments '/resume' guard, safeResumeAction= wiring) still read app-shell.tsx.
use-shell-resume.ts added to renderer-shell-source-helpers sourcePaths (Round B/E
precedent) so combined-source contracts see it.
Gates: desktop 2740/2740, ui 196/196, typecheck 0, check-dead-css clean, knip
desktop+ui exit 0, AUDIT_PORT_BASE=23900 alignment auditor exit 0 (all 10 fixtures
clean, real renderer). CDP turn-narrative branch-vs-baseline (real Electron, light+dark
1280) byte-identical (sha256 match) — proven render no-op.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@zhiiw@likun666661@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat(runtime): add durable safe-boundary resume phases 0-2 - #1223

Merged
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2
Jul 19, 2026
Merged

feat(runtime): add durable safe-boundary resume phases 0-2#1223
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2

Conversation

@zhiiw

Copy link
Copy Markdown
Contributor

Summary

Implements the Runtime Resume foundation from Phase 0 through Phase 2, with RuntimeEvent as the single canonical recovery fact source.

  • Phase 0: fail-closed RuntimeEvent replay projection, unmatched tool-call parking, terminal repair, and real process-crash coverage.
  • Phase 1: safe-boundary continuation into fresh Run/Turn/Invocation identities, durable continuation claims, plan/execute separation, and execution-time revalidation.
  • Phase 2: sticky SQLite canonical RuntimeEvent storage, transactional T1/T2 tool boundaries, operation identity/CAS, import/export, and one canonical writer.
  • Phase 2.5 convergence: RecoveryResolver is the sole decision authority; tool journal tables are rebuildable projections, not a parallel truth source.

The Desktop manual recovery entry is attached to the interrupted-turn banner as Safe resume. CLI/TUI retain /resume. Park reasons are translated for users while stable reason codes remain available to diagnostics.

Related to #186.

Safety and rollout

Both capabilities remain disabled by default:

FlagBehavior
MAKA_RUNTIME_SQLITE_CANONICAL=1Triggers a sticky, one-way workspace migration to runtime.sqlite; disabling the flag does not return that workspace to JSONL.
MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1Enables safe-boundary recovery surfaces and Desktop startup auto-resume; these paths may call the provider and consume tokens.

Ambiguous tool outcomes are parked, never blindly retried. Phase 3 reconciliation is intentionally outside this PR.

Dogfooding and regression fixes

Real restart/resume testing found and fixed:

  1. second-generation continuation envelopes dropping source-only replay context;
  2. continuation replay/history reconstruction gaps across ancestor runs;
  3. continuation runs being misclassified as child-agent runs in model, UI, compaction, and repair projections;
  4. canonical continuation-start facts being treated as unsupported chat events;
  5. empty materialized continuation replay reaching the provider.

The final recovery gates also reject duplicate dispatch/response facts and unknown runtime protocol markers as corruption.

Known limitations

  • Automatic pre-migration backup is not implemented yet.
  • Populated SQLite v2-to-v4 upgrade coverage remains to be added.
  • Crash harness support targets Linux/macOS; Windows remains limited support.
  • definitely_not_dispatched is classified but its Phase 3 automated recovery policy is not implemented.
  • Legacy tool-projection rebuilding remains compatibility-only and must not become a second recovery truth source.
  • Manual and startup auto-resume still share one feature flag; they should be split before broader rollout.

Verification

  • npm run typecheck (all workspaces)
  • RecoveryResolver suite: 11 passed
  • empty continuation replay provider guard: passed
  • Desktop interrupted-banner routing and eligibility: 6 passed
  • user-facing recovery copy: 3 passed
  • changed-file Biome lint and git diff --check

@likun666661

likun666661 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Review conclusion

This is a novel and technically strong approach to runtime resume. The PR does not try to revive an old JavaScript stack, provider stream, Promise, or operating-system process. Instead, it reconstructs durable execution facts, proves that a continuation boundary is safe, and then starts a new Run, Invocation, and Turn.

The most important design choice is the distinction between replaying tool history and re-executing tool side effects:

  • completed function_call and function_response pairs are replayed to the provider as history;
  • the previous tool implementation is not executed again;
  • an unmatched or ambiguous tool call is parked rather than retried;
  • contradictory recovery facts fail closed.

This gives Maka a useful resume primitive without claiming exactly-once execution for arbitrary Bash commands, filesystem mutations, or external APIs.

Problem being solved

The existing RuntimeEvent ledger could answer what had been recorded, but it could not reliably answer whether a tool might already have crossed the side-effect boundary.

Two crash windows are particularly dangerous:

  1. The tool implementation starts before the canonical call fact is durably committed.
  2. The tool finishes its side effect, but the process crashes before the canonical response is durably committed.

In either case, treating a missing result as proof that the tool did not execute can duplicate writes, deletes, deployments, payments, or remote requests. The previous startup repair path could converge a dangling Run to a failed terminal state, but it could not safely continue execution.

This PR solves that by adding a durable tool-dispatch boundary, deterministic recovery projection, and safe-boundary continuation.

Module view

flowchart LR
subgraph Entry["Product entry points"]
Desktop["Desktop Safe resume and startup recovery"]
CLI["CLI and TUI /resume"]
end
subgraph Control["Recovery control plane"]
SessionManager["SessionManager<br/>candidate discovery and lifecycle"]
Inspector["ContinuationSafetyInspector<br/>cwd, workspace, tools, background work"]
Planner["RuntimeContinuationPlanner<br/>continue or park"]
Resolver["RecoveryResolver<br/>single decision authority"]
end
subgraph Execution["Execution plane"]
Kernel["RuntimeKernel<br/>revalidation and claim"]
AgentRun["AgentRun<br/>fresh execution identity"]
Runner["RuntimeRunner<br/>continuation-start fact"]
Backend["AiSdkFlow and AiSdkBackend<br/>provider replay"]
ToolRuntime["ToolRuntime<br/>preflight, T1, implementation, T2"]
end
subgraph Persistence["Persistence"]
Bootstrap["openRuntimeEventPersistence"]
SQLite[("runtime.sqlite<br/>canonical RuntimeEvent store")]
Projection["tool_journal_events and tool_operations<br/>rebuildable projections"]
JSONL["Legacy RuntimeEvent JSONL<br/>import and explicit export"]
end
Desktop --> SessionManager
CLI --> SessionManager
SessionManager --> Inspector
SessionManager --> Planner
Inspector --> Planner
SQLite --> Planner
Planner --> Resolver
Resolver --> Planner
Planner -->|"approved continuation"| Kernel
Planner -->|"park with stable reason codes"| Desktop
Kernel --> AgentRun
AgentRun --> Runner
Runner --> Backend
Backend --> ToolRuntime
Runner -->|"continuation-start"| SQLite
ToolRuntime -->|"T1 and T2"| SQLite
SQLite -.->|"synchronous projection"| Projection
Bootstrap --> SQLite
Bootstrap -.->|"unmigrated workspace and flag disabled"| JSONL
JSONL -->|"sticky, idempotent import"| SQLite
Loading

The architectural boundary is clear:

  • RuntimeEvent is the canonical recovery fact source.
  • RecoveryResolver is the only component that interprets tool recovery state.
  • Tool journal and operation tables are query projections and can be rebuilt from RuntimeEvents.
  • Product surfaces consume stable plans and reason codes instead of implementing their own recovery logic.

How the tool boundary closes the crash windows

sequenceDiagram
autonumber
participant Provider as Model Provider
participant Backend as AiSdkBackend
participant Runtime as ToolRuntime
participant Store as SqliteRuntimeStore
participant Tool as Tool Implementation
Provider->>Backend: function_call
Backend->>Runtime: execute tool
Runtime->>Runtime: validate args, availability, loop, permissions, and runtime guards
Runtime->>Store: T1 commitToolPrepared
Note over Store: Insert or verify function_call<br/>append toolDispatch RuntimeEvent<br/>update journal and operation projection
alt T1 fails
Store--xRuntime: rollback
Note right of Runtime: Tool implementation is never called
else T1 commits
Store-->>Runtime: durable dispatch boundary
Runtime->>Tool: execute implementation
Tool-->>Runtime: result or error
Runtime->>Store: T2 commitToolOutcome
Note over Store: Append function_response<br/>append outcome journal<br/>CAS operation state
alt T2 fails
Store--xRuntime: rollback
Note right of Runtime: Result is not returned to the next model step
else T2 commits
Store-->>Runtime: durable outcome
Runtime-->>Backend: tool_result
Backend-->>Provider: next provider step
end
end
Loading

This establishes two important invariants:

  1. If T1 fails, the tool implementation runs zero times.
  2. If T2 fails, the tool result cannot reach the next model step.

The external side effect still occurs between two short database transactions, so SQLite cannot make an arbitrary external operation atomic. Instead, a crash between T1 and T2 becomes an explicit indeterminate state that recovery must park or reconcile later.

How safe continuation works

sequenceDiagram
autonumber
participant Entry as User or Startup Recovery
participant Manager as SessionManager
participant Inspector as Safety Inspector
participant Planner as Continuation Planner
participant Resolver as RecoveryResolver
participant Store as Durable Stores
participant Kernel as RuntimeKernel
participant Provider as Model Provider
Entry->>Manager: resumeLatest
Manager->>Store: find latest failed or cancelled session-inline Run
Manager->>Inspector: inspect cwd, workspace identity, tool catalog, and background work
Inspector-->>Manager: authoritative safety observation
Manager->>Planner: plan source boundary
Planner->>Store: read Run header, RuntimeEvents, and continuation ancestors
Planner->>Resolver: resolve tool recovery facts
Resolver-->>Planner: completed, indeterminate, not dispatched, or corruption
alt Any safety condition fails
Planner-->>Manager: park with stable reason codes
Manager-->>Entry: explain why resume is unavailable
Note over Entry,Provider: Provider is not called
else Boundary is safe
Planner-->>Manager: continuation plan with fresh identities and safety snapshot
Manager->>Kernel: execute continuation
Kernel->>Store: re-read source terminal, high-water, identity, and replay
Kernel->>Inspector: revalidate external safety facts
Kernel->>Store: create target Run with continuationSource claim
Kernel->>Store: commit continuation-start RuntimeEvent
Kernel->>Provider: replay committed history without a duplicate user message
Provider-->>Kernel: continue from the validated boundary
end
Loading

Planning and execution are intentionally separate. A valid plan is not treated as an execution lease. Immediately before execution, the Runtime re-reads durable state and rechecks workspace identity, active operations, tool availability, source high-water, and replay equality.

The continuation also creates new execution identity:

source Session / Invocation / Run / Turn
-> validated RuntimeEvent high-water
-> new Invocation / Run / Turn
-> durable continuation-start
-> provider replay without a duplicate user message

The source ledger is not mutated by the continuation.

Recovery decisions

Durable factsResolver decisionCurrent behavior
call + matching responsecompletedReplay call and response as provider history
call + dispatch + no responseindeterminatePark; never blindly retry
call + no dispatch + new protocol markerdefinitely not dispatchedClassified, but currently still parked until Phase 3 policy exists
call + no dispatch + legacy or unknown protocolindeterminatePark
orphan, duplicate, or identity-conflicting factscorruptionFail closed

This is effectively tool-log replay: the provider receives the completed tool interaction as historical context, while the Runtime avoids replaying the old implementation or side effect.

What this PR introduces

AreaIntroduced capability
Recovery semanticsRecoveryResolver, deterministic tool-operation projection, stable diagnostics and park reasons
ContinuationRuntimeContinuationPlanner, fresh execution identities, durable continuation claim, execution-time revalidation
Tool protocoltoolDispatch RuntimeEvent, protocol marker, deterministic operation ID, canonical argument hash, recovery mode
Transaction boundaryRuntimeCommitSink, T1 commitToolPrepared, T2 commitToolOutcome, CAS updates
Canonical storageSQLite RuntimeEvent store, WAL, synchronous=FULL, foreign keys, schema migrations
CompatibilitySticky JSONL-to-SQLite import, source fingerprints, explicit JSONL export
Product surfacesDesktop Safe resume, startup continuation, CLI and TUI /resume, user-facing park copy
ValidationProcess-crash harnesses, replay tests, continuation tests, SQLite crash tests, and Linux/macOS CI coverage

Current boundaries

The conservative scope is appropriate, but it is important to preserve these limitations:

  • this PR does not re-execute historical tool implementations;
  • Phase 3 reconciliation and restricted verification are not implemented;
  • definitely_not_dispatched is classified but does not yet trigger automatic recovery;
  • general Bash and unknown external side effects remain parked;
  • replay-safe, idempotent, reconcile, and reattach tool contracts are future work;
  • SQLite migration is sticky and does not yet provide automatic pre-migration backup;
  • manual resume and startup auto-resume currently share one feature flag;
  • multi-worker leases and fencing are intentionally out of scope.

Future value of checkpoints and Git snapshots

The next major step that could make this resume system broadly practical is coupling the RuntimeEvent high-water with a workspace checkpoint, especially a Git-backed snapshot.

That would let recovery validate or restore both sides of the continuation boundary:

  1. the model and tool interaction history; and
  2. the exact agent-visible workspace state associated with that history.

A durable boundary containing:

RuntimeEvent high-water
+ workspace checkpoint reference
+ Git commit or tree snapshot
+ workspace identity

would allow Maka to resume after a crash without assuming that the current filesystem still matches the interrupted Run. It would also make planning and execution revalidation stronger because the continuation could prove that conversational state and workspace state refer to the same execution boundary.

Overall assessment

This is an excellent foundation and a genuinely interesting resume design.

The durable tool-call protocol makes safe history replay possible without conflating replay with side-effect re-execution. The fail-closed behavior is correct, the new execution identity keeps lineage understandable, and the single canonical recovery authority avoids parallel state machines.

With checkpoint and Git snapshot integration, this design could evolve from safe conversational continuation into a practical end-to-end runtime resume system.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

Conflict-resolution verification update

The branch was reconciled with current main through local merge commits 81edf2ba (Phase 1) and f0653e02 (Phase 2), then pushed without including local dogfooding files.

Additional compatibility fixes made during the merge:

  • aligned SQLite canonical storage with the current terminal durability barrier;
  • taught the exact-shape RuntimeEvent decoder about toolDispatch, runtimeProtocol, and operationId;
  • kept legacy JSONL import independent of a surviving run.json header;
  • preserved continuation invocation identity and session-inline projection semantics;
  • retained current steering, strict-recovery, child-agent, and Desktop IPC contracts.

Verification after reconciliation:

  • npm run build:test — passed across all workspaces;
  • npm run typecheck — passed across all workspaces;
  • Runtime resume/continuation/runner/tool-boundary focused suites — 62 passed;
  • SQLite runtime store — 10 passed;
  • legacy JSONL-to-SQLite transfer — 5 passed;
  • real-process Runtime resume crash harness — passed;
  • Desktop interrupted-banner routing — 4 passed;
  • focused SessionManager continuation/canonical/UI projection cases — 3 passed.

Windows-only file fsync/signal harness limitations remain documented; Linux/macOS are the supported crash-durability targets.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

CI follow-up is complete on head \

@zhiiw
zhiiw marked this pull request as ready for review July 19, 2026 10:40
@Astro-Han
Astro-Han merged commit b5e015a into apache:mainJul 19, 2026
5 checks passed
@zhiiw
zhiiw deleted the codex/runtime-resume-phase2 branch July 19, 2026 11:03
jackwener added a commit that referenced this pull request Jul 19, 2026
…nip zero-hint, resume hook, QR fixture (#1241)
* feat(core): add settings-bots-onboarding visual-smoke scenario + botOnboardingProvider
* feat(desktop): hold-in-waiting bot onboarding adapter + fixture state for QR capture
* feat(desktop): auto-open bot scan-login modal under settings-bots-onboarding fixture
* chore(scripts): register settings-bots-onboarding in capture + audit scenario lists
* fix(desktop): wait for QR paint before auto-capturing bot-onboarding fixture
* docs(notes): frontend architecture map — measured baseline + 8 staged rounds (2026-07-19)
Re-ran the census (wc over non-test TS/TSX for desktop main/preload/renderer +
packages/ui; CSS totals; top-10 hotspot table incl. visual-smoke-fixture.ts marked
OUT-OF-SCOPE other-sessions' territory). Staged R1 knip de-rot, R2 app-shell
resume-cluster extraction, R3 visual-smoke split (BLOCKED on concurrent fixture branch),
R4-R6 main.ts extractions (require maintainer-approved contract re-pins — registerIpc /
startup / tool-assembly / modelSupportsVision direct-pinned), R7 provider-connection-detail
decomposition (needs its own blade plan), R8 CSS raw-hex residue (verified clean on this
tip — #1085 already converted it).
* chore(knip): de-rot config — clear all 13 stale hints (R1)
knip reported 13 configuration hints (7 desktop + 6 ui), all redundant/stale config,
zero code impact:
- Drop 'overlayscrollbars' from desktop ignoreDependencies — now redundant (knip no
longer flags it; the package is owned by packages/ui and only referenced there). The
overlay-scrollbars contract forbids declaring it as a *desktop dependency*, not as an
ignore — removing a redundant ignore is compatible and coverage is unchanged.
- Delete the dead 'src/renderer/**/*.test.ts' entry glob (no matches — renderer has zero
test files; all 338 desktop tests live under src/main/__tests__/*.test.ts).
- Drop 11 redundant explicit entries knip already auto-detects: desktop
preload.ts / renderer/main.tsx / playwright.config.ts / .storybook/main.ts / dev.mjs
(vite / playwright / storybook / npm-scripts plugins), and all 6 ui entries
(index.ts, icons.tsx, artifact-preview-registry.ts, assistant-stream.ts, maka-uri.ts,
smooth-stream.ts) resolved from packages/ui package.json#exports.
Non-redundant entries kept verbatim (main.ts, overlay/*, main test glob, e2e specs,
storybook preview, stories, browser-observe-act-smoke.mjs, both ui test globs). No broad
ignores added — coverage is not weakened. knip --workspace apps/desktop and packages/ui
now both exit 0 AND report zero config hints (was: exit 0 with 13 hints).
* refactor(renderer): extract app-shell resume cluster into use-shell-resume (R2)
The #1223 safe-boundary resume cluster moves out of app-shell.tsx into a new
use-shell-resume.ts hook, following the use-shell-connections / use-shell-chat-model
house style (options object; state + handler returned; stable identities preserved).
Pure move, zero behavior change: the two useState declarations (resumePendingSessionId,
resumeParkDescriptionBySession) and the resumeInterruptedSession handler move verbatim;
activeId/toastApi/shellCopy/uiLocale are injected as options. app-shell keeps the banner
JSX wiring (safeResumeAction=) and the sendWithAttachments guard. resumeParkToastCopy is
no longer imported into app-shell (it now lives in the hook). app-shell.tsx 1686 -> 1654
(-32 lines).
Contract re-pin (never deletes coverage): runtime-resume-routing-contract.test.ts read
the handler-shape assertions (resumeInterruptedSession / resumeLatest(sessionId) /
resumeParkToastCopy / no rejectionReasons.join) from app-shell.tsx single-file; they now
read use-shell-resume.ts where that logic lives, while the app-shell-specific assertions
(sendWithAttachments '/resume' guard, safeResumeAction= wiring) still read app-shell.tsx.
use-shell-resume.ts added to renderer-shell-source-helpers sourcePaths (Round B/E
precedent) so combined-source contracts see it.
Gates: desktop 2740/2740, ui 196/196, typecheck 0, check-dead-css clean, knip
desktop+ui exit 0, AUDIT_PORT_BASE=23900 alignment auditor exit 0 (all 10 fixtures
clean, real renderer). CDP turn-narrative branch-vs-baseline (real Electron, light+dark
1280) byte-identical (sha256 match) — proven render no-op.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@zhiiw@likun666661@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): add durable safe-boundary resume phases 0-2 - #1223

Merged
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2
Jul 19, 2026
Merged

feat(runtime): add durable safe-boundary resume phases 0-2#1223
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2

Conversation

@zhiiw

Copy link
Copy Markdown
Contributor

Summary

Implements the Runtime Resume foundation from Phase 0 through Phase 2, with RuntimeEvent as the single canonical recovery fact source.

  • Phase 0: fail-closed RuntimeEvent replay projection, unmatched tool-call parking, terminal repair, and real process-crash coverage.
  • Phase 1: safe-boundary continuation into fresh Run/Turn/Invocation identities, durable continuation claims, plan/execute separation, and execution-time revalidation.
  • Phase 2: sticky SQLite canonical RuntimeEvent storage, transactional T1/T2 tool boundaries, operation identity/CAS, import/export, and one canonical writer.
  • Phase 2.5 convergence: RecoveryResolver is the sole decision authority; tool journal tables are rebuildable projections, not a parallel truth source.

The Desktop manual recovery entry is attached to the interrupted-turn banner as Safe resume. CLI/TUI retain /resume. Park reasons are translated for users while stable reason codes remain available to diagnostics.

Related to #186.

Safety and rollout

Both capabilities remain disabled by default:

FlagBehavior
MAKA_RUNTIME_SQLITE_CANONICAL=1Triggers a sticky, one-way workspace migration to runtime.sqlite; disabling the flag does not return that workspace to JSONL.
MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1Enables safe-boundary recovery surfaces and Desktop startup auto-resume; these paths may call the provider and consume tokens.

Ambiguous tool outcomes are parked, never blindly retried. Phase 3 reconciliation is intentionally outside this PR.

Dogfooding and regression fixes

Real restart/resume testing found and fixed:

  1. second-generation continuation envelopes dropping source-only replay context;
  2. continuation replay/history reconstruction gaps across ancestor runs;
  3. continuation runs being misclassified as child-agent runs in model, UI, compaction, and repair projections;
  4. canonical continuation-start facts being treated as unsupported chat events;
  5. empty materialized continuation replay reaching the provider.

The final recovery gates also reject duplicate dispatch/response facts and unknown runtime protocol markers as corruption.

Known limitations

  • Automatic pre-migration backup is not implemented yet.
  • Populated SQLite v2-to-v4 upgrade coverage remains to be added.
  • Crash harness support targets Linux/macOS; Windows remains limited support.
  • definitely_not_dispatched is classified but its Phase 3 automated recovery policy is not implemented.
  • Legacy tool-projection rebuilding remains compatibility-only and must not become a second recovery truth source.
  • Manual and startup auto-resume still share one feature flag; they should be split before broader rollout.

Verification

  • npm run typecheck (all workspaces)
  • RecoveryResolver suite: 11 passed
  • empty continuation replay provider guard: passed
  • Desktop interrupted-banner routing and eligibility: 6 passed
  • user-facing recovery copy: 3 passed
  • changed-file Biome lint and git diff --check

@likun666661

likun666661 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Review conclusion

This is a novel and technically strong approach to runtime resume. The PR does not try to revive an old JavaScript stack, provider stream, Promise, or operating-system process. Instead, it reconstructs durable execution facts, proves that a continuation boundary is safe, and then starts a new Run, Invocation, and Turn.

The most important design choice is the distinction between replaying tool history and re-executing tool side effects:

  • completed function_call and function_response pairs are replayed to the provider as history;
  • the previous tool implementation is not executed again;
  • an unmatched or ambiguous tool call is parked rather than retried;
  • contradictory recovery facts fail closed.

This gives Maka a useful resume primitive without claiming exactly-once execution for arbitrary Bash commands, filesystem mutations, or external APIs.

Problem being solved

The existing RuntimeEvent ledger could answer what had been recorded, but it could not reliably answer whether a tool might already have crossed the side-effect boundary.

Two crash windows are particularly dangerous:

  1. The tool implementation starts before the canonical call fact is durably committed.
  2. The tool finishes its side effect, but the process crashes before the canonical response is durably committed.

In either case, treating a missing result as proof that the tool did not execute can duplicate writes, deletes, deployments, payments, or remote requests. The previous startup repair path could converge a dangling Run to a failed terminal state, but it could not safely continue execution.

This PR solves that by adding a durable tool-dispatch boundary, deterministic recovery projection, and safe-boundary continuation.

Module view

flowchart LR
subgraph Entry["Product entry points"]
Desktop["Desktop Safe resume and startup recovery"]
CLI["CLI and TUI /resume"]
end
subgraph Control["Recovery control plane"]
SessionManager["SessionManager<br/>candidate discovery and lifecycle"]
Inspector["ContinuationSafetyInspector<br/>cwd, workspace, tools, background work"]
Planner["RuntimeContinuationPlanner<br/>continue or park"]
Resolver["RecoveryResolver<br/>single decision authority"]
end
subgraph Execution["Execution plane"]
Kernel["RuntimeKernel<br/>revalidation and claim"]
AgentRun["AgentRun<br/>fresh execution identity"]
Runner["RuntimeRunner<br/>continuation-start fact"]
Backend["AiSdkFlow and AiSdkBackend<br/>provider replay"]
ToolRuntime["ToolRuntime<br/>preflight, T1, implementation, T2"]
end
subgraph Persistence["Persistence"]
Bootstrap["openRuntimeEventPersistence"]
SQLite[("runtime.sqlite<br/>canonical RuntimeEvent store")]
Projection["tool_journal_events and tool_operations<br/>rebuildable projections"]
JSONL["Legacy RuntimeEvent JSONL<br/>import and explicit export"]
end
Desktop --> SessionManager
CLI --> SessionManager
SessionManager --> Inspector
SessionManager --> Planner
Inspector --> Planner
SQLite --> Planner
Planner --> Resolver
Resolver --> Planner
Planner -->|"approved continuation"| Kernel
Planner -->|"park with stable reason codes"| Desktop
Kernel --> AgentRun
AgentRun --> Runner
Runner --> Backend
Backend --> ToolRuntime
Runner -->|"continuation-start"| SQLite
ToolRuntime -->|"T1 and T2"| SQLite
SQLite -.->|"synchronous projection"| Projection
Bootstrap --> SQLite
Bootstrap -.->|"unmigrated workspace and flag disabled"| JSONL
JSONL -->|"sticky, idempotent import"| SQLite
Loading

The architectural boundary is clear:

  • RuntimeEvent is the canonical recovery fact source.
  • RecoveryResolver is the only component that interprets tool recovery state.
  • Tool journal and operation tables are query projections and can be rebuilt from RuntimeEvents.
  • Product surfaces consume stable plans and reason codes instead of implementing their own recovery logic.

How the tool boundary closes the crash windows

sequenceDiagram
autonumber
participant Provider as Model Provider
participant Backend as AiSdkBackend
participant Runtime as ToolRuntime
participant Store as SqliteRuntimeStore
participant Tool as Tool Implementation
Provider->>Backend: function_call
Backend->>Runtime: execute tool
Runtime->>Runtime: validate args, availability, loop, permissions, and runtime guards
Runtime->>Store: T1 commitToolPrepared
Note over Store: Insert or verify function_call<br/>append toolDispatch RuntimeEvent<br/>update journal and operation projection
alt T1 fails
Store--xRuntime: rollback
Note right of Runtime: Tool implementation is never called
else T1 commits
Store-->>Runtime: durable dispatch boundary
Runtime->>Tool: execute implementation
Tool-->>Runtime: result or error
Runtime->>Store: T2 commitToolOutcome
Note over Store: Append function_response<br/>append outcome journal<br/>CAS operation state
alt T2 fails
Store--xRuntime: rollback
Note right of Runtime: Result is not returned to the next model step
else T2 commits
Store-->>Runtime: durable outcome
Runtime-->>Backend: tool_result
Backend-->>Provider: next provider step
end
end
Loading

This establishes two important invariants:

  1. If T1 fails, the tool implementation runs zero times.
  2. If T2 fails, the tool result cannot reach the next model step.

The external side effect still occurs between two short database transactions, so SQLite cannot make an arbitrary external operation atomic. Instead, a crash between T1 and T2 becomes an explicit indeterminate state that recovery must park or reconcile later.

How safe continuation works

sequenceDiagram
autonumber
participant Entry as User or Startup Recovery
participant Manager as SessionManager
participant Inspector as Safety Inspector
participant Planner as Continuation Planner
participant Resolver as RecoveryResolver
participant Store as Durable Stores
participant Kernel as RuntimeKernel
participant Provider as Model Provider
Entry->>Manager: resumeLatest
Manager->>Store: find latest failed or cancelled session-inline Run
Manager->>Inspector: inspect cwd, workspace identity, tool catalog, and background work
Inspector-->>Manager: authoritative safety observation
Manager->>Planner: plan source boundary
Planner->>Store: read Run header, RuntimeEvents, and continuation ancestors
Planner->>Resolver: resolve tool recovery facts
Resolver-->>Planner: completed, indeterminate, not dispatched, or corruption
alt Any safety condition fails
Planner-->>Manager: park with stable reason codes
Manager-->>Entry: explain why resume is unavailable
Note over Entry,Provider: Provider is not called
else Boundary is safe
Planner-->>Manager: continuation plan with fresh identities and safety snapshot
Manager->>Kernel: execute continuation
Kernel->>Store: re-read source terminal, high-water, identity, and replay
Kernel->>Inspector: revalidate external safety facts
Kernel->>Store: create target Run with continuationSource claim
Kernel->>Store: commit continuation-start RuntimeEvent
Kernel->>Provider: replay committed history without a duplicate user message
Provider-->>Kernel: continue from the validated boundary
end
Loading

Planning and execution are intentionally separate. A valid plan is not treated as an execution lease. Immediately before execution, the Runtime re-reads durable state and rechecks workspace identity, active operations, tool availability, source high-water, and replay equality.

The continuation also creates new execution identity:

source Session / Invocation / Run / Turn
-> validated RuntimeEvent high-water
-> new Invocation / Run / Turn
-> durable continuation-start
-> provider replay without a duplicate user message

The source ledger is not mutated by the continuation.

Recovery decisions

Durable factsResolver decisionCurrent behavior
call + matching responsecompletedReplay call and response as provider history
call + dispatch + no responseindeterminatePark; never blindly retry
call + no dispatch + new protocol markerdefinitely not dispatchedClassified, but currently still parked until Phase 3 policy exists
call + no dispatch + legacy or unknown protocolindeterminatePark
orphan, duplicate, or identity-conflicting factscorruptionFail closed

This is effectively tool-log replay: the provider receives the completed tool interaction as historical context, while the Runtime avoids replaying the old implementation or side effect.

What this PR introduces

AreaIntroduced capability
Recovery semanticsRecoveryResolver, deterministic tool-operation projection, stable diagnostics and park reasons
ContinuationRuntimeContinuationPlanner, fresh execution identities, durable continuation claim, execution-time revalidation
Tool protocoltoolDispatch RuntimeEvent, protocol marker, deterministic operation ID, canonical argument hash, recovery mode
Transaction boundaryRuntimeCommitSink, T1 commitToolPrepared, T2 commitToolOutcome, CAS updates
Canonical storageSQLite RuntimeEvent store, WAL, synchronous=FULL, foreign keys, schema migrations
CompatibilitySticky JSONL-to-SQLite import, source fingerprints, explicit JSONL export
Product surfacesDesktop Safe resume, startup continuation, CLI and TUI /resume, user-facing park copy
ValidationProcess-crash harnesses, replay tests, continuation tests, SQLite crash tests, and Linux/macOS CI coverage

Current boundaries

The conservative scope is appropriate, but it is important to preserve these limitations:

  • this PR does not re-execute historical tool implementations;
  • Phase 3 reconciliation and restricted verification are not implemented;
  • definitely_not_dispatched is classified but does not yet trigger automatic recovery;
  • general Bash and unknown external side effects remain parked;
  • replay-safe, idempotent, reconcile, and reattach tool contracts are future work;
  • SQLite migration is sticky and does not yet provide automatic pre-migration backup;
  • manual resume and startup auto-resume currently share one feature flag;
  • multi-worker leases and fencing are intentionally out of scope.

Future value of checkpoints and Git snapshots

The next major step that could make this resume system broadly practical is coupling the RuntimeEvent high-water with a workspace checkpoint, especially a Git-backed snapshot.

That would let recovery validate or restore both sides of the continuation boundary:

  1. the model and tool interaction history; and
  2. the exact agent-visible workspace state associated with that history.

A durable boundary containing:

RuntimeEvent high-water
+ workspace checkpoint reference
+ Git commit or tree snapshot
+ workspace identity

would allow Maka to resume after a crash without assuming that the current filesystem still matches the interrupted Run. It would also make planning and execution revalidation stronger because the continuation could prove that conversational state and workspace state refer to the same execution boundary.

Overall assessment

This is an excellent foundation and a genuinely interesting resume design.

The durable tool-call protocol makes safe history replay possible without conflating replay with side-effect re-execution. The fail-closed behavior is correct, the new execution identity keeps lineage understandable, and the single canonical recovery authority avoids parallel state machines.

With checkpoint and Git snapshot integration, this design could evolve from safe conversational continuation into a practical end-to-end runtime resume system.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

Conflict-resolution verification update

The branch was reconciled with current main through local merge commits 81edf2ba (Phase 1) and f0653e02 (Phase 2), then pushed without including local dogfooding files.

Additional compatibility fixes made during the merge:

  • aligned SQLite canonical storage with the current terminal durability barrier;
  • taught the exact-shape RuntimeEvent decoder about toolDispatch, runtimeProtocol, and operationId;
  • kept legacy JSONL import independent of a surviving run.json header;
  • preserved continuation invocation identity and session-inline projection semantics;
  • retained current steering, strict-recovery, child-agent, and Desktop IPC contracts.

Verification after reconciliation:

  • npm run build:test — passed across all workspaces;
  • npm run typecheck — passed across all workspaces;
  • Runtime resume/continuation/runner/tool-boundary focused suites — 62 passed;
  • SQLite runtime store — 10 passed;
  • legacy JSONL-to-SQLite transfer — 5 passed;
  • real-process Runtime resume crash harness — passed;
  • Desktop interrupted-banner routing — 4 passed;
  • focused SessionManager continuation/canonical/UI projection cases — 3 passed.

Windows-only file fsync/signal harness limitations remain documented; Linux/macOS are the supported crash-durability targets.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

CI follow-up is complete on head \

@zhiiw
zhiiw marked this pull request as ready for review July 19, 2026 10:40
@Astro-Han
Astro-Han merged commit b5e015a into apache:mainJul 19, 2026
5 checks passed
@zhiiw
zhiiw deleted the codex/runtime-resume-phase2 branch July 19, 2026 11:03
jackwener added a commit that referenced this pull request Jul 19, 2026
…nip zero-hint, resume hook, QR fixture (#1241)
* feat(core): add settings-bots-onboarding visual-smoke scenario + botOnboardingProvider
* feat(desktop): hold-in-waiting bot onboarding adapter + fixture state for QR capture
* feat(desktop): auto-open bot scan-login modal under settings-bots-onboarding fixture
* chore(scripts): register settings-bots-onboarding in capture + audit scenario lists
* fix(desktop): wait for QR paint before auto-capturing bot-onboarding fixture
* docs(notes): frontend architecture map — measured baseline + 8 staged rounds (2026-07-19)
Re-ran the census (wc over non-test TS/TSX for desktop main/preload/renderer +
packages/ui; CSS totals; top-10 hotspot table incl. visual-smoke-fixture.ts marked
OUT-OF-SCOPE other-sessions' territory). Staged R1 knip de-rot, R2 app-shell
resume-cluster extraction, R3 visual-smoke split (BLOCKED on concurrent fixture branch),
R4-R6 main.ts extractions (require maintainer-approved contract re-pins — registerIpc /
startup / tool-assembly / modelSupportsVision direct-pinned), R7 provider-connection-detail
decomposition (needs its own blade plan), R8 CSS raw-hex residue (verified clean on this
tip — #1085 already converted it).
* chore(knip): de-rot config — clear all 13 stale hints (R1)
knip reported 13 configuration hints (7 desktop + 6 ui), all redundant/stale config,
zero code impact:
- Drop 'overlayscrollbars' from desktop ignoreDependencies — now redundant (knip no
longer flags it; the package is owned by packages/ui and only referenced there). The
overlay-scrollbars contract forbids declaring it as a *desktop dependency*, not as an
ignore — removing a redundant ignore is compatible and coverage is unchanged.
- Delete the dead 'src/renderer/**/*.test.ts' entry glob (no matches — renderer has zero
test files; all 338 desktop tests live under src/main/__tests__/*.test.ts).
- Drop 11 redundant explicit entries knip already auto-detects: desktop
preload.ts / renderer/main.tsx / playwright.config.ts / .storybook/main.ts / dev.mjs
(vite / playwright / storybook / npm-scripts plugins), and all 6 ui entries
(index.ts, icons.tsx, artifact-preview-registry.ts, assistant-stream.ts, maka-uri.ts,
smooth-stream.ts) resolved from packages/ui package.json#exports.
Non-redundant entries kept verbatim (main.ts, overlay/*, main test glob, e2e specs,
storybook preview, stories, browser-observe-act-smoke.mjs, both ui test globs). No broad
ignores added — coverage is not weakened. knip --workspace apps/desktop and packages/ui
now both exit 0 AND report zero config hints (was: exit 0 with 13 hints).
* refactor(renderer): extract app-shell resume cluster into use-shell-resume (R2)
The #1223 safe-boundary resume cluster moves out of app-shell.tsx into a new
use-shell-resume.ts hook, following the use-shell-connections / use-shell-chat-model
house style (options object; state + handler returned; stable identities preserved).
Pure move, zero behavior change: the two useState declarations (resumePendingSessionId,
resumeParkDescriptionBySession) and the resumeInterruptedSession handler move verbatim;
activeId/toastApi/shellCopy/uiLocale are injected as options. app-shell keeps the banner
JSX wiring (safeResumeAction=) and the sendWithAttachments guard. resumeParkToastCopy is
no longer imported into app-shell (it now lives in the hook). app-shell.tsx 1686 -> 1654
(-32 lines).
Contract re-pin (never deletes coverage): runtime-resume-routing-contract.test.ts read
the handler-shape assertions (resumeInterruptedSession / resumeLatest(sessionId) /
resumeParkToastCopy / no rejectionReasons.join) from app-shell.tsx single-file; they now
read use-shell-resume.ts where that logic lives, while the app-shell-specific assertions
(sendWithAttachments '/resume' guard, safeResumeAction= wiring) still read app-shell.tsx.
use-shell-resume.ts added to renderer-shell-source-helpers sourcePaths (Round B/E
precedent) so combined-source contracts see it.
Gates: desktop 2740/2740, ui 196/196, typecheck 0, check-dead-css clean, knip
desktop+ui exit 0, AUDIT_PORT_BASE=23900 alignment auditor exit 0 (all 10 fixtures
clean, real renderer). CDP turn-narrative branch-vs-baseline (real Electron, light+dark
1280) byte-identical (sha256 match) — proven render no-op.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@zhiiw@likun666661@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): add durable safe-boundary resume phases 0-2 - #1223

Merged
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2
Jul 19, 2026
Merged

feat(runtime): add durable safe-boundary resume phases 0-2#1223
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2

Conversation

@zhiiw

Copy link
Copy Markdown
Contributor

Summary

Implements the Runtime Resume foundation from Phase 0 through Phase 2, with RuntimeEvent as the single canonical recovery fact source.

  • Phase 0: fail-closed RuntimeEvent replay projection, unmatched tool-call parking, terminal repair, and real process-crash coverage.
  • Phase 1: safe-boundary continuation into fresh Run/Turn/Invocation identities, durable continuation claims, plan/execute separation, and execution-time revalidation.
  • Phase 2: sticky SQLite canonical RuntimeEvent storage, transactional T1/T2 tool boundaries, operation identity/CAS, import/export, and one canonical writer.
  • Phase 2.5 convergence: RecoveryResolver is the sole decision authority; tool journal tables are rebuildable projections, not a parallel truth source.

The Desktop manual recovery entry is attached to the interrupted-turn banner as Safe resume. CLI/TUI retain /resume. Park reasons are translated for users while stable reason codes remain available to diagnostics.

Related to #186.

Safety and rollout

Both capabilities remain disabled by default:

FlagBehavior
MAKA_RUNTIME_SQLITE_CANONICAL=1Triggers a sticky, one-way workspace migration to runtime.sqlite; disabling the flag does not return that workspace to JSONL.
MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1Enables safe-boundary recovery surfaces and Desktop startup auto-resume; these paths may call the provider and consume tokens.

Ambiguous tool outcomes are parked, never blindly retried. Phase 3 reconciliation is intentionally outside this PR.

Dogfooding and regression fixes

Real restart/resume testing found and fixed:

  1. second-generation continuation envelopes dropping source-only replay context;
  2. continuation replay/history reconstruction gaps across ancestor runs;
  3. continuation runs being misclassified as child-agent runs in model, UI, compaction, and repair projections;
  4. canonical continuation-start facts being treated as unsupported chat events;
  5. empty materialized continuation replay reaching the provider.

The final recovery gates also reject duplicate dispatch/response facts and unknown runtime protocol markers as corruption.

Known limitations

  • Automatic pre-migration backup is not implemented yet.
  • Populated SQLite v2-to-v4 upgrade coverage remains to be added.
  • Crash harness support targets Linux/macOS; Windows remains limited support.
  • definitely_not_dispatched is classified but its Phase 3 automated recovery policy is not implemented.
  • Legacy tool-projection rebuilding remains compatibility-only and must not become a second recovery truth source.
  • Manual and startup auto-resume still share one feature flag; they should be split before broader rollout.

Verification

  • npm run typecheck (all workspaces)
  • RecoveryResolver suite: 11 passed
  • empty continuation replay provider guard: passed
  • Desktop interrupted-banner routing and eligibility: 6 passed
  • user-facing recovery copy: 3 passed
  • changed-file Biome lint and git diff --check

@likun666661

likun666661 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Review conclusion

This is a novel and technically strong approach to runtime resume. The PR does not try to revive an old JavaScript stack, provider stream, Promise, or operating-system process. Instead, it reconstructs durable execution facts, proves that a continuation boundary is safe, and then starts a new Run, Invocation, and Turn.

The most important design choice is the distinction between replaying tool history and re-executing tool side effects:

  • completed function_call and function_response pairs are replayed to the provider as history;
  • the previous tool implementation is not executed again;
  • an unmatched or ambiguous tool call is parked rather than retried;
  • contradictory recovery facts fail closed.

This gives Maka a useful resume primitive without claiming exactly-once execution for arbitrary Bash commands, filesystem mutations, or external APIs.

Problem being solved

The existing RuntimeEvent ledger could answer what had been recorded, but it could not reliably answer whether a tool might already have crossed the side-effect boundary.

Two crash windows are particularly dangerous:

  1. The tool implementation starts before the canonical call fact is durably committed.
  2. The tool finishes its side effect, but the process crashes before the canonical response is durably committed.

In either case, treating a missing result as proof that the tool did not execute can duplicate writes, deletes, deployments, payments, or remote requests. The previous startup repair path could converge a dangling Run to a failed terminal state, but it could not safely continue execution.

This PR solves that by adding a durable tool-dispatch boundary, deterministic recovery projection, and safe-boundary continuation.

Module view

flowchart LR
subgraph Entry["Product entry points"]
Desktop["Desktop Safe resume and startup recovery"]
CLI["CLI and TUI /resume"]
end
subgraph Control["Recovery control plane"]
SessionManager["SessionManager<br/>candidate discovery and lifecycle"]
Inspector["ContinuationSafetyInspector<br/>cwd, workspace, tools, background work"]
Planner["RuntimeContinuationPlanner<br/>continue or park"]
Resolver["RecoveryResolver<br/>single decision authority"]
end
subgraph Execution["Execution plane"]
Kernel["RuntimeKernel<br/>revalidation and claim"]
AgentRun["AgentRun<br/>fresh execution identity"]
Runner["RuntimeRunner<br/>continuation-start fact"]
Backend["AiSdkFlow and AiSdkBackend<br/>provider replay"]
ToolRuntime["ToolRuntime<br/>preflight, T1, implementation, T2"]
end
subgraph Persistence["Persistence"]
Bootstrap["openRuntimeEventPersistence"]
SQLite[("runtime.sqlite<br/>canonical RuntimeEvent store")]
Projection["tool_journal_events and tool_operations<br/>rebuildable projections"]
JSONL["Legacy RuntimeEvent JSONL<br/>import and explicit export"]
end
Desktop --> SessionManager
CLI --> SessionManager
SessionManager --> Inspector
SessionManager --> Planner
Inspector --> Planner
SQLite --> Planner
Planner --> Resolver
Resolver --> Planner
Planner -->|"approved continuation"| Kernel
Planner -->|"park with stable reason codes"| Desktop
Kernel --> AgentRun
AgentRun --> Runner
Runner --> Backend
Backend --> ToolRuntime
Runner -->|"continuation-start"| SQLite
ToolRuntime -->|"T1 and T2"| SQLite
SQLite -.->|"synchronous projection"| Projection
Bootstrap --> SQLite
Bootstrap -.->|"unmigrated workspace and flag disabled"| JSONL
JSONL -->|"sticky, idempotent import"| SQLite
Loading

The architectural boundary is clear:

  • RuntimeEvent is the canonical recovery fact source.
  • RecoveryResolver is the only component that interprets tool recovery state.
  • Tool journal and operation tables are query projections and can be rebuilt from RuntimeEvents.
  • Product surfaces consume stable plans and reason codes instead of implementing their own recovery logic.

How the tool boundary closes the crash windows

sequenceDiagram
autonumber
participant Provider as Model Provider
participant Backend as AiSdkBackend
participant Runtime as ToolRuntime
participant Store as SqliteRuntimeStore
participant Tool as Tool Implementation
Provider->>Backend: function_call
Backend->>Runtime: execute tool
Runtime->>Runtime: validate args, availability, loop, permissions, and runtime guards
Runtime->>Store: T1 commitToolPrepared
Note over Store: Insert or verify function_call<br/>append toolDispatch RuntimeEvent<br/>update journal and operation projection
alt T1 fails
Store--xRuntime: rollback
Note right of Runtime: Tool implementation is never called
else T1 commits
Store-->>Runtime: durable dispatch boundary
Runtime->>Tool: execute implementation
Tool-->>Runtime: result or error
Runtime->>Store: T2 commitToolOutcome
Note over Store: Append function_response<br/>append outcome journal<br/>CAS operation state
alt T2 fails
Store--xRuntime: rollback
Note right of Runtime: Result is not returned to the next model step
else T2 commits
Store-->>Runtime: durable outcome
Runtime-->>Backend: tool_result
Backend-->>Provider: next provider step
end
end
Loading

This establishes two important invariants:

  1. If T1 fails, the tool implementation runs zero times.
  2. If T2 fails, the tool result cannot reach the next model step.

The external side effect still occurs between two short database transactions, so SQLite cannot make an arbitrary external operation atomic. Instead, a crash between T1 and T2 becomes an explicit indeterminate state that recovery must park or reconcile later.

How safe continuation works

sequenceDiagram
autonumber
participant Entry as User or Startup Recovery
participant Manager as SessionManager
participant Inspector as Safety Inspector
participant Planner as Continuation Planner
participant Resolver as RecoveryResolver
participant Store as Durable Stores
participant Kernel as RuntimeKernel
participant Provider as Model Provider
Entry->>Manager: resumeLatest
Manager->>Store: find latest failed or cancelled session-inline Run
Manager->>Inspector: inspect cwd, workspace identity, tool catalog, and background work
Inspector-->>Manager: authoritative safety observation
Manager->>Planner: plan source boundary
Planner->>Store: read Run header, RuntimeEvents, and continuation ancestors
Planner->>Resolver: resolve tool recovery facts
Resolver-->>Planner: completed, indeterminate, not dispatched, or corruption
alt Any safety condition fails
Planner-->>Manager: park with stable reason codes
Manager-->>Entry: explain why resume is unavailable
Note over Entry,Provider: Provider is not called
else Boundary is safe
Planner-->>Manager: continuation plan with fresh identities and safety snapshot
Manager->>Kernel: execute continuation
Kernel->>Store: re-read source terminal, high-water, identity, and replay
Kernel->>Inspector: revalidate external safety facts
Kernel->>Store: create target Run with continuationSource claim
Kernel->>Store: commit continuation-start RuntimeEvent
Kernel->>Provider: replay committed history without a duplicate user message
Provider-->>Kernel: continue from the validated boundary
end
Loading

Planning and execution are intentionally separate. A valid plan is not treated as an execution lease. Immediately before execution, the Runtime re-reads durable state and rechecks workspace identity, active operations, tool availability, source high-water, and replay equality.

The continuation also creates new execution identity:

source Session / Invocation / Run / Turn
-> validated RuntimeEvent high-water
-> new Invocation / Run / Turn
-> durable continuation-start
-> provider replay without a duplicate user message

The source ledger is not mutated by the continuation.

Recovery decisions

Durable factsResolver decisionCurrent behavior
call + matching responsecompletedReplay call and response as provider history
call + dispatch + no responseindeterminatePark; never blindly retry
call + no dispatch + new protocol markerdefinitely not dispatchedClassified, but currently still parked until Phase 3 policy exists
call + no dispatch + legacy or unknown protocolindeterminatePark
orphan, duplicate, or identity-conflicting factscorruptionFail closed

This is effectively tool-log replay: the provider receives the completed tool interaction as historical context, while the Runtime avoids replaying the old implementation or side effect.

What this PR introduces

AreaIntroduced capability
Recovery semanticsRecoveryResolver, deterministic tool-operation projection, stable diagnostics and park reasons
ContinuationRuntimeContinuationPlanner, fresh execution identities, durable continuation claim, execution-time revalidation
Tool protocoltoolDispatch RuntimeEvent, protocol marker, deterministic operation ID, canonical argument hash, recovery mode
Transaction boundaryRuntimeCommitSink, T1 commitToolPrepared, T2 commitToolOutcome, CAS updates
Canonical storageSQLite RuntimeEvent store, WAL, synchronous=FULL, foreign keys, schema migrations
CompatibilitySticky JSONL-to-SQLite import, source fingerprints, explicit JSONL export
Product surfacesDesktop Safe resume, startup continuation, CLI and TUI /resume, user-facing park copy
ValidationProcess-crash harnesses, replay tests, continuation tests, SQLite crash tests, and Linux/macOS CI coverage

Current boundaries

The conservative scope is appropriate, but it is important to preserve these limitations:

  • this PR does not re-execute historical tool implementations;
  • Phase 3 reconciliation and restricted verification are not implemented;
  • definitely_not_dispatched is classified but does not yet trigger automatic recovery;
  • general Bash and unknown external side effects remain parked;
  • replay-safe, idempotent, reconcile, and reattach tool contracts are future work;
  • SQLite migration is sticky and does not yet provide automatic pre-migration backup;
  • manual resume and startup auto-resume currently share one feature flag;
  • multi-worker leases and fencing are intentionally out of scope.

Future value of checkpoints and Git snapshots

The next major step that could make this resume system broadly practical is coupling the RuntimeEvent high-water with a workspace checkpoint, especially a Git-backed snapshot.

That would let recovery validate or restore both sides of the continuation boundary:

  1. the model and tool interaction history; and
  2. the exact agent-visible workspace state associated with that history.

A durable boundary containing:

RuntimeEvent high-water
+ workspace checkpoint reference
+ Git commit or tree snapshot
+ workspace identity

would allow Maka to resume after a crash without assuming that the current filesystem still matches the interrupted Run. It would also make planning and execution revalidation stronger because the continuation could prove that conversational state and workspace state refer to the same execution boundary.

Overall assessment

This is an excellent foundation and a genuinely interesting resume design.

The durable tool-call protocol makes safe history replay possible without conflating replay with side-effect re-execution. The fail-closed behavior is correct, the new execution identity keeps lineage understandable, and the single canonical recovery authority avoids parallel state machines.

With checkpoint and Git snapshot integration, this design could evolve from safe conversational continuation into a practical end-to-end runtime resume system.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

Conflict-resolution verification update

The branch was reconciled with current main through local merge commits 81edf2ba (Phase 1) and f0653e02 (Phase 2), then pushed without including local dogfooding files.

Additional compatibility fixes made during the merge:

  • aligned SQLite canonical storage with the current terminal durability barrier;
  • taught the exact-shape RuntimeEvent decoder about toolDispatch, runtimeProtocol, and operationId;
  • kept legacy JSONL import independent of a surviving run.json header;
  • preserved continuation invocation identity and session-inline projection semantics;
  • retained current steering, strict-recovery, child-agent, and Desktop IPC contracts.

Verification after reconciliation:

  • npm run build:test — passed across all workspaces;
  • npm run typecheck — passed across all workspaces;
  • Runtime resume/continuation/runner/tool-boundary focused suites — 62 passed;
  • SQLite runtime store — 10 passed;
  • legacy JSONL-to-SQLite transfer — 5 passed;
  • real-process Runtime resume crash harness — passed;
  • Desktop interrupted-banner routing — 4 passed;
  • focused SessionManager continuation/canonical/UI projection cases — 3 passed.

Windows-only file fsync/signal harness limitations remain documented; Linux/macOS are the supported crash-durability targets.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

CI follow-up is complete on head \

@zhiiw
zhiiw marked this pull request as ready for review July 19, 2026 10:40
@Astro-Han
Astro-Han merged commit b5e015a into apache:mainJul 19, 2026
5 checks passed
@zhiiw
zhiiw deleted the codex/runtime-resume-phase2 branch July 19, 2026 11:03
jackwener added a commit that referenced this pull request Jul 19, 2026
…nip zero-hint, resume hook, QR fixture (#1241)
* feat(core): add settings-bots-onboarding visual-smoke scenario + botOnboardingProvider
* feat(desktop): hold-in-waiting bot onboarding adapter + fixture state for QR capture
* feat(desktop): auto-open bot scan-login modal under settings-bots-onboarding fixture
* chore(scripts): register settings-bots-onboarding in capture + audit scenario lists
* fix(desktop): wait for QR paint before auto-capturing bot-onboarding fixture
* docs(notes): frontend architecture map — measured baseline + 8 staged rounds (2026-07-19)
Re-ran the census (wc over non-test TS/TSX for desktop main/preload/renderer +
packages/ui; CSS totals; top-10 hotspot table incl. visual-smoke-fixture.ts marked
OUT-OF-SCOPE other-sessions' territory). Staged R1 knip de-rot, R2 app-shell
resume-cluster extraction, R3 visual-smoke split (BLOCKED on concurrent fixture branch),
R4-R6 main.ts extractions (require maintainer-approved contract re-pins — registerIpc /
startup / tool-assembly / modelSupportsVision direct-pinned), R7 provider-connection-detail
decomposition (needs its own blade plan), R8 CSS raw-hex residue (verified clean on this
tip — #1085 already converted it).
* chore(knip): de-rot config — clear all 13 stale hints (R1)
knip reported 13 configuration hints (7 desktop + 6 ui), all redundant/stale config,
zero code impact:
- Drop 'overlayscrollbars' from desktop ignoreDependencies — now redundant (knip no
longer flags it; the package is owned by packages/ui and only referenced there). The
overlay-scrollbars contract forbids declaring it as a *desktop dependency*, not as an
ignore — removing a redundant ignore is compatible and coverage is unchanged.
- Delete the dead 'src/renderer/**/*.test.ts' entry glob (no matches — renderer has zero
test files; all 338 desktop tests live under src/main/__tests__/*.test.ts).
- Drop 11 redundant explicit entries knip already auto-detects: desktop
preload.ts / renderer/main.tsx / playwright.config.ts / .storybook/main.ts / dev.mjs
(vite / playwright / storybook / npm-scripts plugins), and all 6 ui entries
(index.ts, icons.tsx, artifact-preview-registry.ts, assistant-stream.ts, maka-uri.ts,
smooth-stream.ts) resolved from packages/ui package.json#exports.
Non-redundant entries kept verbatim (main.ts, overlay/*, main test glob, e2e specs,
storybook preview, stories, browser-observe-act-smoke.mjs, both ui test globs). No broad
ignores added — coverage is not weakened. knip --workspace apps/desktop and packages/ui
now both exit 0 AND report zero config hints (was: exit 0 with 13 hints).
* refactor(renderer): extract app-shell resume cluster into use-shell-resume (R2)
The #1223 safe-boundary resume cluster moves out of app-shell.tsx into a new
use-shell-resume.ts hook, following the use-shell-connections / use-shell-chat-model
house style (options object; state + handler returned; stable identities preserved).
Pure move, zero behavior change: the two useState declarations (resumePendingSessionId,
resumeParkDescriptionBySession) and the resumeInterruptedSession handler move verbatim;
activeId/toastApi/shellCopy/uiLocale are injected as options. app-shell keeps the banner
JSX wiring (safeResumeAction=) and the sendWithAttachments guard. resumeParkToastCopy is
no longer imported into app-shell (it now lives in the hook). app-shell.tsx 1686 -> 1654
(-32 lines).
Contract re-pin (never deletes coverage): runtime-resume-routing-contract.test.ts read
the handler-shape assertions (resumeInterruptedSession / resumeLatest(sessionId) /
resumeParkToastCopy / no rejectionReasons.join) from app-shell.tsx single-file; they now
read use-shell-resume.ts where that logic lives, while the app-shell-specific assertions
(sendWithAttachments '/resume' guard, safeResumeAction= wiring) still read app-shell.tsx.
use-shell-resume.ts added to renderer-shell-source-helpers sourcePaths (Round B/E
precedent) so combined-source contracts see it.
Gates: desktop 2740/2740, ui 196/196, typecheck 0, check-dead-css clean, knip
desktop+ui exit 0, AUDIT_PORT_BASE=23900 alignment auditor exit 0 (all 10 fixtures
clean, real renderer). CDP turn-narrative branch-vs-baseline (real Electron, light+dark
1280) byte-identical (sha256 match) — proven render no-op.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@zhiiw@likun666661@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat(runtime): add durable safe-boundary resume phases 0-2 - #1223

Merged
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2
Jul 19, 2026
Merged

feat(runtime): add durable safe-boundary resume phases 0-2#1223
Astro-Han merged 23 commits into
apache:mainfrom
zhiiw:codex/runtime-resume-phase2

Conversation

@zhiiw

Copy link
Copy Markdown
Contributor

Summary

Implements the Runtime Resume foundation from Phase 0 through Phase 2, with RuntimeEvent as the single canonical recovery fact source.

  • Phase 0: fail-closed RuntimeEvent replay projection, unmatched tool-call parking, terminal repair, and real process-crash coverage.
  • Phase 1: safe-boundary continuation into fresh Run/Turn/Invocation identities, durable continuation claims, plan/execute separation, and execution-time revalidation.
  • Phase 2: sticky SQLite canonical RuntimeEvent storage, transactional T1/T2 tool boundaries, operation identity/CAS, import/export, and one canonical writer.
  • Phase 2.5 convergence: RecoveryResolver is the sole decision authority; tool journal tables are rebuildable projections, not a parallel truth source.

The Desktop manual recovery entry is attached to the interrupted-turn banner as Safe resume. CLI/TUI retain /resume. Park reasons are translated for users while stable reason codes remain available to diagnostics.

Related to #186.

Safety and rollout

Both capabilities remain disabled by default:

FlagBehavior
MAKA_RUNTIME_SQLITE_CANONICAL=1Triggers a sticky, one-way workspace migration to runtime.sqlite; disabling the flag does not return that workspace to JSONL.
MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1Enables safe-boundary recovery surfaces and Desktop startup auto-resume; these paths may call the provider and consume tokens.

Ambiguous tool outcomes are parked, never blindly retried. Phase 3 reconciliation is intentionally outside this PR.

Dogfooding and regression fixes

Real restart/resume testing found and fixed:

  1. second-generation continuation envelopes dropping source-only replay context;
  2. continuation replay/history reconstruction gaps across ancestor runs;
  3. continuation runs being misclassified as child-agent runs in model, UI, compaction, and repair projections;
  4. canonical continuation-start facts being treated as unsupported chat events;
  5. empty materialized continuation replay reaching the provider.

The final recovery gates also reject duplicate dispatch/response facts and unknown runtime protocol markers as corruption.

Known limitations

  • Automatic pre-migration backup is not implemented yet.
  • Populated SQLite v2-to-v4 upgrade coverage remains to be added.
  • Crash harness support targets Linux/macOS; Windows remains limited support.
  • definitely_not_dispatched is classified but its Phase 3 automated recovery policy is not implemented.
  • Legacy tool-projection rebuilding remains compatibility-only and must not become a second recovery truth source.
  • Manual and startup auto-resume still share one feature flag; they should be split before broader rollout.

Verification

  • npm run typecheck (all workspaces)
  • RecoveryResolver suite: 11 passed
  • empty continuation replay provider guard: passed
  • Desktop interrupted-banner routing and eligibility: 6 passed
  • user-facing recovery copy: 3 passed
  • changed-file Biome lint and git diff --check

@likun666661

likun666661 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Review conclusion

This is a novel and technically strong approach to runtime resume. The PR does not try to revive an old JavaScript stack, provider stream, Promise, or operating-system process. Instead, it reconstructs durable execution facts, proves that a continuation boundary is safe, and then starts a new Run, Invocation, and Turn.

The most important design choice is the distinction between replaying tool history and re-executing tool side effects:

  • completed function_call and function_response pairs are replayed to the provider as history;
  • the previous tool implementation is not executed again;
  • an unmatched or ambiguous tool call is parked rather than retried;
  • contradictory recovery facts fail closed.

This gives Maka a useful resume primitive without claiming exactly-once execution for arbitrary Bash commands, filesystem mutations, or external APIs.

Problem being solved

The existing RuntimeEvent ledger could answer what had been recorded, but it could not reliably answer whether a tool might already have crossed the side-effect boundary.

Two crash windows are particularly dangerous:

  1. The tool implementation starts before the canonical call fact is durably committed.
  2. The tool finishes its side effect, but the process crashes before the canonical response is durably committed.

In either case, treating a missing result as proof that the tool did not execute can duplicate writes, deletes, deployments, payments, or remote requests. The previous startup repair path could converge a dangling Run to a failed terminal state, but it could not safely continue execution.

This PR solves that by adding a durable tool-dispatch boundary, deterministic recovery projection, and safe-boundary continuation.

Module view

flowchart LR
subgraph Entry["Product entry points"]
Desktop["Desktop Safe resume and startup recovery"]
CLI["CLI and TUI /resume"]
end
subgraph Control["Recovery control plane"]
SessionManager["SessionManager<br/>candidate discovery and lifecycle"]
Inspector["ContinuationSafetyInspector<br/>cwd, workspace, tools, background work"]
Planner["RuntimeContinuationPlanner<br/>continue or park"]
Resolver["RecoveryResolver<br/>single decision authority"]
end
subgraph Execution["Execution plane"]
Kernel["RuntimeKernel<br/>revalidation and claim"]
AgentRun["AgentRun<br/>fresh execution identity"]
Runner["RuntimeRunner<br/>continuation-start fact"]
Backend["AiSdkFlow and AiSdkBackend<br/>provider replay"]
ToolRuntime["ToolRuntime<br/>preflight, T1, implementation, T2"]
end
subgraph Persistence["Persistence"]
Bootstrap["openRuntimeEventPersistence"]
SQLite[("runtime.sqlite<br/>canonical RuntimeEvent store")]
Projection["tool_journal_events and tool_operations<br/>rebuildable projections"]
JSONL["Legacy RuntimeEvent JSONL<br/>import and explicit export"]
end
Desktop --> SessionManager
CLI --> SessionManager
SessionManager --> Inspector
SessionManager --> Planner
Inspector --> Planner
SQLite --> Planner
Planner --> Resolver
Resolver --> Planner
Planner -->|"approved continuation"| Kernel
Planner -->|"park with stable reason codes"| Desktop
Kernel --> AgentRun
AgentRun --> Runner
Runner --> Backend
Backend --> ToolRuntime
Runner -->|"continuation-start"| SQLite
ToolRuntime -->|"T1 and T2"| SQLite
SQLite -.->|"synchronous projection"| Projection
Bootstrap --> SQLite
Bootstrap -.->|"unmigrated workspace and flag disabled"| JSONL
JSONL -->|"sticky, idempotent import"| SQLite
Loading

The architectural boundary is clear:

  • RuntimeEvent is the canonical recovery fact source.
  • RecoveryResolver is the only component that interprets tool recovery state.
  • Tool journal and operation tables are query projections and can be rebuilt from RuntimeEvents.
  • Product surfaces consume stable plans and reason codes instead of implementing their own recovery logic.

How the tool boundary closes the crash windows

sequenceDiagram
autonumber
participant Provider as Model Provider
participant Backend as AiSdkBackend
participant Runtime as ToolRuntime
participant Store as SqliteRuntimeStore
participant Tool as Tool Implementation
Provider->>Backend: function_call
Backend->>Runtime: execute tool
Runtime->>Runtime: validate args, availability, loop, permissions, and runtime guards
Runtime->>Store: T1 commitToolPrepared
Note over Store: Insert or verify function_call<br/>append toolDispatch RuntimeEvent<br/>update journal and operation projection
alt T1 fails
Store--xRuntime: rollback
Note right of Runtime: Tool implementation is never called
else T1 commits
Store-->>Runtime: durable dispatch boundary
Runtime->>Tool: execute implementation
Tool-->>Runtime: result or error
Runtime->>Store: T2 commitToolOutcome
Note over Store: Append function_response<br/>append outcome journal<br/>CAS operation state
alt T2 fails
Store--xRuntime: rollback
Note right of Runtime: Result is not returned to the next model step
else T2 commits
Store-->>Runtime: durable outcome
Runtime-->>Backend: tool_result
Backend-->>Provider: next provider step
end
end
Loading

This establishes two important invariants:

  1. If T1 fails, the tool implementation runs zero times.
  2. If T2 fails, the tool result cannot reach the next model step.

The external side effect still occurs between two short database transactions, so SQLite cannot make an arbitrary external operation atomic. Instead, a crash between T1 and T2 becomes an explicit indeterminate state that recovery must park or reconcile later.

How safe continuation works

sequenceDiagram
autonumber
participant Entry as User or Startup Recovery
participant Manager as SessionManager
participant Inspector as Safety Inspector
participant Planner as Continuation Planner
participant Resolver as RecoveryResolver
participant Store as Durable Stores
participant Kernel as RuntimeKernel
participant Provider as Model Provider
Entry->>Manager: resumeLatest
Manager->>Store: find latest failed or cancelled session-inline Run
Manager->>Inspector: inspect cwd, workspace identity, tool catalog, and background work
Inspector-->>Manager: authoritative safety observation
Manager->>Planner: plan source boundary
Planner->>Store: read Run header, RuntimeEvents, and continuation ancestors
Planner->>Resolver: resolve tool recovery facts
Resolver-->>Planner: completed, indeterminate, not dispatched, or corruption
alt Any safety condition fails
Planner-->>Manager: park with stable reason codes
Manager-->>Entry: explain why resume is unavailable
Note over Entry,Provider: Provider is not called
else Boundary is safe
Planner-->>Manager: continuation plan with fresh identities and safety snapshot
Manager->>Kernel: execute continuation
Kernel->>Store: re-read source terminal, high-water, identity, and replay
Kernel->>Inspector: revalidate external safety facts
Kernel->>Store: create target Run with continuationSource claim
Kernel->>Store: commit continuation-start RuntimeEvent
Kernel->>Provider: replay committed history without a duplicate user message
Provider-->>Kernel: continue from the validated boundary
end
Loading

Planning and execution are intentionally separate. A valid plan is not treated as an execution lease. Immediately before execution, the Runtime re-reads durable state and rechecks workspace identity, active operations, tool availability, source high-water, and replay equality.

The continuation also creates new execution identity:

source Session / Invocation / Run / Turn
-> validated RuntimeEvent high-water
-> new Invocation / Run / Turn
-> durable continuation-start
-> provider replay without a duplicate user message

The source ledger is not mutated by the continuation.

Recovery decisions

Durable factsResolver decisionCurrent behavior
call + matching responsecompletedReplay call and response as provider history
call + dispatch + no responseindeterminatePark; never blindly retry
call + no dispatch + new protocol markerdefinitely not dispatchedClassified, but currently still parked until Phase 3 policy exists
call + no dispatch + legacy or unknown protocolindeterminatePark
orphan, duplicate, or identity-conflicting factscorruptionFail closed

This is effectively tool-log replay: the provider receives the completed tool interaction as historical context, while the Runtime avoids replaying the old implementation or side effect.

What this PR introduces

AreaIntroduced capability
Recovery semanticsRecoveryResolver, deterministic tool-operation projection, stable diagnostics and park reasons
ContinuationRuntimeContinuationPlanner, fresh execution identities, durable continuation claim, execution-time revalidation
Tool protocoltoolDispatch RuntimeEvent, protocol marker, deterministic operation ID, canonical argument hash, recovery mode
Transaction boundaryRuntimeCommitSink, T1 commitToolPrepared, T2 commitToolOutcome, CAS updates
Canonical storageSQLite RuntimeEvent store, WAL, synchronous=FULL, foreign keys, schema migrations
CompatibilitySticky JSONL-to-SQLite import, source fingerprints, explicit JSONL export
Product surfacesDesktop Safe resume, startup continuation, CLI and TUI /resume, user-facing park copy
ValidationProcess-crash harnesses, replay tests, continuation tests, SQLite crash tests, and Linux/macOS CI coverage

Current boundaries

The conservative scope is appropriate, but it is important to preserve these limitations:

  • this PR does not re-execute historical tool implementations;
  • Phase 3 reconciliation and restricted verification are not implemented;
  • definitely_not_dispatched is classified but does not yet trigger automatic recovery;
  • general Bash and unknown external side effects remain parked;
  • replay-safe, idempotent, reconcile, and reattach tool contracts are future work;
  • SQLite migration is sticky and does not yet provide automatic pre-migration backup;
  • manual resume and startup auto-resume currently share one feature flag;
  • multi-worker leases and fencing are intentionally out of scope.

Future value of checkpoints and Git snapshots

The next major step that could make this resume system broadly practical is coupling the RuntimeEvent high-water with a workspace checkpoint, especially a Git-backed snapshot.

That would let recovery validate or restore both sides of the continuation boundary:

  1. the model and tool interaction history; and
  2. the exact agent-visible workspace state associated with that history.

A durable boundary containing:

RuntimeEvent high-water
+ workspace checkpoint reference
+ Git commit or tree snapshot
+ workspace identity

would allow Maka to resume after a crash without assuming that the current filesystem still matches the interrupted Run. It would also make planning and execution revalidation stronger because the continuation could prove that conversational state and workspace state refer to the same execution boundary.

Overall assessment

This is an excellent foundation and a genuinely interesting resume design.

The durable tool-call protocol makes safe history replay possible without conflating replay with side-effect re-execution. The fail-closed behavior is correct, the new execution identity keeps lineage understandable, and the single canonical recovery authority avoids parallel state machines.

With checkpoint and Git snapshot integration, this design could evolve from safe conversational continuation into a practical end-to-end runtime resume system.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

Conflict-resolution verification update

The branch was reconciled with current main through local merge commits 81edf2ba (Phase 1) and f0653e02 (Phase 2), then pushed without including local dogfooding files.

Additional compatibility fixes made during the merge:

  • aligned SQLite canonical storage with the current terminal durability barrier;
  • taught the exact-shape RuntimeEvent decoder about toolDispatch, runtimeProtocol, and operationId;
  • kept legacy JSONL import independent of a surviving run.json header;
  • preserved continuation invocation identity and session-inline projection semantics;
  • retained current steering, strict-recovery, child-agent, and Desktop IPC contracts.

Verification after reconciliation:

  • npm run build:test — passed across all workspaces;
  • npm run typecheck — passed across all workspaces;
  • Runtime resume/continuation/runner/tool-boundary focused suites — 62 passed;
  • SQLite runtime store — 10 passed;
  • legacy JSONL-to-SQLite transfer — 5 passed;
  • real-process Runtime resume crash harness — passed;
  • Desktop interrupted-banner routing — 4 passed;
  • focused SessionManager continuation/canonical/UI projection cases — 3 passed.

Windows-only file fsync/signal harness limitations remain documented; Linux/macOS are the supported crash-durability targets.

@zhiiw

Copy link
Copy Markdown
ContributorAuthor

CI follow-up is complete on head \

@zhiiw
zhiiw marked this pull request as ready for review July 19, 2026 10:40
@Astro-Han
Astro-Han merged commit b5e015a into apache:mainJul 19, 2026
5 checks passed
@zhiiw
zhiiw deleted the codex/runtime-resume-phase2 branch July 19, 2026 11:03
jackwener added a commit that referenced this pull request Jul 19, 2026
…nip zero-hint, resume hook, QR fixture (#1241)
* feat(core): add settings-bots-onboarding visual-smoke scenario + botOnboardingProvider
* feat(desktop): hold-in-waiting bot onboarding adapter + fixture state for QR capture
* feat(desktop): auto-open bot scan-login modal under settings-bots-onboarding fixture
* chore(scripts): register settings-bots-onboarding in capture + audit scenario lists
* fix(desktop): wait for QR paint before auto-capturing bot-onboarding fixture
* docs(notes): frontend architecture map — measured baseline + 8 staged rounds (2026-07-19)
Re-ran the census (wc over non-test TS/TSX for desktop main/preload/renderer +
packages/ui; CSS totals; top-10 hotspot table incl. visual-smoke-fixture.ts marked
OUT-OF-SCOPE other-sessions' territory). Staged R1 knip de-rot, R2 app-shell
resume-cluster extraction, R3 visual-smoke split (BLOCKED on concurrent fixture branch),
R4-R6 main.ts extractions (require maintainer-approved contract re-pins — registerIpc /
startup / tool-assembly / modelSupportsVision direct-pinned), R7 provider-connection-detail
decomposition (needs its own blade plan), R8 CSS raw-hex residue (verified clean on this
tip — #1085 already converted it).
* chore(knip): de-rot config — clear all 13 stale hints (R1)
knip reported 13 configuration hints (7 desktop + 6 ui), all redundant/stale config,
zero code impact:
- Drop 'overlayscrollbars' from desktop ignoreDependencies — now redundant (knip no
longer flags it; the package is owned by packages/ui and only referenced there). The
overlay-scrollbars contract forbids declaring it as a *desktop dependency*, not as an
ignore — removing a redundant ignore is compatible and coverage is unchanged.
- Delete the dead 'src/renderer/**/*.test.ts' entry glob (no matches — renderer has zero
test files; all 338 desktop tests live under src/main/__tests__/*.test.ts).
- Drop 11 redundant explicit entries knip already auto-detects: desktop
preload.ts / renderer/main.tsx / playwright.config.ts / .storybook/main.ts / dev.mjs
(vite / playwright / storybook / npm-scripts plugins), and all 6 ui entries
(index.ts, icons.tsx, artifact-preview-registry.ts, assistant-stream.ts, maka-uri.ts,
smooth-stream.ts) resolved from packages/ui package.json#exports.
Non-redundant entries kept verbatim (main.ts, overlay/*, main test glob, e2e specs,
storybook preview, stories, browser-observe-act-smoke.mjs, both ui test globs). No broad
ignores added — coverage is not weakened. knip --workspace apps/desktop and packages/ui
now both exit 0 AND report zero config hints (was: exit 0 with 13 hints).
* refactor(renderer): extract app-shell resume cluster into use-shell-resume (R2)
The #1223 safe-boundary resume cluster moves out of app-shell.tsx into a new
use-shell-resume.ts hook, following the use-shell-connections / use-shell-chat-model
house style (options object; state + handler returned; stable identities preserved).
Pure move, zero behavior change: the two useState declarations (resumePendingSessionId,
resumeParkDescriptionBySession) and the resumeInterruptedSession handler move verbatim;
activeId/toastApi/shellCopy/uiLocale are injected as options. app-shell keeps the banner
JSX wiring (safeResumeAction=) and the sendWithAttachments guard. resumeParkToastCopy is
no longer imported into app-shell (it now lives in the hook). app-shell.tsx 1686 -> 1654
(-32 lines).
Contract re-pin (never deletes coverage): runtime-resume-routing-contract.test.ts read
the handler-shape assertions (resumeInterruptedSession / resumeLatest(sessionId) /
resumeParkToastCopy / no rejectionReasons.join) from app-shell.tsx single-file; they now
read use-shell-resume.ts where that logic lives, while the app-shell-specific assertions
(sendWithAttachments '/resume' guard, safeResumeAction= wiring) still read app-shell.tsx.
use-shell-resume.ts added to renderer-shell-source-helpers sourcePaths (Round B/E
precedent) so combined-source contracts see it.
Gates: desktop 2740/2740, ui 196/196, typecheck 0, check-dead-css clean, knip
desktop+ui exit 0, AUDIT_PORT_BASE=23900 alignment auditor exit 0 (all 10 fixtures
clean, real renderer). CDP turn-narrative branch-vs-baseline (real Electron, light+dark
1280) byte-identical (sha256 match) — proven render no-op.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@zhiiw@likun666661@Astro-Han