Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 29 additions & 78 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -152,7 +152,7 @@ Start with [ARCHITECTURE.md](./ARCHITECTURE.md). It provides the system map, cod
apps/desktop/ Electron main / preload / React renderer

packages/core/ Pure contracts for Sessions, Events, Permissions, and Connections
packages/storage/ SQLite operational state, legacy importers, and payload stores
packages/storage/ SQLite operational state, configuration, and payload stores
packages/runtime/ AgentRun, model adapters, tools, context, and recovery
packages/headless/ TaskRun, Autonomous Loop, Self-check, eval, and AHE
packages/cli/ TUI and non-interactive CLI
Expand All@@ -168,83 +168,49 @@ Maka stores workspace data under Electron `userData` by default:

```text
<Electron userData>/workspaces/default/
runtime.sqlite
llm-connections.json
credentials.json
settings.json
sessions/
artifacts/
```

Current boundaries that matter:

- Sessionsand connection metadata live in the local filesystem;
- Sessions, messages, execution ledgers, workflows, usage, Automations, Daily Review, and Headless TaskRuns live in `runtime.sqlite`;
- Runtime credentials such as API keys, bot tokens, and proxy passwords currently live in local plaintext `credentials.json`, behind the OS account boundary, with POSIX directory mode `0700` and file mode `0600` enforced;
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless; Electron `safeStorage` only decrypts pre-existing legacy token files once at desktop startup (#1125);
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless. Pre-existing Electron `safeStorage` credential/token files are not imported; affected users must re-authenticate;
- Renderer does not receive plaintext credentials. File writes, Shell, and dangerous tool calls pass through the permission engine;
- Headless real-model evaluation fails closed by default and requires an explicit external isolation boundary.

Read [SECURITY.md](./SECURITY.md) for security reporting and policy, and [docs/README.md](./docs/README.md) for current privacy and sandbox contracts.

## Runtime storage and recovery

RuntimeEvent persistence is always canonical in `runtime.sqlite`. On the first
write, Maka batch-idempotently imports legacy RuntimeEvent JSONL without
rewriting it. Legacy-only workspaces remain available to read-only inspection
until that first write.

Session metadata and Agent Graph control tables now use the same process-local
operational database owner and the same `runtime.sqlite` transaction authority.
The first operational open copies a WAL-consistent `sessions.sqlite` source
into `runtime.sqlite`, validates every source row, and records the source digest
and result in `cutover_journal`. An interrupted copy resumes without partial
rows; a legacy database changed after cutover fails closed. The old database is
retained as migration evidence but is no longer a production writer. Session
transcript bodies remain append-only JSONL.

Core execution state now shares that authority too: AgentRun headers and event
ledgers, event projections, root-turn admissions and source proofs,
Interactions, Host Epoch message receipts, and ShellRun records are canonical
in `runtime.sqlite` across CLI, Desktop, Runtime Host, and Headless. Each legacy
file store is fingerprinted and imported through its own durable
`cutover_journal` entry before the corresponding repository opens. Copy and
validation are one SQLite transaction, retries are idempotent, and a changed
legacy source after cutover fails closed. The legacy files are retained only as
migration evidence; new execution writes do not modify them.

Workflow state is migrating in reviewable slices. Task Ledger events and
projections, Plan events and projections, Deep Research events, and Plan
Reminder records are now canonical in `runtime.sqlite`.
Desktop and Runtime Host production wiring opens these SQLite repositories;
their JSON/JSONL predecessors are read only during a fingerprinted, crash-safe
cutover and are never updated by later mutations.

Usage telemetry and pricing authority now use that same operational database.
Legacy `telemetry.json` and `pricing.json` sources are decoded together and
fingerprinted before their rows and pricing revision are committed atomically.
After cutover, Desktop and Runtime Host write only `runtime.sqlite`; the source
files remain unchanged as migration evidence.

Artifact metadata and lifecycle state now follow the same rule. Payload bytes
remain files, but their records are canonical in `runtime.sqlite` after a
fingerprinted `metadata.jsonl` cutover. Payload publication keeps its durable
staging/link protocol: recovery removes bytes whose metadata transaction did not
commit and preserves committed bytes while cleaning staging residue. Purge
intent recovery likewise completes against the SQLite metadata authority.

Selected-session bundle export now reads that SQLite authority through a
WAL-consistent snapshot, verifies that retained legacy evidence still matches
its completed cutover, and writes only the selected Artifact rows into the
bundle's `runtime.sqlite`. Payload bytes are copied under the Artifact writer
lock, cross-session rows and payloads are excluded, and bundles no longer emit
`artifacts/metadata.jsonl`.

Full operational backup now uses the shared database owner's online SQLite
backup API rather than copying `runtime.sqlite` or its WAL sidecars. A strict
manifest binds the standalone database snapshot to every active session
transcript and canonical Artifact payload by size and SHA-256. Restore verifies
SQLite integrity, foreign keys, supported schema versions, relational identity
sets, transcript decodability, and the exact manifested file tree before
atomically publishing a new state root. Interrupted backup or restore staging
is removed and can be retried without changing either source.
`runtime.sqlite` is the sole operational authority. It owns RuntimeEvents,
session metadata and message history, Agent Graph control, core execution state,
workflow state, usage and pricing, Artifact metadata, Automations, Daily Review,
and Headless TaskRuns. Artifact payload bytes remain regular files under
`artifacts/`; connections, credentials, settings, MCP configuration, skills,
and device identity remain configuration files.

This storage generation does not import earlier File/JSONL authorities. On
upgrade, legacy session titles may still be discoverable through current
metadata, but conversation history that exists only in legacy transcript files
is not copied into `session_messages` and opens as an empty thread. Likewise,
pre-version or `safeStorage`-encrypted credential/token files are not migrated;
users with only those copies must re-authenticate. This data-loss boundary is
intentional for this release and must be considered before upgrading an
existing workspace.

Full operational backup uses the database owner's online SQLite backup API and
copies canonical Artifact payloads under the Artifact writer lock. Its manifest
binds every file by size and SHA-256. Validation checks the standalone SQLite
snapshot's integrity, foreign keys, schema registry and required tables,
decodes canonical session-message and Artifact records, and verifies Artifact
payload sizes against SQLite metadata before restore. Backup and restore use
owner-only file modes, file and directory synchronization, staging, and atomic
publication.

Headless trajectory hydration now consumes a frozen selected-session export
from that SQLite Artifact authority. The cell publishes `trajectory-state`
Expand All@@ -254,21 +220,6 @@ validated snapshot. It does not copy a live WAL or fall back to
`artifacts/metadata.jsonl`. Missing, corrupt, unsupported, or mismatched
evidence fails closed to a summary trajectory instead of mixing authorities.

The remaining storage work is deliberately classified rather than implied
complete:

- Artifact metadata no longer exposes a production JSONL writer;
`artifacts/metadata.jsonl` is accepted only as fingerprinted, read-only
cutover evidence, and can be removed after the migration/cutover matrix is
complete;
- StoredMessage transcript bodies remain append-only JSONL;
- automation, connections, credentials, settings, MCP configuration, skills,
and device identity are configuration state and stay outside this operational
migration;
- Headless TaskRun evaluation ledgers, imported foreign-session caches, Daily
Review archives, and quote-cleanup bookkeeping are separate product/evaluation
domains and are not part of issue #1649.

Runtime continuation remains opt-in:

- `MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1` enables the Desktop interrupted-turn
Expand Down
11 changes: 5 additions & 6 deletions SECURITY.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,9 +97,9 @@ supply their own boundary. See
is the single authority every surface — Desktop, TUI, headless —
reads and writes, under the same OS-account and 0o700/0o600 boundary
as other runtime credentials. Electron safeStorage is not part of
this boundary anymore; Desktop startup imports pre-existing
safeStorage-encrypted token files into the store once and removes
them (#1125).
this boundary anymore. Pre-existing safeStorage-encrypted credential
or token files are not imported; users with only those copies must
re-authenticate.
3. **Renderer process sandbox + preload IPC bridge.** The
renderer cannot reach files, network, or shell directly. Every
IPC handler in `apps/desktop/src/main/main.ts` is the trust
Expand DownExpand Up@@ -152,7 +152,7 @@ are welcome as ordinary issues, not security advisories.
Beyond security, Maka treats the following as user-facing
privacy commitments:

- **Workspace JSONL stays local.** Session messages, tool
- **Workspace state stays local.** Session messages, tool
results, telemetry, settings are stored under
`app.getPath('userData')`. Cloud sync is not shipped.
- **Tool query strings are NEVER logged.** The WebSearch tool's
Expand All@@ -179,8 +179,7 @@ boundary in §2.3 was crossed. Examples:
looser than the 0o700/0o600 boundary.
- A production OAuth path writes or requires a safeStorage-encrypted
token copy again. `credentials.json` is the single documented token
authority; safeStorage exists only inside the one-shot legacy import
(#1125).
authority; there is no safeStorage legacy import fallback.
- A cleartext secret crosses main→renderer IPC under any
circumstance (including error paths, settings preview, IPC
result envelopes, log lines).
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ The Electron desktop app: `main` (Node/Electron main process) + `preload` (conte

Sub-folders: `browser/` (embedded browser view), `oauth/`, `search/` (thread search), `web-search/`, `types/`. The browser IPC handler itself (`browser-ipc-main.ts`) is flat in `src/main/`, not under `browser/`.

`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (credential migration, connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.
`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.

## IPC contract

Expand Down
5 changes: 2 additions & 3 deletions apps/desktop/e2e/fixtures.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -356,9 +356,8 @@ export const test = base.extend<{
);
},
// Stale sessions: boots the e2e-fixture `stale-sessions` fixture — one
// healthy session (zai-live, secret seeded), one unlocked fake session
// (opened active), and one locked legacy session whose connection is
// gone. Exercises the #1038 health-notice authority against real IPC
// healthy session (zai-live, secret seeded) and one locked fake-backend session
// (opened active). Exercises the #1038 health-notice authority against real IPC
// (connection list, hasSecret probe, connectionLocked summaries).
// Readiness = turns on screen: the fake session is open.
staleSessionsWindow: async ({}, use) => {
Expand Down
25 changes: 14 additions & 11 deletions apps/desktop/e2e/session-health-notice.spec.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,14 +3,15 @@
// "will the next send fail?" from the same facts as the send gate
// (connection list, hasSecret probe, connectionLocked on the summary).
//
// The stale-sessions e2e-fixture seeds the exact on-disk states, and
// both stale sessions carry user messages, so storage self-heals them to
// `connectionLocked: true` on first read — the send can neither use
// their connections nor silently rebind, even though a healthy default
// The stale-sessions e2e-fixture seeds the exact on-disk states: one
// locked fake-backend session and one healthy ai-sdk session. The stale
// session carries user messages, so storage self-heals it to
// `connectionLocked: true` on first read — the send can neither use its
// connection nor silently rebind, even though a healthy default
// connection exists. The old "default exists && enabled" proxy hid the
// notice in exactly this state (#1038 case 1); it must now show.
// The silent-rebind counterpart (unlocked empty stale session) is
// covered by the projection and notice unit tests.
// The deleted-connection and legacy-backend notice variants are covered
// by deriveSessionHealthNotice unit tests.

import { test, expect } from './fixtures';

Expand DownExpand Up@@ -39,7 +40,7 @@ async function probeNoticeAlignment(page: import('@playwright/test').Page) {
});
}

test('locked stale sessions show the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
test('a locked stale session shows the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
// Active = stale fake session (locked by its history): notice shows.
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();

Expand All@@ -48,12 +49,14 @@ test('locked stale sessions show the health notice even with a ready default', a
expect(alignment.leftDelta).toBeLessThanOrEqual(1);
expect(alignment.rightDelta).toBeLessThanOrEqual(1);

// Switch to the locked legacy session → its deleted-connection notice.
// A healthy session must not show the notice.
await page.getByRole('button', { name: '展开侧边栏' }).click();
await page.getByText('旧的 Claude 连接会话').first().click();
await expect(page.getByText('连接已删除')).toBeVisible();
await page.getByText('正常会话(Z.ai Live)').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeHidden();

// Click-through lands in Settings · 模型.
// Back on the stale session, click-through lands in Settings · 模型.
await page.getByText('旧的本地模拟会话').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();
await page.getByRole('button', { name: '去模型' }).click();
await expect(page.getByLabel('设置内容')).toBeVisible();
// The connection list itself, not just the settings shell: `add-connection`
Expand Down
27 changes: 12 additions & 15 deletions apps/desktop/src/main/__tests__/automation-persistence-e2e.test.ts
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
/**
* End-to-end: durable cron persistence + cross-session query/management.
*
* Exercises the REAL host wiring (createMainAutomationWiring) against a REAL
* FileAutomationStore on a real temp workspace, simulating an app restart:
* Exercises the real host wiring against the operational SQLite authority,
* simulating an app restart:
*
* session A creates a durable cron ──sync──► <workspace>/automations.json
* │
* (restart: a fresh wiring loads it) ◄──loadAll────────┘
* │
* session A creates a durable cron ──sync──► runtime.sqlite
* (restart: a fresh wiring loads it) ◄──loadAll───────┘
* session B (never saw it) lists / pauses / resumes / deletes it
*
* This is the query-and-persistence loop the reviewer asked for: a persisted
Expand All@@ -18,10 +16,11 @@

import { strict as assert } from 'node:assert';
import { describe, it, before, after } from 'node:test';
import { mkdtemp, rm, readFile } from 'node:fs/promises';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { MakaToolContext, MakaTool } from '@maka/runtime';
import { createAutomationStore } from '@maka/storage';
import { createMainAutomationWiring } from '../automation-wiring.js';

function ctx(sessionId: string): MakaToolContext {
Expand DownExpand Up@@ -60,15 +59,13 @@ function automationTool(wiring: ReturnType<typeof makeWiring>): MakaTool {
}

async function readStore(workspaceRoot: string): Promise<Array<{ id: string; name: string }>> {
try {
const raw = await readFile(join(workspaceRoot, 'automations.json'), 'utf8');
return (JSON.parse(raw) as { automations: Array<{ id: string; name: string }> }).automations;
} catch {
return [];
}
return (await createAutomationStore(workspaceRoot).loadAll()).map(({ id, name }) => ({
id,
name,
}));
}

/** The store sync is fire-and-forget; poll the file until it settles. */
/** Store sync is fire-and-forget; poll the authority until it settles. */
async function waitForStore(
workspaceRoot: string,
predicate: (rows: Array<{ id: string; name: string }>) => boolean,
Expand DownExpand Up@@ -175,7 +172,7 @@ describe('E2E: durable cron persistence + cross-session query/management', () =>
});

describe('E2E: a cron-disabled host (CLI) sharing the workspace never clobbers durable crons', () => {
it('a heartbeat-only wiring neither loads nor overwrites the owner\'s automations.json', async () => {
it('a heartbeat-only wiring neither loads nor overwrites durable Automations', async () => {
const ws = await mkdtemp(join(tmpdir(), 'maka-automation-clobber-'));
try {
// ── owner (cron-enabled, desktop) creates a durable cron ──────────────
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
refactor(storage): make SQLite the sole operational authority by jackwener · Pull Request #1994 · apache/maka · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 29 additions & 78 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -152,7 +152,7 @@ Start with [ARCHITECTURE.md](./ARCHITECTURE.md). It provides the system map, cod
apps/desktop/ Electron main / preload / React renderer

packages/core/ Pure contracts for Sessions, Events, Permissions, and Connections
packages/storage/ SQLite operational state, legacy importers, and payload stores
packages/storage/ SQLite operational state, configuration, and payload stores
packages/runtime/ AgentRun, model adapters, tools, context, and recovery
packages/headless/ TaskRun, Autonomous Loop, Self-check, eval, and AHE
packages/cli/ TUI and non-interactive CLI
Expand All@@ -168,83 +168,49 @@ Maka stores workspace data under Electron `userData` by default:

```text
<Electron userData>/workspaces/default/
runtime.sqlite
llm-connections.json
credentials.json
settings.json
sessions/
artifacts/
```

Current boundaries that matter:

- Sessionsand connection metadata live in the local filesystem;
- Sessions, messages, execution ledgers, workflows, usage, Automations, Daily Review, and Headless TaskRuns live in `runtime.sqlite`;
- Runtime credentials such as API keys, bot tokens, and proxy passwords currently live in local plaintext `credentials.json`, behind the OS account boundary, with POSIX directory mode `0700` and file mode `0600` enforced;
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless; Electron `safeStorage` only decrypts pre-existing legacy token files once at desktop startup (#1125);
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless. Pre-existing Electron `safeStorage` credential/token files are not imported; affected users must re-authenticate;
- Renderer does not receive plaintext credentials. File writes, Shell, and dangerous tool calls pass through the permission engine;
- Headless real-model evaluation fails closed by default and requires an explicit external isolation boundary.

Read [SECURITY.md](./SECURITY.md) for security reporting and policy, and [docs/README.md](./docs/README.md) for current privacy and sandbox contracts.

## Runtime storage and recovery

RuntimeEvent persistence is always canonical in `runtime.sqlite`. On the first
write, Maka batch-idempotently imports legacy RuntimeEvent JSONL without
rewriting it. Legacy-only workspaces remain available to read-only inspection
until that first write.

Session metadata and Agent Graph control tables now use the same process-local
operational database owner and the same `runtime.sqlite` transaction authority.
The first operational open copies a WAL-consistent `sessions.sqlite` source
into `runtime.sqlite`, validates every source row, and records the source digest
and result in `cutover_journal`. An interrupted copy resumes without partial
rows; a legacy database changed after cutover fails closed. The old database is
retained as migration evidence but is no longer a production writer. Session
transcript bodies remain append-only JSONL.

Core execution state now shares that authority too: AgentRun headers and event
ledgers, event projections, root-turn admissions and source proofs,
Interactions, Host Epoch message receipts, and ShellRun records are canonical
in `runtime.sqlite` across CLI, Desktop, Runtime Host, and Headless. Each legacy
file store is fingerprinted and imported through its own durable
`cutover_journal` entry before the corresponding repository opens. Copy and
validation are one SQLite transaction, retries are idempotent, and a changed
legacy source after cutover fails closed. The legacy files are retained only as
migration evidence; new execution writes do not modify them.

Workflow state is migrating in reviewable slices. Task Ledger events and
projections, Plan events and projections, Deep Research events, and Plan
Reminder records are now canonical in `runtime.sqlite`.
Desktop and Runtime Host production wiring opens these SQLite repositories;
their JSON/JSONL predecessors are read only during a fingerprinted, crash-safe
cutover and are never updated by later mutations.

Usage telemetry and pricing authority now use that same operational database.
Legacy `telemetry.json` and `pricing.json` sources are decoded together and
fingerprinted before their rows and pricing revision are committed atomically.
After cutover, Desktop and Runtime Host write only `runtime.sqlite`; the source
files remain unchanged as migration evidence.

Artifact metadata and lifecycle state now follow the same rule. Payload bytes
remain files, but their records are canonical in `runtime.sqlite` after a
fingerprinted `metadata.jsonl` cutover. Payload publication keeps its durable
staging/link protocol: recovery removes bytes whose metadata transaction did not
commit and preserves committed bytes while cleaning staging residue. Purge
intent recovery likewise completes against the SQLite metadata authority.

Selected-session bundle export now reads that SQLite authority through a
WAL-consistent snapshot, verifies that retained legacy evidence still matches
its completed cutover, and writes only the selected Artifact rows into the
bundle's `runtime.sqlite`. Payload bytes are copied under the Artifact writer
lock, cross-session rows and payloads are excluded, and bundles no longer emit
`artifacts/metadata.jsonl`.

Full operational backup now uses the shared database owner's online SQLite
backup API rather than copying `runtime.sqlite` or its WAL sidecars. A strict
manifest binds the standalone database snapshot to every active session
transcript and canonical Artifact payload by size and SHA-256. Restore verifies
SQLite integrity, foreign keys, supported schema versions, relational identity
sets, transcript decodability, and the exact manifested file tree before
atomically publishing a new state root. Interrupted backup or restore staging
is removed and can be retried without changing either source.
`runtime.sqlite` is the sole operational authority. It owns RuntimeEvents,
session metadata and message history, Agent Graph control, core execution state,
workflow state, usage and pricing, Artifact metadata, Automations, Daily Review,
and Headless TaskRuns. Artifact payload bytes remain regular files under
`artifacts/`; connections, credentials, settings, MCP configuration, skills,
and device identity remain configuration files.

This storage generation does not import earlier File/JSONL authorities. On
upgrade, legacy session titles may still be discoverable through current
metadata, but conversation history that exists only in legacy transcript files
is not copied into `session_messages` and opens as an empty thread. Likewise,
pre-version or `safeStorage`-encrypted credential/token files are not migrated;
users with only those copies must re-authenticate. This data-loss boundary is
intentional for this release and must be considered before upgrading an
existing workspace.

Full operational backup uses the database owner's online SQLite backup API and
copies canonical Artifact payloads under the Artifact writer lock. Its manifest
binds every file by size and SHA-256. Validation checks the standalone SQLite
snapshot's integrity, foreign keys, schema registry and required tables,
decodes canonical session-message and Artifact records, and verifies Artifact
payload sizes against SQLite metadata before restore. Backup and restore use
owner-only file modes, file and directory synchronization, staging, and atomic
publication.

Headless trajectory hydration now consumes a frozen selected-session export
from that SQLite Artifact authority. The cell publishes `trajectory-state`
Expand All@@ -254,21 +220,6 @@ validated snapshot. It does not copy a live WAL or fall back to
`artifacts/metadata.jsonl`. Missing, corrupt, unsupported, or mismatched
evidence fails closed to a summary trajectory instead of mixing authorities.

The remaining storage work is deliberately classified rather than implied
complete:

- Artifact metadata no longer exposes a production JSONL writer;
`artifacts/metadata.jsonl` is accepted only as fingerprinted, read-only
cutover evidence, and can be removed after the migration/cutover matrix is
complete;
- StoredMessage transcript bodies remain append-only JSONL;
- automation, connections, credentials, settings, MCP configuration, skills,
and device identity are configuration state and stay outside this operational
migration;
- Headless TaskRun evaluation ledgers, imported foreign-session caches, Daily
Review archives, and quote-cleanup bookkeeping are separate product/evaluation
domains and are not part of issue #1649.

Runtime continuation remains opt-in:

- `MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1` enables the Desktop interrupted-turn
Expand Down
11 changes: 5 additions & 6 deletions SECURITY.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,9 +97,9 @@ supply their own boundary. See
is the single authority every surface — Desktop, TUI, headless —
reads and writes, under the same OS-account and 0o700/0o600 boundary
as other runtime credentials. Electron safeStorage is not part of
this boundary anymore; Desktop startup imports pre-existing
safeStorage-encrypted token files into the store once and removes
them (#1125).
this boundary anymore. Pre-existing safeStorage-encrypted credential
or token files are not imported; users with only those copies must
re-authenticate.
3. **Renderer process sandbox + preload IPC bridge.** The
renderer cannot reach files, network, or shell directly. Every
IPC handler in `apps/desktop/src/main/main.ts` is the trust
Expand DownExpand Up@@ -152,7 +152,7 @@ are welcome as ordinary issues, not security advisories.
Beyond security, Maka treats the following as user-facing
privacy commitments:

- **Workspace JSONL stays local.** Session messages, tool
- **Workspace state stays local.** Session messages, tool
results, telemetry, settings are stored under
`app.getPath('userData')`. Cloud sync is not shipped.
- **Tool query strings are NEVER logged.** The WebSearch tool's
Expand All@@ -179,8 +179,7 @@ boundary in §2.3 was crossed. Examples:
looser than the 0o700/0o600 boundary.
- A production OAuth path writes or requires a safeStorage-encrypted
token copy again. `credentials.json` is the single documented token
authority; safeStorage exists only inside the one-shot legacy import
(#1125).
authority; there is no safeStorage legacy import fallback.
- A cleartext secret crosses main→renderer IPC under any
circumstance (including error paths, settings preview, IPC
result envelopes, log lines).
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ The Electron desktop app: `main` (Node/Electron main process) + `preload` (conte

Sub-folders: `browser/` (embedded browser view), `oauth/`, `search/` (thread search), `web-search/`, `types/`. The browser IPC handler itself (`browser-ipc-main.ts`) is flat in `src/main/`, not under `browser/`.

`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (credential migration, connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.
`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.

## IPC contract

Expand Down
5 changes: 2 additions & 3 deletions apps/desktop/e2e/fixtures.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -356,9 +356,8 @@ export const test = base.extend<{
);
},
// Stale sessions: boots the e2e-fixture `stale-sessions` fixture — one
// healthy session (zai-live, secret seeded), one unlocked fake session
// (opened active), and one locked legacy session whose connection is
// gone. Exercises the #1038 health-notice authority against real IPC
// healthy session (zai-live, secret seeded) and one locked fake-backend session
// (opened active). Exercises the #1038 health-notice authority against real IPC
// (connection list, hasSecret probe, connectionLocked summaries).
// Readiness = turns on screen: the fake session is open.
staleSessionsWindow: async ({}, use) => {
Expand Down
25 changes: 14 additions & 11 deletions apps/desktop/e2e/session-health-notice.spec.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,14 +3,15 @@
// "will the next send fail?" from the same facts as the send gate
// (connection list, hasSecret probe, connectionLocked on the summary).
//
// The stale-sessions e2e-fixture seeds the exact on-disk states, and
// both stale sessions carry user messages, so storage self-heals them to
// `connectionLocked: true` on first read — the send can neither use
// their connections nor silently rebind, even though a healthy default
// The stale-sessions e2e-fixture seeds the exact on-disk states: one
// locked fake-backend session and one healthy ai-sdk session. The stale
// session carries user messages, so storage self-heals it to
// `connectionLocked: true` on first read — the send can neither use its
// connection nor silently rebind, even though a healthy default
// connection exists. The old "default exists && enabled" proxy hid the
// notice in exactly this state (#1038 case 1); it must now show.
// The silent-rebind counterpart (unlocked empty stale session) is
// covered by the projection and notice unit tests.
// The deleted-connection and legacy-backend notice variants are covered
// by deriveSessionHealthNotice unit tests.

import { test, expect } from './fixtures';

Expand DownExpand Up@@ -39,7 +40,7 @@ async function probeNoticeAlignment(page: import('@playwright/test').Page) {
});
}

test('locked stale sessions show the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
test('a locked stale session shows the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
// Active = stale fake session (locked by its history): notice shows.
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();

Expand All@@ -48,12 +49,14 @@ test('locked stale sessions show the health notice even with a ready default', a
expect(alignment.leftDelta).toBeLessThanOrEqual(1);
expect(alignment.rightDelta).toBeLessThanOrEqual(1);

// Switch to the locked legacy session → its deleted-connection notice.
// A healthy session must not show the notice.
await page.getByRole('button', { name: '展开侧边栏' }).click();
await page.getByText('旧的 Claude 连接会话').first().click();
await expect(page.getByText('连接已删除')).toBeVisible();
await page.getByText('正常会话(Z.ai Live)').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeHidden();

// Click-through lands in Settings · 模型.
// Back on the stale session, click-through lands in Settings · 模型.
await page.getByText('旧的本地模拟会话').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();
await page.getByRole('button', { name: '去模型' }).click();
await expect(page.getByLabel('设置内容')).toBeVisible();
// The connection list itself, not just the settings shell: `add-connection`
Expand Down
27 changes: 12 additions & 15 deletions apps/desktop/src/main/__tests__/automation-persistence-e2e.test.ts
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
/**
* End-to-end: durable cron persistence + cross-session query/management.
*
* Exercises the REAL host wiring (createMainAutomationWiring) against a REAL
* FileAutomationStore on a real temp workspace, simulating an app restart:
* Exercises the real host wiring against the operational SQLite authority,
* simulating an app restart:
*
* session A creates a durable cron ──sync──► <workspace>/automations.json
* │
* (restart: a fresh wiring loads it) ◄──loadAll────────┘
* │
* session A creates a durable cron ──sync──► runtime.sqlite
* (restart: a fresh wiring loads it) ◄──loadAll───────┘
* session B (never saw it) lists / pauses / resumes / deletes it
*
* This is the query-and-persistence loop the reviewer asked for: a persisted
Expand All@@ -18,10 +16,11 @@

import { strict as assert } from 'node:assert';
import { describe, it, before, after } from 'node:test';
import { mkdtemp, rm, readFile } from 'node:fs/promises';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { MakaToolContext, MakaTool } from '@maka/runtime';
import { createAutomationStore } from '@maka/storage';
import { createMainAutomationWiring } from '../automation-wiring.js';

function ctx(sessionId: string): MakaToolContext {
Expand DownExpand Up@@ -60,15 +59,13 @@ function automationTool(wiring: ReturnType<typeof makeWiring>): MakaTool {
}

async function readStore(workspaceRoot: string): Promise<Array<{ id: string; name: string }>> {
try {
const raw = await readFile(join(workspaceRoot, 'automations.json'), 'utf8');
return (JSON.parse(raw) as { automations: Array<{ id: string; name: string }> }).automations;
} catch {
return [];
}
return (await createAutomationStore(workspaceRoot).loadAll()).map(({ id, name }) => ({
id,
name,
}));
}

/** The store sync is fire-and-forget; poll the file until it settles. */
/** Store sync is fire-and-forget; poll the authority until it settles. */
async function waitForStore(
workspaceRoot: string,
predicate: (rows: Array<{ id: string; name: string }>) => boolean,
Expand DownExpand Up@@ -175,7 +172,7 @@ describe('E2E: durable cron persistence + cross-session query/management', () =>
});

describe('E2E: a cron-disabled host (CLI) sharing the workspace never clobbers durable crons', () => {
it('a heartbeat-only wiring neither loads nor overwrites the owner\'s automations.json', async () => {
it('a heartbeat-only wiring neither loads nor overwrites durable Automations', async () => {
const ws = await mkdtemp(join(tmpdir(), 'maka-automation-clobber-'));
try {
// ── owner (cron-enabled, desktop) creates a durable cron ──────────────
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' refactor(storage): make SQLite the sole operational authority by jackwener · Pull Request #1994 · apache/maka · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 29 additions & 78 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -152,7 +152,7 @@ Start with [ARCHITECTURE.md](./ARCHITECTURE.md). It provides the system map, cod
apps/desktop/ Electron main / preload / React renderer

packages/core/ Pure contracts for Sessions, Events, Permissions, and Connections
packages/storage/ SQLite operational state, legacy importers, and payload stores
packages/storage/ SQLite operational state, configuration, and payload stores
packages/runtime/ AgentRun, model adapters, tools, context, and recovery
packages/headless/ TaskRun, Autonomous Loop, Self-check, eval, and AHE
packages/cli/ TUI and non-interactive CLI
Expand All@@ -168,83 +168,49 @@ Maka stores workspace data under Electron `userData` by default:

```text
<Electron userData>/workspaces/default/
runtime.sqlite
llm-connections.json
credentials.json
settings.json
sessions/
artifacts/
```

Current boundaries that matter:

- Sessionsand connection metadata live in the local filesystem;
- Sessions, messages, execution ledgers, workflows, usage, Automations, Daily Review, and Headless TaskRuns live in `runtime.sqlite`;
- Runtime credentials such as API keys, bot tokens, and proxy passwords currently live in local plaintext `credentials.json`, behind the OS account boundary, with POSIX directory mode `0700` and file mode `0600` enforced;
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless; Electron `safeStorage` only decrypts pre-existing legacy token files once at desktop startup (#1125);
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless. Pre-existing Electron `safeStorage` credential/token files are not imported; affected users must re-authenticate;
- Renderer does not receive plaintext credentials. File writes, Shell, and dangerous tool calls pass through the permission engine;
- Headless real-model evaluation fails closed by default and requires an explicit external isolation boundary.

Read [SECURITY.md](./SECURITY.md) for security reporting and policy, and [docs/README.md](./docs/README.md) for current privacy and sandbox contracts.

## Runtime storage and recovery

RuntimeEvent persistence is always canonical in `runtime.sqlite`. On the first
write, Maka batch-idempotently imports legacy RuntimeEvent JSONL without
rewriting it. Legacy-only workspaces remain available to read-only inspection
until that first write.

Session metadata and Agent Graph control tables now use the same process-local
operational database owner and the same `runtime.sqlite` transaction authority.
The first operational open copies a WAL-consistent `sessions.sqlite` source
into `runtime.sqlite`, validates every source row, and records the source digest
and result in `cutover_journal`. An interrupted copy resumes without partial
rows; a legacy database changed after cutover fails closed. The old database is
retained as migration evidence but is no longer a production writer. Session
transcript bodies remain append-only JSONL.

Core execution state now shares that authority too: AgentRun headers and event
ledgers, event projections, root-turn admissions and source proofs,
Interactions, Host Epoch message receipts, and ShellRun records are canonical
in `runtime.sqlite` across CLI, Desktop, Runtime Host, and Headless. Each legacy
file store is fingerprinted and imported through its own durable
`cutover_journal` entry before the corresponding repository opens. Copy and
validation are one SQLite transaction, retries are idempotent, and a changed
legacy source after cutover fails closed. The legacy files are retained only as
migration evidence; new execution writes do not modify them.

Workflow state is migrating in reviewable slices. Task Ledger events and
projections, Plan events and projections, Deep Research events, and Plan
Reminder records are now canonical in `runtime.sqlite`.
Desktop and Runtime Host production wiring opens these SQLite repositories;
their JSON/JSONL predecessors are read only during a fingerprinted, crash-safe
cutover and are never updated by later mutations.

Usage telemetry and pricing authority now use that same operational database.
Legacy `telemetry.json` and `pricing.json` sources are decoded together and
fingerprinted before their rows and pricing revision are committed atomically.
After cutover, Desktop and Runtime Host write only `runtime.sqlite`; the source
files remain unchanged as migration evidence.

Artifact metadata and lifecycle state now follow the same rule. Payload bytes
remain files, but their records are canonical in `runtime.sqlite` after a
fingerprinted `metadata.jsonl` cutover. Payload publication keeps its durable
staging/link protocol: recovery removes bytes whose metadata transaction did not
commit and preserves committed bytes while cleaning staging residue. Purge
intent recovery likewise completes against the SQLite metadata authority.

Selected-session bundle export now reads that SQLite authority through a
WAL-consistent snapshot, verifies that retained legacy evidence still matches
its completed cutover, and writes only the selected Artifact rows into the
bundle's `runtime.sqlite`. Payload bytes are copied under the Artifact writer
lock, cross-session rows and payloads are excluded, and bundles no longer emit
`artifacts/metadata.jsonl`.

Full operational backup now uses the shared database owner's online SQLite
backup API rather than copying `runtime.sqlite` or its WAL sidecars. A strict
manifest binds the standalone database snapshot to every active session
transcript and canonical Artifact payload by size and SHA-256. Restore verifies
SQLite integrity, foreign keys, supported schema versions, relational identity
sets, transcript decodability, and the exact manifested file tree before
atomically publishing a new state root. Interrupted backup or restore staging
is removed and can be retried without changing either source.
`runtime.sqlite` is the sole operational authority. It owns RuntimeEvents,
session metadata and message history, Agent Graph control, core execution state,
workflow state, usage and pricing, Artifact metadata, Automations, Daily Review,
and Headless TaskRuns. Artifact payload bytes remain regular files under
`artifacts/`; connections, credentials, settings, MCP configuration, skills,
and device identity remain configuration files.

This storage generation does not import earlier File/JSONL authorities. On
upgrade, legacy session titles may still be discoverable through current
metadata, but conversation history that exists only in legacy transcript files
is not copied into `session_messages` and opens as an empty thread. Likewise,
pre-version or `safeStorage`-encrypted credential/token files are not migrated;
users with only those copies must re-authenticate. This data-loss boundary is
intentional for this release and must be considered before upgrading an
existing workspace.

Full operational backup uses the database owner's online SQLite backup API and
copies canonical Artifact payloads under the Artifact writer lock. Its manifest
binds every file by size and SHA-256. Validation checks the standalone SQLite
snapshot's integrity, foreign keys, schema registry and required tables,
decodes canonical session-message and Artifact records, and verifies Artifact
payload sizes against SQLite metadata before restore. Backup and restore use
owner-only file modes, file and directory synchronization, staging, and atomic
publication.

Headless trajectory hydration now consumes a frozen selected-session export
from that SQLite Artifact authority. The cell publishes `trajectory-state`
Expand All@@ -254,21 +220,6 @@ validated snapshot. It does not copy a live WAL or fall back to
`artifacts/metadata.jsonl`. Missing, corrupt, unsupported, or mismatched
evidence fails closed to a summary trajectory instead of mixing authorities.

The remaining storage work is deliberately classified rather than implied
complete:

- Artifact metadata no longer exposes a production JSONL writer;
`artifacts/metadata.jsonl` is accepted only as fingerprinted, read-only
cutover evidence, and can be removed after the migration/cutover matrix is
complete;
- StoredMessage transcript bodies remain append-only JSONL;
- automation, connections, credentials, settings, MCP configuration, skills,
and device identity are configuration state and stay outside this operational
migration;
- Headless TaskRun evaluation ledgers, imported foreign-session caches, Daily
Review archives, and quote-cleanup bookkeeping are separate product/evaluation
domains and are not part of issue #1649.

Runtime continuation remains opt-in:

- `MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1` enables the Desktop interrupted-turn
Expand Down
11 changes: 5 additions & 6 deletions SECURITY.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,9 +97,9 @@ supply their own boundary. See
is the single authority every surface — Desktop, TUI, headless —
reads and writes, under the same OS-account and 0o700/0o600 boundary
as other runtime credentials. Electron safeStorage is not part of
this boundary anymore; Desktop startup imports pre-existing
safeStorage-encrypted token files into the store once and removes
them (#1125).
this boundary anymore. Pre-existing safeStorage-encrypted credential
or token files are not imported; users with only those copies must
re-authenticate.
3. **Renderer process sandbox + preload IPC bridge.** The
renderer cannot reach files, network, or shell directly. Every
IPC handler in `apps/desktop/src/main/main.ts` is the trust
Expand DownExpand Up@@ -152,7 +152,7 @@ are welcome as ordinary issues, not security advisories.
Beyond security, Maka treats the following as user-facing
privacy commitments:

- **Workspace JSONL stays local.** Session messages, tool
- **Workspace state stays local.** Session messages, tool
results, telemetry, settings are stored under
`app.getPath('userData')`. Cloud sync is not shipped.
- **Tool query strings are NEVER logged.** The WebSearch tool's
Expand All@@ -179,8 +179,7 @@ boundary in §2.3 was crossed. Examples:
looser than the 0o700/0o600 boundary.
- A production OAuth path writes or requires a safeStorage-encrypted
token copy again. `credentials.json` is the single documented token
authority; safeStorage exists only inside the one-shot legacy import
(#1125).
authority; there is no safeStorage legacy import fallback.
- A cleartext secret crosses main→renderer IPC under any
circumstance (including error paths, settings preview, IPC
result envelopes, log lines).
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ The Electron desktop app: `main` (Node/Electron main process) + `preload` (conte

Sub-folders: `browser/` (embedded browser view), `oauth/`, `search/` (thread search), `web-search/`, `types/`. The browser IPC handler itself (`browser-ipc-main.ts`) is flat in `src/main/`, not under `browser/`.

`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (credential migration, connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.
`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.

## IPC contract

Expand Down
5 changes: 2 additions & 3 deletions apps/desktop/e2e/fixtures.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -356,9 +356,8 @@ export const test = base.extend<{
);
},
// Stale sessions: boots the e2e-fixture `stale-sessions` fixture — one
// healthy session (zai-live, secret seeded), one unlocked fake session
// (opened active), and one locked legacy session whose connection is
// gone. Exercises the #1038 health-notice authority against real IPC
// healthy session (zai-live, secret seeded) and one locked fake-backend session
// (opened active). Exercises the #1038 health-notice authority against real IPC
// (connection list, hasSecret probe, connectionLocked summaries).
// Readiness = turns on screen: the fake session is open.
staleSessionsWindow: async ({}, use) => {
Expand Down
25 changes: 14 additions & 11 deletions apps/desktop/e2e/session-health-notice.spec.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,14 +3,15 @@
// "will the next send fail?" from the same facts as the send gate
// (connection list, hasSecret probe, connectionLocked on the summary).
//
// The stale-sessions e2e-fixture seeds the exact on-disk states, and
// both stale sessions carry user messages, so storage self-heals them to
// `connectionLocked: true` on first read — the send can neither use
// their connections nor silently rebind, even though a healthy default
// The stale-sessions e2e-fixture seeds the exact on-disk states: one
// locked fake-backend session and one healthy ai-sdk session. The stale
// session carries user messages, so storage self-heals it to
// `connectionLocked: true` on first read — the send can neither use its
// connection nor silently rebind, even though a healthy default
// connection exists. The old "default exists && enabled" proxy hid the
// notice in exactly this state (#1038 case 1); it must now show.
// The silent-rebind counterpart (unlocked empty stale session) is
// covered by the projection and notice unit tests.
// The deleted-connection and legacy-backend notice variants are covered
// by deriveSessionHealthNotice unit tests.

import { test, expect } from './fixtures';

Expand DownExpand Up@@ -39,7 +40,7 @@ async function probeNoticeAlignment(page: import('@playwright/test').Page) {
});
}

test('locked stale sessions show the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
test('a locked stale session shows the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
// Active = stale fake session (locked by its history): notice shows.
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();

Expand All@@ -48,12 +49,14 @@ test('locked stale sessions show the health notice even with a ready default', a
expect(alignment.leftDelta).toBeLessThanOrEqual(1);
expect(alignment.rightDelta).toBeLessThanOrEqual(1);

// Switch to the locked legacy session → its deleted-connection notice.
// A healthy session must not show the notice.
await page.getByRole('button', { name: '展开侧边栏' }).click();
await page.getByText('旧的 Claude 连接会话').first().click();
await expect(page.getByText('连接已删除')).toBeVisible();
await page.getByText('正常会话(Z.ai Live)').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeHidden();

// Click-through lands in Settings · 模型.
// Back on the stale session, click-through lands in Settings · 模型.
await page.getByText('旧的本地模拟会话').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();
await page.getByRole('button', { name: '去模型' }).click();
await expect(page.getByLabel('设置内容')).toBeVisible();
// The connection list itself, not just the settings shell: `add-connection`
Expand Down
27 changes: 12 additions & 15 deletions apps/desktop/src/main/__tests__/automation-persistence-e2e.test.ts
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
/**
* End-to-end: durable cron persistence + cross-session query/management.
*
* Exercises the REAL host wiring (createMainAutomationWiring) against a REAL
* FileAutomationStore on a real temp workspace, simulating an app restart:
* Exercises the real host wiring against the operational SQLite authority,
* simulating an app restart:
*
* session A creates a durable cron ──sync──► <workspace>/automations.json
* │
* (restart: a fresh wiring loads it) ◄──loadAll────────┘
* │
* session A creates a durable cron ──sync──► runtime.sqlite
* (restart: a fresh wiring loads it) ◄──loadAll───────┘
* session B (never saw it) lists / pauses / resumes / deletes it
*
* This is the query-and-persistence loop the reviewer asked for: a persisted
Expand All@@ -18,10 +16,11 @@

import { strict as assert } from 'node:assert';
import { describe, it, before, after } from 'node:test';
import { mkdtemp, rm, readFile } from 'node:fs/promises';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { MakaToolContext, MakaTool } from '@maka/runtime';
import { createAutomationStore } from '@maka/storage';
import { createMainAutomationWiring } from '../automation-wiring.js';

function ctx(sessionId: string): MakaToolContext {
Expand DownExpand Up@@ -60,15 +59,13 @@ function automationTool(wiring: ReturnType<typeof makeWiring>): MakaTool {
}

async function readStore(workspaceRoot: string): Promise<Array<{ id: string; name: string }>> {
try {
const raw = await readFile(join(workspaceRoot, 'automations.json'), 'utf8');
return (JSON.parse(raw) as { automations: Array<{ id: string; name: string }> }).automations;
} catch {
return [];
}
return (await createAutomationStore(workspaceRoot).loadAll()).map(({ id, name }) => ({
id,
name,
}));
}

/** The store sync is fire-and-forget; poll the file until it settles. */
/** Store sync is fire-and-forget; poll the authority until it settles. */
async function waitForStore(
workspaceRoot: string,
predicate: (rows: Array<{ id: string; name: string }>) => boolean,
Expand DownExpand Up@@ -175,7 +172,7 @@ describe('E2E: durable cron persistence + cross-session query/management', () =>
});

describe('E2E: a cron-disabled host (CLI) sharing the workspace never clobbers durable crons', () => {
it('a heartbeat-only wiring neither loads nor overwrites the owner\'s automations.json', async () => {
it('a heartbeat-only wiring neither loads nor overwrites durable Automations', async () => {
const ws = await mkdtemp(join(tmpdir(), 'maka-automation-clobber-'));
try {
// ── owner (cron-enabled, desktop) creates a durable cron ──────────────
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' refactor(storage): make SQLite the sole operational authority by jackwener · Pull Request #1994 · apache/maka · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 29 additions & 78 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -152,7 +152,7 @@ Start with [ARCHITECTURE.md](./ARCHITECTURE.md). It provides the system map, cod
apps/desktop/ Electron main / preload / React renderer

packages/core/ Pure contracts for Sessions, Events, Permissions, and Connections
packages/storage/ SQLite operational state, legacy importers, and payload stores
packages/storage/ SQLite operational state, configuration, and payload stores
packages/runtime/ AgentRun, model adapters, tools, context, and recovery
packages/headless/ TaskRun, Autonomous Loop, Self-check, eval, and AHE
packages/cli/ TUI and non-interactive CLI
Expand All@@ -168,83 +168,49 @@ Maka stores workspace data under Electron `userData` by default:

```text
<Electron userData>/workspaces/default/
runtime.sqlite
llm-connections.json
credentials.json
settings.json
sessions/
artifacts/
```

Current boundaries that matter:

- Sessionsand connection metadata live in the local filesystem;
- Sessions, messages, execution ledgers, workflows, usage, Automations, Daily Review, and Headless TaskRuns live in `runtime.sqlite`;
- Runtime credentials such as API keys, bot tokens, and proxy passwords currently live in local plaintext `credentials.json`, behind the OS account boundary, with POSIX directory mode `0700` and file mode `0600` enforced;
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless; Electron `safeStorage` only decrypts pre-existing legacy token files once at desktop startup (#1125);
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless. Pre-existing Electron `safeStorage` credential/token files are not imported; affected users must re-authenticate;
- Renderer does not receive plaintext credentials. File writes, Shell, and dangerous tool calls pass through the permission engine;
- Headless real-model evaluation fails closed by default and requires an explicit external isolation boundary.

Read [SECURITY.md](./SECURITY.md) for security reporting and policy, and [docs/README.md](./docs/README.md) for current privacy and sandbox contracts.

## Runtime storage and recovery

RuntimeEvent persistence is always canonical in `runtime.sqlite`. On the first
write, Maka batch-idempotently imports legacy RuntimeEvent JSONL without
rewriting it. Legacy-only workspaces remain available to read-only inspection
until that first write.

Session metadata and Agent Graph control tables now use the same process-local
operational database owner and the same `runtime.sqlite` transaction authority.
The first operational open copies a WAL-consistent `sessions.sqlite` source
into `runtime.sqlite`, validates every source row, and records the source digest
and result in `cutover_journal`. An interrupted copy resumes without partial
rows; a legacy database changed after cutover fails closed. The old database is
retained as migration evidence but is no longer a production writer. Session
transcript bodies remain append-only JSONL.

Core execution state now shares that authority too: AgentRun headers and event
ledgers, event projections, root-turn admissions and source proofs,
Interactions, Host Epoch message receipts, and ShellRun records are canonical
in `runtime.sqlite` across CLI, Desktop, Runtime Host, and Headless. Each legacy
file store is fingerprinted and imported through its own durable
`cutover_journal` entry before the corresponding repository opens. Copy and
validation are one SQLite transaction, retries are idempotent, and a changed
legacy source after cutover fails closed. The legacy files are retained only as
migration evidence; new execution writes do not modify them.

Workflow state is migrating in reviewable slices. Task Ledger events and
projections, Plan events and projections, Deep Research events, and Plan
Reminder records are now canonical in `runtime.sqlite`.
Desktop and Runtime Host production wiring opens these SQLite repositories;
their JSON/JSONL predecessors are read only during a fingerprinted, crash-safe
cutover and are never updated by later mutations.

Usage telemetry and pricing authority now use that same operational database.
Legacy `telemetry.json` and `pricing.json` sources are decoded together and
fingerprinted before their rows and pricing revision are committed atomically.
After cutover, Desktop and Runtime Host write only `runtime.sqlite`; the source
files remain unchanged as migration evidence.

Artifact metadata and lifecycle state now follow the same rule. Payload bytes
remain files, but their records are canonical in `runtime.sqlite` after a
fingerprinted `metadata.jsonl` cutover. Payload publication keeps its durable
staging/link protocol: recovery removes bytes whose metadata transaction did not
commit and preserves committed bytes while cleaning staging residue. Purge
intent recovery likewise completes against the SQLite metadata authority.

Selected-session bundle export now reads that SQLite authority through a
WAL-consistent snapshot, verifies that retained legacy evidence still matches
its completed cutover, and writes only the selected Artifact rows into the
bundle's `runtime.sqlite`. Payload bytes are copied under the Artifact writer
lock, cross-session rows and payloads are excluded, and bundles no longer emit
`artifacts/metadata.jsonl`.

Full operational backup now uses the shared database owner's online SQLite
backup API rather than copying `runtime.sqlite` or its WAL sidecars. A strict
manifest binds the standalone database snapshot to every active session
transcript and canonical Artifact payload by size and SHA-256. Restore verifies
SQLite integrity, foreign keys, supported schema versions, relational identity
sets, transcript decodability, and the exact manifested file tree before
atomically publishing a new state root. Interrupted backup or restore staging
is removed and can be retried without changing either source.
`runtime.sqlite` is the sole operational authority. It owns RuntimeEvents,
session metadata and message history, Agent Graph control, core execution state,
workflow state, usage and pricing, Artifact metadata, Automations, Daily Review,
and Headless TaskRuns. Artifact payload bytes remain regular files under
`artifacts/`; connections, credentials, settings, MCP configuration, skills,
and device identity remain configuration files.

This storage generation does not import earlier File/JSONL authorities. On
upgrade, legacy session titles may still be discoverable through current
metadata, but conversation history that exists only in legacy transcript files
is not copied into `session_messages` and opens as an empty thread. Likewise,
pre-version or `safeStorage`-encrypted credential/token files are not migrated;
users with only those copies must re-authenticate. This data-loss boundary is
intentional for this release and must be considered before upgrading an
existing workspace.

Full operational backup uses the database owner's online SQLite backup API and
copies canonical Artifact payloads under the Artifact writer lock. Its manifest
binds every file by size and SHA-256. Validation checks the standalone SQLite
snapshot's integrity, foreign keys, schema registry and required tables,
decodes canonical session-message and Artifact records, and verifies Artifact
payload sizes against SQLite metadata before restore. Backup and restore use
owner-only file modes, file and directory synchronization, staging, and atomic
publication.

Headless trajectory hydration now consumes a frozen selected-session export
from that SQLite Artifact authority. The cell publishes `trajectory-state`
Expand All@@ -254,21 +220,6 @@ validated snapshot. It does not copy a live WAL or fall back to
`artifacts/metadata.jsonl`. Missing, corrupt, unsupported, or mismatched
evidence fails closed to a summary trajectory instead of mixing authorities.

The remaining storage work is deliberately classified rather than implied
complete:

- Artifact metadata no longer exposes a production JSONL writer;
`artifacts/metadata.jsonl` is accepted only as fingerprinted, read-only
cutover evidence, and can be removed after the migration/cutover matrix is
complete;
- StoredMessage transcript bodies remain append-only JSONL;
- automation, connections, credentials, settings, MCP configuration, skills,
and device identity are configuration state and stay outside this operational
migration;
- Headless TaskRun evaluation ledgers, imported foreign-session caches, Daily
Review archives, and quote-cleanup bookkeeping are separate product/evaluation
domains and are not part of issue #1649.

Runtime continuation remains opt-in:

- `MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1` enables the Desktop interrupted-turn
Expand Down
11 changes: 5 additions & 6 deletions SECURITY.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,9 +97,9 @@ supply their own boundary. See
is the single authority every surface — Desktop, TUI, headless —
reads and writes, under the same OS-account and 0o700/0o600 boundary
as other runtime credentials. Electron safeStorage is not part of
this boundary anymore; Desktop startup imports pre-existing
safeStorage-encrypted token files into the store once and removes
them (#1125).
this boundary anymore. Pre-existing safeStorage-encrypted credential
or token files are not imported; users with only those copies must
re-authenticate.
3. **Renderer process sandbox + preload IPC bridge.** The
renderer cannot reach files, network, or shell directly. Every
IPC handler in `apps/desktop/src/main/main.ts` is the trust
Expand DownExpand Up@@ -152,7 +152,7 @@ are welcome as ordinary issues, not security advisories.
Beyond security, Maka treats the following as user-facing
privacy commitments:

- **Workspace JSONL stays local.** Session messages, tool
- **Workspace state stays local.** Session messages, tool
results, telemetry, settings are stored under
`app.getPath('userData')`. Cloud sync is not shipped.
- **Tool query strings are NEVER logged.** The WebSearch tool's
Expand All@@ -179,8 +179,7 @@ boundary in §2.3 was crossed. Examples:
looser than the 0o700/0o600 boundary.
- A production OAuth path writes or requires a safeStorage-encrypted
token copy again. `credentials.json` is the single documented token
authority; safeStorage exists only inside the one-shot legacy import
(#1125).
authority; there is no safeStorage legacy import fallback.
- A cleartext secret crosses main→renderer IPC under any
circumstance (including error paths, settings preview, IPC
result envelopes, log lines).
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ The Electron desktop app: `main` (Node/Electron main process) + `preload` (conte

Sub-folders: `browser/` (embedded browser view), `oauth/`, `search/` (thread search), `web-search/`, `types/`. The browser IPC handler itself (`browser-ipc-main.ts`) is flat in `src/main/`, not under `browser/`.

`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (credential migration, connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.
`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.

## IPC contract

Expand Down
5 changes: 2 additions & 3 deletions apps/desktop/e2e/fixtures.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -356,9 +356,8 @@ export const test = base.extend<{
);
},
// Stale sessions: boots the e2e-fixture `stale-sessions` fixture — one
// healthy session (zai-live, secret seeded), one unlocked fake session
// (opened active), and one locked legacy session whose connection is
// gone. Exercises the #1038 health-notice authority against real IPC
// healthy session (zai-live, secret seeded) and one locked fake-backend session
// (opened active). Exercises the #1038 health-notice authority against real IPC
// (connection list, hasSecret probe, connectionLocked summaries).
// Readiness = turns on screen: the fake session is open.
staleSessionsWindow: async ({}, use) => {
Expand Down
25 changes: 14 additions & 11 deletions apps/desktop/e2e/session-health-notice.spec.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,14 +3,15 @@
// "will the next send fail?" from the same facts as the send gate
// (connection list, hasSecret probe, connectionLocked on the summary).
//
// The stale-sessions e2e-fixture seeds the exact on-disk states, and
// both stale sessions carry user messages, so storage self-heals them to
// `connectionLocked: true` on first read — the send can neither use
// their connections nor silently rebind, even though a healthy default
// The stale-sessions e2e-fixture seeds the exact on-disk states: one
// locked fake-backend session and one healthy ai-sdk session. The stale
// session carries user messages, so storage self-heals it to
// `connectionLocked: true` on first read — the send can neither use its
// connection nor silently rebind, even though a healthy default
// connection exists. The old "default exists && enabled" proxy hid the
// notice in exactly this state (#1038 case 1); it must now show.
// The silent-rebind counterpart (unlocked empty stale session) is
// covered by the projection and notice unit tests.
// The deleted-connection and legacy-backend notice variants are covered
// by deriveSessionHealthNotice unit tests.

import { test, expect } from './fixtures';

Expand DownExpand Up@@ -39,7 +40,7 @@ async function probeNoticeAlignment(page: import('@playwright/test').Page) {
});
}

test('locked stale sessions show the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
test('a locked stale session shows the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
// Active = stale fake session (locked by its history): notice shows.
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();

Expand All@@ -48,12 +49,14 @@ test('locked stale sessions show the health notice even with a ready default', a
expect(alignment.leftDelta).toBeLessThanOrEqual(1);
expect(alignment.rightDelta).toBeLessThanOrEqual(1);

// Switch to the locked legacy session → its deleted-connection notice.
// A healthy session must not show the notice.
await page.getByRole('button', { name: '展开侧边栏' }).click();
await page.getByText('旧的 Claude 连接会话').first().click();
await expect(page.getByText('连接已删除')).toBeVisible();
await page.getByText('正常会话(Z.ai Live)').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeHidden();

// Click-through lands in Settings · 模型.
// Back on the stale session, click-through lands in Settings · 模型.
await page.getByText('旧的本地模拟会话').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();
await page.getByRole('button', { name: '去模型' }).click();
await expect(page.getByLabel('设置内容')).toBeVisible();
// The connection list itself, not just the settings shell: `add-connection`
Expand Down
27 changes: 12 additions & 15 deletions apps/desktop/src/main/__tests__/automation-persistence-e2e.test.ts
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
/**
* End-to-end: durable cron persistence + cross-session query/management.
*
* Exercises the REAL host wiring (createMainAutomationWiring) against a REAL
* FileAutomationStore on a real temp workspace, simulating an app restart:
* Exercises the real host wiring against the operational SQLite authority,
* simulating an app restart:
*
* session A creates a durable cron ──sync──► <workspace>/automations.json
* │
* (restart: a fresh wiring loads it) ◄──loadAll────────┘
* │
* session A creates a durable cron ──sync──► runtime.sqlite
* (restart: a fresh wiring loads it) ◄──loadAll───────┘
* session B (never saw it) lists / pauses / resumes / deletes it
*
* This is the query-and-persistence loop the reviewer asked for: a persisted
Expand All@@ -18,10 +16,11 @@

import { strict as assert } from 'node:assert';
import { describe, it, before, after } from 'node:test';
import { mkdtemp, rm, readFile } from 'node:fs/promises';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { MakaToolContext, MakaTool } from '@maka/runtime';
import { createAutomationStore } from '@maka/storage';
import { createMainAutomationWiring } from '../automation-wiring.js';

function ctx(sessionId: string): MakaToolContext {
Expand DownExpand Up@@ -60,15 +59,13 @@ function automationTool(wiring: ReturnType<typeof makeWiring>): MakaTool {
}

async function readStore(workspaceRoot: string): Promise<Array<{ id: string; name: string }>> {
try {
const raw = await readFile(join(workspaceRoot, 'automations.json'), 'utf8');
return (JSON.parse(raw) as { automations: Array<{ id: string; name: string }> }).automations;
} catch {
return [];
}
return (await createAutomationStore(workspaceRoot).loadAll()).map(({ id, name }) => ({
id,
name,
}));
}

/** The store sync is fire-and-forget; poll the file until it settles. */
/** Store sync is fire-and-forget; poll the authority until it settles. */
async function waitForStore(
workspaceRoot: string,
predicate: (rows: Array<{ id: string; name: string }>) => boolean,
Expand DownExpand Up@@ -175,7 +172,7 @@ describe('E2E: durable cron persistence + cross-session query/management', () =>
});

describe('E2E: a cron-disabled host (CLI) sharing the workspace never clobbers durable crons', () => {
it('a heartbeat-only wiring neither loads nor overwrites the owner\'s automations.json', async () => {
it('a heartbeat-only wiring neither loads nor overwrites durable Automations', async () => {
const ws = await mkdtemp(join(tmpdir(), 'maka-automation-clobber-'));
try {
// ── owner (cron-enabled, desktop) creates a durable cron ──────────────
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' refactor(storage): make SQLite the sole operational authority by jackwener · Pull Request #1994 · apache/maka · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 29 additions & 78 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -152,7 +152,7 @@ Start with [ARCHITECTURE.md](./ARCHITECTURE.md). It provides the system map, cod
apps/desktop/ Electron main / preload / React renderer

packages/core/ Pure contracts for Sessions, Events, Permissions, and Connections
packages/storage/ SQLite operational state, legacy importers, and payload stores
packages/storage/ SQLite operational state, configuration, and payload stores
packages/runtime/ AgentRun, model adapters, tools, context, and recovery
packages/headless/ TaskRun, Autonomous Loop, Self-check, eval, and AHE
packages/cli/ TUI and non-interactive CLI
Expand All@@ -168,83 +168,49 @@ Maka stores workspace data under Electron `userData` by default:

```text
<Electron userData>/workspaces/default/
runtime.sqlite
llm-connections.json
credentials.json
settings.json
sessions/
artifacts/
```

Current boundaries that matter:

- Sessionsand connection metadata live in the local filesystem;
- Sessions, messages, execution ledgers, workflows, usage, Automations, Daily Review, and Headless TaskRuns live in `runtime.sqlite`;
- Runtime credentials such as API keys, bot tokens, and proxy passwords currently live in local plaintext `credentials.json`, behind the OS account boundary, with POSIX directory mode `0700` and file mode `0600` enforced;
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless; Electron `safeStorage` only decrypts pre-existing legacy token files once at desktop startup (#1125);
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless. Pre-existing Electron `safeStorage` credential/token files are not imported; affected users must re-authenticate;
- Renderer does not receive plaintext credentials. File writes, Shell, and dangerous tool calls pass through the permission engine;
- Headless real-model evaluation fails closed by default and requires an explicit external isolation boundary.

Read [SECURITY.md](./SECURITY.md) for security reporting and policy, and [docs/README.md](./docs/README.md) for current privacy and sandbox contracts.

## Runtime storage and recovery

RuntimeEvent persistence is always canonical in `runtime.sqlite`. On the first
write, Maka batch-idempotently imports legacy RuntimeEvent JSONL without
rewriting it. Legacy-only workspaces remain available to read-only inspection
until that first write.

Session metadata and Agent Graph control tables now use the same process-local
operational database owner and the same `runtime.sqlite` transaction authority.
The first operational open copies a WAL-consistent `sessions.sqlite` source
into `runtime.sqlite`, validates every source row, and records the source digest
and result in `cutover_journal`. An interrupted copy resumes without partial
rows; a legacy database changed after cutover fails closed. The old database is
retained as migration evidence but is no longer a production writer. Session
transcript bodies remain append-only JSONL.

Core execution state now shares that authority too: AgentRun headers and event
ledgers, event projections, root-turn admissions and source proofs,
Interactions, Host Epoch message receipts, and ShellRun records are canonical
in `runtime.sqlite` across CLI, Desktop, Runtime Host, and Headless. Each legacy
file store is fingerprinted and imported through its own durable
`cutover_journal` entry before the corresponding repository opens. Copy and
validation are one SQLite transaction, retries are idempotent, and a changed
legacy source after cutover fails closed. The legacy files are retained only as
migration evidence; new execution writes do not modify them.

Workflow state is migrating in reviewable slices. Task Ledger events and
projections, Plan events and projections, Deep Research events, and Plan
Reminder records are now canonical in `runtime.sqlite`.
Desktop and Runtime Host production wiring opens these SQLite repositories;
their JSON/JSONL predecessors are read only during a fingerprinted, crash-safe
cutover and are never updated by later mutations.

Usage telemetry and pricing authority now use that same operational database.
Legacy `telemetry.json` and `pricing.json` sources are decoded together and
fingerprinted before their rows and pricing revision are committed atomically.
After cutover, Desktop and Runtime Host write only `runtime.sqlite`; the source
files remain unchanged as migration evidence.

Artifact metadata and lifecycle state now follow the same rule. Payload bytes
remain files, but their records are canonical in `runtime.sqlite` after a
fingerprinted `metadata.jsonl` cutover. Payload publication keeps its durable
staging/link protocol: recovery removes bytes whose metadata transaction did not
commit and preserves committed bytes while cleaning staging residue. Purge
intent recovery likewise completes against the SQLite metadata authority.

Selected-session bundle export now reads that SQLite authority through a
WAL-consistent snapshot, verifies that retained legacy evidence still matches
its completed cutover, and writes only the selected Artifact rows into the
bundle's `runtime.sqlite`. Payload bytes are copied under the Artifact writer
lock, cross-session rows and payloads are excluded, and bundles no longer emit
`artifacts/metadata.jsonl`.

Full operational backup now uses the shared database owner's online SQLite
backup API rather than copying `runtime.sqlite` or its WAL sidecars. A strict
manifest binds the standalone database snapshot to every active session
transcript and canonical Artifact payload by size and SHA-256. Restore verifies
SQLite integrity, foreign keys, supported schema versions, relational identity
sets, transcript decodability, and the exact manifested file tree before
atomically publishing a new state root. Interrupted backup or restore staging
is removed and can be retried without changing either source.
`runtime.sqlite` is the sole operational authority. It owns RuntimeEvents,
session metadata and message history, Agent Graph control, core execution state,
workflow state, usage and pricing, Artifact metadata, Automations, Daily Review,
and Headless TaskRuns. Artifact payload bytes remain regular files under
`artifacts/`; connections, credentials, settings, MCP configuration, skills,
and device identity remain configuration files.

This storage generation does not import earlier File/JSONL authorities. On
upgrade, legacy session titles may still be discoverable through current
metadata, but conversation history that exists only in legacy transcript files
is not copied into `session_messages` and opens as an empty thread. Likewise,
pre-version or `safeStorage`-encrypted credential/token files are not migrated;
users with only those copies must re-authenticate. This data-loss boundary is
intentional for this release and must be considered before upgrading an
existing workspace.

Full operational backup uses the database owner's online SQLite backup API and
copies canonical Artifact payloads under the Artifact writer lock. Its manifest
binds every file by size and SHA-256. Validation checks the standalone SQLite
snapshot's integrity, foreign keys, schema registry and required tables,
decodes canonical session-message and Artifact records, and verifies Artifact
payload sizes against SQLite metadata before restore. Backup and restore use
owner-only file modes, file and directory synchronization, staging, and atomic
publication.

Headless trajectory hydration now consumes a frozen selected-session export
from that SQLite Artifact authority. The cell publishes `trajectory-state`
Expand All@@ -254,21 +220,6 @@ validated snapshot. It does not copy a live WAL or fall back to
`artifacts/metadata.jsonl`. Missing, corrupt, unsupported, or mismatched
evidence fails closed to a summary trajectory instead of mixing authorities.

The remaining storage work is deliberately classified rather than implied
complete:

- Artifact metadata no longer exposes a production JSONL writer;
`artifacts/metadata.jsonl` is accepted only as fingerprinted, read-only
cutover evidence, and can be removed after the migration/cutover matrix is
complete;
- StoredMessage transcript bodies remain append-only JSONL;
- automation, connections, credentials, settings, MCP configuration, skills,
and device identity are configuration state and stay outside this operational
migration;
- Headless TaskRun evaluation ledgers, imported foreign-session caches, Daily
Review archives, and quote-cleanup bookkeeping are separate product/evaluation
domains and are not part of issue #1649.

Runtime continuation remains opt-in:

- `MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1` enables the Desktop interrupted-turn
Expand Down
11 changes: 5 additions & 6 deletions SECURITY.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,9 +97,9 @@ supply their own boundary. See
is the single authority every surface — Desktop, TUI, headless —
reads and writes, under the same OS-account and 0o700/0o600 boundary
as other runtime credentials. Electron safeStorage is not part of
this boundary anymore; Desktop startup imports pre-existing
safeStorage-encrypted token files into the store once and removes
them (#1125).
this boundary anymore. Pre-existing safeStorage-encrypted credential
or token files are not imported; users with only those copies must
re-authenticate.
3. **Renderer process sandbox + preload IPC bridge.** The
renderer cannot reach files, network, or shell directly. Every
IPC handler in `apps/desktop/src/main/main.ts` is the trust
Expand DownExpand Up@@ -152,7 +152,7 @@ are welcome as ordinary issues, not security advisories.
Beyond security, Maka treats the following as user-facing
privacy commitments:

- **Workspace JSONL stays local.** Session messages, tool
- **Workspace state stays local.** Session messages, tool
results, telemetry, settings are stored under
`app.getPath('userData')`. Cloud sync is not shipped.
- **Tool query strings are NEVER logged.** The WebSearch tool's
Expand All@@ -179,8 +179,7 @@ boundary in §2.3 was crossed. Examples:
looser than the 0o700/0o600 boundary.
- A production OAuth path writes or requires a safeStorage-encrypted
token copy again. `credentials.json` is the single documented token
authority; safeStorage exists only inside the one-shot legacy import
(#1125).
authority; there is no safeStorage legacy import fallback.
- A cleartext secret crosses main→renderer IPC under any
circumstance (including error paths, settings preview, IPC
result envelopes, log lines).
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ The Electron desktop app: `main` (Node/Electron main process) + `preload` (conte

Sub-folders: `browser/` (embedded browser view), `oauth/`, `search/` (thread search), `web-search/`, `types/`. The browser IPC handler itself (`browser-ipc-main.ts`) is flat in `src/main/`, not under `browser/`.

`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (credential migration, connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.
`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.

## IPC contract

Expand Down
5 changes: 2 additions & 3 deletions apps/desktop/e2e/fixtures.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -356,9 +356,8 @@ export const test = base.extend<{
);
},
// Stale sessions: boots the e2e-fixture `stale-sessions` fixture — one
// healthy session (zai-live, secret seeded), one unlocked fake session
// (opened active), and one locked legacy session whose connection is
// gone. Exercises the #1038 health-notice authority against real IPC
// healthy session (zai-live, secret seeded) and one locked fake-backend session
// (opened active). Exercises the #1038 health-notice authority against real IPC
// (connection list, hasSecret probe, connectionLocked summaries).
// Readiness = turns on screen: the fake session is open.
staleSessionsWindow: async ({}, use) => {
Expand Down
25 changes: 14 additions & 11 deletions apps/desktop/e2e/session-health-notice.spec.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,14 +3,15 @@
// "will the next send fail?" from the same facts as the send gate
// (connection list, hasSecret probe, connectionLocked on the summary).
//
// The stale-sessions e2e-fixture seeds the exact on-disk states, and
// both stale sessions carry user messages, so storage self-heals them to
// `connectionLocked: true` on first read — the send can neither use
// their connections nor silently rebind, even though a healthy default
// The stale-sessions e2e-fixture seeds the exact on-disk states: one
// locked fake-backend session and one healthy ai-sdk session. The stale
// session carries user messages, so storage self-heals it to
// `connectionLocked: true` on first read — the send can neither use its
// connection nor silently rebind, even though a healthy default
// connection exists. The old "default exists && enabled" proxy hid the
// notice in exactly this state (#1038 case 1); it must now show.
// The silent-rebind counterpart (unlocked empty stale session) is
// covered by the projection and notice unit tests.
// The deleted-connection and legacy-backend notice variants are covered
// by deriveSessionHealthNotice unit tests.

import { test, expect } from './fixtures';

Expand DownExpand Up@@ -39,7 +40,7 @@ async function probeNoticeAlignment(page: import('@playwright/test').Page) {
});
}

test('locked stale sessions show the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
test('a locked stale session shows the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
// Active = stale fake session (locked by its history): notice shows.
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();

Expand All@@ -48,12 +49,14 @@ test('locked stale sessions show the health notice even with a ready default', a
expect(alignment.leftDelta).toBeLessThanOrEqual(1);
expect(alignment.rightDelta).toBeLessThanOrEqual(1);

// Switch to the locked legacy session → its deleted-connection notice.
// A healthy session must not show the notice.
await page.getByRole('button', { name: '展开侧边栏' }).click();
await page.getByText('旧的 Claude 连接会话').first().click();
await expect(page.getByText('连接已删除')).toBeVisible();
await page.getByText('正常会话(Z.ai Live)').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeHidden();

// Click-through lands in Settings · 模型.
// Back on the stale session, click-through lands in Settings · 模型.
await page.getByText('旧的本地模拟会话').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();
await page.getByRole('button', { name: '去模型' }).click();
await expect(page.getByLabel('设置内容')).toBeVisible();
// The connection list itself, not just the settings shell: `add-connection`
Expand Down
27 changes: 12 additions & 15 deletions apps/desktop/src/main/__tests__/automation-persistence-e2e.test.ts
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
/**
* End-to-end: durable cron persistence + cross-session query/management.
*
* Exercises the REAL host wiring (createMainAutomationWiring) against a REAL
* FileAutomationStore on a real temp workspace, simulating an app restart:
* Exercises the real host wiring against the operational SQLite authority,
* simulating an app restart:
*
* session A creates a durable cron ──sync──► <workspace>/automations.json
* │
* (restart: a fresh wiring loads it) ◄──loadAll────────┘
* │
* session A creates a durable cron ──sync──► runtime.sqlite
* (restart: a fresh wiring loads it) ◄──loadAll───────┘
* session B (never saw it) lists / pauses / resumes / deletes it
*
* This is the query-and-persistence loop the reviewer asked for: a persisted
Expand All@@ -18,10 +16,11 @@

import { strict as assert } from 'node:assert';
import { describe, it, before, after } from 'node:test';
import { mkdtemp, rm, readFile } from 'node:fs/promises';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { MakaToolContext, MakaTool } from '@maka/runtime';
import { createAutomationStore } from '@maka/storage';
import { createMainAutomationWiring } from '../automation-wiring.js';

function ctx(sessionId: string): MakaToolContext {
Expand DownExpand Up@@ -60,15 +59,13 @@ function automationTool(wiring: ReturnType<typeof makeWiring>): MakaTool {
}

async function readStore(workspaceRoot: string): Promise<Array<{ id: string; name: string }>> {
try {
const raw = await readFile(join(workspaceRoot, 'automations.json'), 'utf8');
return (JSON.parse(raw) as { automations: Array<{ id: string; name: string }> }).automations;
} catch {
return [];
}
return (await createAutomationStore(workspaceRoot).loadAll()).map(({ id, name }) => ({
id,
name,
}));
}

/** The store sync is fire-and-forget; poll the file until it settles. */
/** Store sync is fire-and-forget; poll the authority until it settles. */
async function waitForStore(
workspaceRoot: string,
predicate: (rows: Array<{ id: string; name: string }>) => boolean,
Expand DownExpand Up@@ -175,7 +172,7 @@ describe('E2E: durable cron persistence + cross-session query/management', () =>
});

describe('E2E: a cron-disabled host (CLI) sharing the workspace never clobbers durable crons', () => {
it('a heartbeat-only wiring neither loads nor overwrites the owner\'s automations.json', async () => {
it('a heartbeat-only wiring neither loads nor overwrites durable Automations', async () => {
const ws = await mkdtemp(join(tmpdir(), 'maka-automation-clobber-'));
try {
// ── owner (cron-enabled, desktop) creates a durable cron ──────────────
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' refactor(storage): make SQLite the sole operational authority by jackwener · Pull Request #1994 · apache/maka · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 29 additions & 78 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -152,7 +152,7 @@ Start with [ARCHITECTURE.md](./ARCHITECTURE.md). It provides the system map, cod
apps/desktop/ Electron main / preload / React renderer

packages/core/ Pure contracts for Sessions, Events, Permissions, and Connections
packages/storage/ SQLite operational state, legacy importers, and payload stores
packages/storage/ SQLite operational state, configuration, and payload stores
packages/runtime/ AgentRun, model adapters, tools, context, and recovery
packages/headless/ TaskRun, Autonomous Loop, Self-check, eval, and AHE
packages/cli/ TUI and non-interactive CLI
Expand All@@ -168,83 +168,49 @@ Maka stores workspace data under Electron `userData` by default:

```text
<Electron userData>/workspaces/default/
runtime.sqlite
llm-connections.json
credentials.json
settings.json
sessions/
artifacts/
```

Current boundaries that matter:

- Sessionsand connection metadata live in the local filesystem;
- Sessions, messages, execution ledgers, workflows, usage, Automations, Daily Review, and Headless TaskRuns live in `runtime.sqlite`;
- Runtime credentials such as API keys, bot tokens, and proxy passwords currently live in local plaintext `credentials.json`, behind the OS account boundary, with POSIX directory mode `0700` and file mode `0600` enforced;
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless; Electron `safeStorage` only decrypts pre-existing legacy token files once at desktop startup (#1125);
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless. Pre-existing Electron `safeStorage` credential/token files are not imported; affected users must re-authenticate;
- Renderer does not receive plaintext credentials. File writes, Shell, and dangerous tool calls pass through the permission engine;
- Headless real-model evaluation fails closed by default and requires an explicit external isolation boundary.

Read [SECURITY.md](./SECURITY.md) for security reporting and policy, and [docs/README.md](./docs/README.md) for current privacy and sandbox contracts.

## Runtime storage and recovery

RuntimeEvent persistence is always canonical in `runtime.sqlite`. On the first
write, Maka batch-idempotently imports legacy RuntimeEvent JSONL without
rewriting it. Legacy-only workspaces remain available to read-only inspection
until that first write.

Session metadata and Agent Graph control tables now use the same process-local
operational database owner and the same `runtime.sqlite` transaction authority.
The first operational open copies a WAL-consistent `sessions.sqlite` source
into `runtime.sqlite`, validates every source row, and records the source digest
and result in `cutover_journal`. An interrupted copy resumes without partial
rows; a legacy database changed after cutover fails closed. The old database is
retained as migration evidence but is no longer a production writer. Session
transcript bodies remain append-only JSONL.

Core execution state now shares that authority too: AgentRun headers and event
ledgers, event projections, root-turn admissions and source proofs,
Interactions, Host Epoch message receipts, and ShellRun records are canonical
in `runtime.sqlite` across CLI, Desktop, Runtime Host, and Headless. Each legacy
file store is fingerprinted and imported through its own durable
`cutover_journal` entry before the corresponding repository opens. Copy and
validation are one SQLite transaction, retries are idempotent, and a changed
legacy source after cutover fails closed. The legacy files are retained only as
migration evidence; new execution writes do not modify them.

Workflow state is migrating in reviewable slices. Task Ledger events and
projections, Plan events and projections, Deep Research events, and Plan
Reminder records are now canonical in `runtime.sqlite`.
Desktop and Runtime Host production wiring opens these SQLite repositories;
their JSON/JSONL predecessors are read only during a fingerprinted, crash-safe
cutover and are never updated by later mutations.

Usage telemetry and pricing authority now use that same operational database.
Legacy `telemetry.json` and `pricing.json` sources are decoded together and
fingerprinted before their rows and pricing revision are committed atomically.
After cutover, Desktop and Runtime Host write only `runtime.sqlite`; the source
files remain unchanged as migration evidence.

Artifact metadata and lifecycle state now follow the same rule. Payload bytes
remain files, but their records are canonical in `runtime.sqlite` after a
fingerprinted `metadata.jsonl` cutover. Payload publication keeps its durable
staging/link protocol: recovery removes bytes whose metadata transaction did not
commit and preserves committed bytes while cleaning staging residue. Purge
intent recovery likewise completes against the SQLite metadata authority.

Selected-session bundle export now reads that SQLite authority through a
WAL-consistent snapshot, verifies that retained legacy evidence still matches
its completed cutover, and writes only the selected Artifact rows into the
bundle's `runtime.sqlite`. Payload bytes are copied under the Artifact writer
lock, cross-session rows and payloads are excluded, and bundles no longer emit
`artifacts/metadata.jsonl`.

Full operational backup now uses the shared database owner's online SQLite
backup API rather than copying `runtime.sqlite` or its WAL sidecars. A strict
manifest binds the standalone database snapshot to every active session
transcript and canonical Artifact payload by size and SHA-256. Restore verifies
SQLite integrity, foreign keys, supported schema versions, relational identity
sets, transcript decodability, and the exact manifested file tree before
atomically publishing a new state root. Interrupted backup or restore staging
is removed and can be retried without changing either source.
`runtime.sqlite` is the sole operational authority. It owns RuntimeEvents,
session metadata and message history, Agent Graph control, core execution state,
workflow state, usage and pricing, Artifact metadata, Automations, Daily Review,
and Headless TaskRuns. Artifact payload bytes remain regular files under
`artifacts/`; connections, credentials, settings, MCP configuration, skills,
and device identity remain configuration files.

This storage generation does not import earlier File/JSONL authorities. On
upgrade, legacy session titles may still be discoverable through current
metadata, but conversation history that exists only in legacy transcript files
is not copied into `session_messages` and opens as an empty thread. Likewise,
pre-version or `safeStorage`-encrypted credential/token files are not migrated;
users with only those copies must re-authenticate. This data-loss boundary is
intentional for this release and must be considered before upgrading an
existing workspace.

Full operational backup uses the database owner's online SQLite backup API and
copies canonical Artifact payloads under the Artifact writer lock. Its manifest
binds every file by size and SHA-256. Validation checks the standalone SQLite
snapshot's integrity, foreign keys, schema registry and required tables,
decodes canonical session-message and Artifact records, and verifies Artifact
payload sizes against SQLite metadata before restore. Backup and restore use
owner-only file modes, file and directory synchronization, staging, and atomic
publication.

Headless trajectory hydration now consumes a frozen selected-session export
from that SQLite Artifact authority. The cell publishes `trajectory-state`
Expand All@@ -254,21 +220,6 @@ validated snapshot. It does not copy a live WAL or fall back to
`artifacts/metadata.jsonl`. Missing, corrupt, unsupported, or mismatched
evidence fails closed to a summary trajectory instead of mixing authorities.

The remaining storage work is deliberately classified rather than implied
complete:

- Artifact metadata no longer exposes a production JSONL writer;
`artifacts/metadata.jsonl` is accepted only as fingerprinted, read-only
cutover evidence, and can be removed after the migration/cutover matrix is
complete;
- StoredMessage transcript bodies remain append-only JSONL;
- automation, connections, credentials, settings, MCP configuration, skills,
and device identity are configuration state and stay outside this operational
migration;
- Headless TaskRun evaluation ledgers, imported foreign-session caches, Daily
Review archives, and quote-cleanup bookkeeping are separate product/evaluation
domains and are not part of issue #1649.

Runtime continuation remains opt-in:

- `MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1` enables the Desktop interrupted-turn
Expand Down
11 changes: 5 additions & 6 deletions SECURITY.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,9 +97,9 @@ supply their own boundary. See
is the single authority every surface — Desktop, TUI, headless —
reads and writes, under the same OS-account and 0o700/0o600 boundary
as other runtime credentials. Electron safeStorage is not part of
this boundary anymore; Desktop startup imports pre-existing
safeStorage-encrypted token files into the store once and removes
them (#1125).
this boundary anymore. Pre-existing safeStorage-encrypted credential
or token files are not imported; users with only those copies must
re-authenticate.
3. **Renderer process sandbox + preload IPC bridge.** The
renderer cannot reach files, network, or shell directly. Every
IPC handler in `apps/desktop/src/main/main.ts` is the trust
Expand DownExpand Up@@ -152,7 +152,7 @@ are welcome as ordinary issues, not security advisories.
Beyond security, Maka treats the following as user-facing
privacy commitments:

- **Workspace JSONL stays local.** Session messages, tool
- **Workspace state stays local.** Session messages, tool
results, telemetry, settings are stored under
`app.getPath('userData')`. Cloud sync is not shipped.
- **Tool query strings are NEVER logged.** The WebSearch tool's
Expand All@@ -179,8 +179,7 @@ boundary in §2.3 was crossed. Examples:
looser than the 0o700/0o600 boundary.
- A production OAuth path writes or requires a safeStorage-encrypted
token copy again. `credentials.json` is the single documented token
authority; safeStorage exists only inside the one-shot legacy import
(#1125).
authority; there is no safeStorage legacy import fallback.
- A cleartext secret crosses main→renderer IPC under any
circumstance (including error paths, settings preview, IPC
result envelopes, log lines).
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ The Electron desktop app: `main` (Node/Electron main process) + `preload` (conte

Sub-folders: `browser/` (embedded browser view), `oauth/`, `search/` (thread search), `web-search/`, `types/`. The browser IPC handler itself (`browser-ipc-main.ts`) is flat in `src/main/`, not under `browser/`.

`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (credential migration, connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.
`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.

## IPC contract

Expand Down
5 changes: 2 additions & 3 deletions apps/desktop/e2e/fixtures.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -356,9 +356,8 @@ export const test = base.extend<{
);
},
// Stale sessions: boots the e2e-fixture `stale-sessions` fixture — one
// healthy session (zai-live, secret seeded), one unlocked fake session
// (opened active), and one locked legacy session whose connection is
// gone. Exercises the #1038 health-notice authority against real IPC
// healthy session (zai-live, secret seeded) and one locked fake-backend session
// (opened active). Exercises the #1038 health-notice authority against real IPC
// (connection list, hasSecret probe, connectionLocked summaries).
// Readiness = turns on screen: the fake session is open.
staleSessionsWindow: async ({}, use) => {
Expand Down
25 changes: 14 additions & 11 deletions apps/desktop/e2e/session-health-notice.spec.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,14 +3,15 @@
// "will the next send fail?" from the same facts as the send gate
// (connection list, hasSecret probe, connectionLocked on the summary).
//
// The stale-sessions e2e-fixture seeds the exact on-disk states, and
// both stale sessions carry user messages, so storage self-heals them to
// `connectionLocked: true` on first read — the send can neither use
// their connections nor silently rebind, even though a healthy default
// The stale-sessions e2e-fixture seeds the exact on-disk states: one
// locked fake-backend session and one healthy ai-sdk session. The stale
// session carries user messages, so storage self-heals it to
// `connectionLocked: true` on first read — the send can neither use its
// connection nor silently rebind, even though a healthy default
// connection exists. The old "default exists && enabled" proxy hid the
// notice in exactly this state (#1038 case 1); it must now show.
// The silent-rebind counterpart (unlocked empty stale session) is
// covered by the projection and notice unit tests.
// The deleted-connection and legacy-backend notice variants are covered
// by deriveSessionHealthNotice unit tests.

import { test, expect } from './fixtures';

Expand DownExpand Up@@ -39,7 +40,7 @@ async function probeNoticeAlignment(page: import('@playwright/test').Page) {
});
}

test('locked stale sessions show the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
test('a locked stale session shows the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
// Active = stale fake session (locked by its history): notice shows.
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();

Expand All@@ -48,12 +49,14 @@ test('locked stale sessions show the health notice even with a ready default', a
expect(alignment.leftDelta).toBeLessThanOrEqual(1);
expect(alignment.rightDelta).toBeLessThanOrEqual(1);

// Switch to the locked legacy session → its deleted-connection notice.
// A healthy session must not show the notice.
await page.getByRole('button', { name: '展开侧边栏' }).click();
await page.getByText('旧的 Claude 连接会话').first().click();
await expect(page.getByText('连接已删除')).toBeVisible();
await page.getByText('正常会话(Z.ai Live)').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeHidden();

// Click-through lands in Settings · 模型.
// Back on the stale session, click-through lands in Settings · 模型.
await page.getByText('旧的本地模拟会话').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();
await page.getByRole('button', { name: '去模型' }).click();
await expect(page.getByLabel('设置内容')).toBeVisible();
// The connection list itself, not just the settings shell: `add-connection`
Expand Down
27 changes: 12 additions & 15 deletions apps/desktop/src/main/__tests__/automation-persistence-e2e.test.ts
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
/**
* End-to-end: durable cron persistence + cross-session query/management.
*
* Exercises the REAL host wiring (createMainAutomationWiring) against a REAL
* FileAutomationStore on a real temp workspace, simulating an app restart:
* Exercises the real host wiring against the operational SQLite authority,
* simulating an app restart:
*
* session A creates a durable cron ──sync──► <workspace>/automations.json
* │
* (restart: a fresh wiring loads it) ◄──loadAll────────┘
* │
* session A creates a durable cron ──sync──► runtime.sqlite
* (restart: a fresh wiring loads it) ◄──loadAll───────┘
* session B (never saw it) lists / pauses / resumes / deletes it
*
* This is the query-and-persistence loop the reviewer asked for: a persisted
Expand All@@ -18,10 +16,11 @@

import { strict as assert } from 'node:assert';
import { describe, it, before, after } from 'node:test';
import { mkdtemp, rm, readFile } from 'node:fs/promises';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { MakaToolContext, MakaTool } from '@maka/runtime';
import { createAutomationStore } from '@maka/storage';
import { createMainAutomationWiring } from '../automation-wiring.js';

function ctx(sessionId: string): MakaToolContext {
Expand DownExpand Up@@ -60,15 +59,13 @@ function automationTool(wiring: ReturnType<typeof makeWiring>): MakaTool {
}

async function readStore(workspaceRoot: string): Promise<Array<{ id: string; name: string }>> {
try {
const raw = await readFile(join(workspaceRoot, 'automations.json'), 'utf8');
return (JSON.parse(raw) as { automations: Array<{ id: string; name: string }> }).automations;
} catch {
return [];
}
return (await createAutomationStore(workspaceRoot).loadAll()).map(({ id, name }) => ({
id,
name,
}));
}

/** The store sync is fire-and-forget; poll the file until it settles. */
/** Store sync is fire-and-forget; poll the authority until it settles. */
async function waitForStore(
workspaceRoot: string,
predicate: (rows: Array<{ id: string; name: string }>) => boolean,
Expand DownExpand Up@@ -175,7 +172,7 @@ describe('E2E: durable cron persistence + cross-session query/management', () =>
});

describe('E2E: a cron-disabled host (CLI) sharing the workspace never clobbers durable crons', () => {
it('a heartbeat-only wiring neither loads nor overwrites the owner\'s automations.json', async () => {
it('a heartbeat-only wiring neither loads nor overwrites durable Automations', async () => {
const ws = await mkdtemp(join(tmpdir(), 'maka-automation-clobber-'));
try {
// ── owner (cron-enabled, desktop) creates a durable cron ──────────────
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' refactor(storage): make SQLite the sole operational authority by jackwener · Pull Request #1994 · apache/maka · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 29 additions & 78 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -152,7 +152,7 @@ Start with [ARCHITECTURE.md](./ARCHITECTURE.md). It provides the system map, cod
apps/desktop/ Electron main / preload / React renderer

packages/core/ Pure contracts for Sessions, Events, Permissions, and Connections
packages/storage/ SQLite operational state, legacy importers, and payload stores
packages/storage/ SQLite operational state, configuration, and payload stores
packages/runtime/ AgentRun, model adapters, tools, context, and recovery
packages/headless/ TaskRun, Autonomous Loop, Self-check, eval, and AHE
packages/cli/ TUI and non-interactive CLI
Expand All@@ -168,83 +168,49 @@ Maka stores workspace data under Electron `userData` by default:

```text
<Electron userData>/workspaces/default/
runtime.sqlite
llm-connections.json
credentials.json
settings.json
sessions/
artifacts/
```

Current boundaries that matter:

- Sessionsand connection metadata live in the local filesystem;
- Sessions, messages, execution ledgers, workflows, usage, Automations, Daily Review, and Headless TaskRuns live in `runtime.sqlite`;
- Runtime credentials such as API keys, bot tokens, and proxy passwords currently live in local plaintext `credentials.json`, behind the OS account boundary, with POSIX directory mode `0700` and file mode `0600` enforced;
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless; Electron `safeStorage` only decrypts pre-existing legacy token files once at desktop startup (#1125);
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless. Pre-existing Electron `safeStorage` credential/token files are not imported; affected users must re-authenticate;
- Renderer does not receive plaintext credentials. File writes, Shell, and dangerous tool calls pass through the permission engine;
- Headless real-model evaluation fails closed by default and requires an explicit external isolation boundary.

Read [SECURITY.md](./SECURITY.md) for security reporting and policy, and [docs/README.md](./docs/README.md) for current privacy and sandbox contracts.

## Runtime storage and recovery

RuntimeEvent persistence is always canonical in `runtime.sqlite`. On the first
write, Maka batch-idempotently imports legacy RuntimeEvent JSONL without
rewriting it. Legacy-only workspaces remain available to read-only inspection
until that first write.

Session metadata and Agent Graph control tables now use the same process-local
operational database owner and the same `runtime.sqlite` transaction authority.
The first operational open copies a WAL-consistent `sessions.sqlite` source
into `runtime.sqlite`, validates every source row, and records the source digest
and result in `cutover_journal`. An interrupted copy resumes without partial
rows; a legacy database changed after cutover fails closed. The old database is
retained as migration evidence but is no longer a production writer. Session
transcript bodies remain append-only JSONL.

Core execution state now shares that authority too: AgentRun headers and event
ledgers, event projections, root-turn admissions and source proofs,
Interactions, Host Epoch message receipts, and ShellRun records are canonical
in `runtime.sqlite` across CLI, Desktop, Runtime Host, and Headless. Each legacy
file store is fingerprinted and imported through its own durable
`cutover_journal` entry before the corresponding repository opens. Copy and
validation are one SQLite transaction, retries are idempotent, and a changed
legacy source after cutover fails closed. The legacy files are retained only as
migration evidence; new execution writes do not modify them.

Workflow state is migrating in reviewable slices. Task Ledger events and
projections, Plan events and projections, Deep Research events, and Plan
Reminder records are now canonical in `runtime.sqlite`.
Desktop and Runtime Host production wiring opens these SQLite repositories;
their JSON/JSONL predecessors are read only during a fingerprinted, crash-safe
cutover and are never updated by later mutations.

Usage telemetry and pricing authority now use that same operational database.
Legacy `telemetry.json` and `pricing.json` sources are decoded together and
fingerprinted before their rows and pricing revision are committed atomically.
After cutover, Desktop and Runtime Host write only `runtime.sqlite`; the source
files remain unchanged as migration evidence.

Artifact metadata and lifecycle state now follow the same rule. Payload bytes
remain files, but their records are canonical in `runtime.sqlite` after a
fingerprinted `metadata.jsonl` cutover. Payload publication keeps its durable
staging/link protocol: recovery removes bytes whose metadata transaction did not
commit and preserves committed bytes while cleaning staging residue. Purge
intent recovery likewise completes against the SQLite metadata authority.

Selected-session bundle export now reads that SQLite authority through a
WAL-consistent snapshot, verifies that retained legacy evidence still matches
its completed cutover, and writes only the selected Artifact rows into the
bundle's `runtime.sqlite`. Payload bytes are copied under the Artifact writer
lock, cross-session rows and payloads are excluded, and bundles no longer emit
`artifacts/metadata.jsonl`.

Full operational backup now uses the shared database owner's online SQLite
backup API rather than copying `runtime.sqlite` or its WAL sidecars. A strict
manifest binds the standalone database snapshot to every active session
transcript and canonical Artifact payload by size and SHA-256. Restore verifies
SQLite integrity, foreign keys, supported schema versions, relational identity
sets, transcript decodability, and the exact manifested file tree before
atomically publishing a new state root. Interrupted backup or restore staging
is removed and can be retried without changing either source.
`runtime.sqlite` is the sole operational authority. It owns RuntimeEvents,
session metadata and message history, Agent Graph control, core execution state,
workflow state, usage and pricing, Artifact metadata, Automations, Daily Review,
and Headless TaskRuns. Artifact payload bytes remain regular files under
`artifacts/`; connections, credentials, settings, MCP configuration, skills,
and device identity remain configuration files.

This storage generation does not import earlier File/JSONL authorities. On
upgrade, legacy session titles may still be discoverable through current
metadata, but conversation history that exists only in legacy transcript files
is not copied into `session_messages` and opens as an empty thread. Likewise,
pre-version or `safeStorage`-encrypted credential/token files are not migrated;
users with only those copies must re-authenticate. This data-loss boundary is
intentional for this release and must be considered before upgrading an
existing workspace.

Full operational backup uses the database owner's online SQLite backup API and
copies canonical Artifact payloads under the Artifact writer lock. Its manifest
binds every file by size and SHA-256. Validation checks the standalone SQLite
snapshot's integrity, foreign keys, schema registry and required tables,
decodes canonical session-message and Artifact records, and verifies Artifact
payload sizes against SQLite metadata before restore. Backup and restore use
owner-only file modes, file and directory synchronization, staging, and atomic
publication.

Headless trajectory hydration now consumes a frozen selected-session export
from that SQLite Artifact authority. The cell publishes `trajectory-state`
Expand All@@ -254,21 +220,6 @@ validated snapshot. It does not copy a live WAL or fall back to
`artifacts/metadata.jsonl`. Missing, corrupt, unsupported, or mismatched
evidence fails closed to a summary trajectory instead of mixing authorities.

The remaining storage work is deliberately classified rather than implied
complete:

- Artifact metadata no longer exposes a production JSONL writer;
`artifacts/metadata.jsonl` is accepted only as fingerprinted, read-only
cutover evidence, and can be removed after the migration/cutover matrix is
complete;
- StoredMessage transcript bodies remain append-only JSONL;
- automation, connections, credentials, settings, MCP configuration, skills,
and device identity are configuration state and stay outside this operational
migration;
- Headless TaskRun evaluation ledgers, imported foreign-session caches, Daily
Review archives, and quote-cleanup bookkeeping are separate product/evaluation
domains and are not part of issue #1649.

Runtime continuation remains opt-in:

- `MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1` enables the Desktop interrupted-turn
Expand Down
11 changes: 5 additions & 6 deletions SECURITY.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,9 +97,9 @@ supply their own boundary. See
is the single authority every surface — Desktop, TUI, headless —
reads and writes, under the same OS-account and 0o700/0o600 boundary
as other runtime credentials. Electron safeStorage is not part of
this boundary anymore; Desktop startup imports pre-existing
safeStorage-encrypted token files into the store once and removes
them (#1125).
this boundary anymore. Pre-existing safeStorage-encrypted credential
or token files are not imported; users with only those copies must
re-authenticate.
3. **Renderer process sandbox + preload IPC bridge.** The
renderer cannot reach files, network, or shell directly. Every
IPC handler in `apps/desktop/src/main/main.ts` is the trust
Expand DownExpand Up@@ -152,7 +152,7 @@ are welcome as ordinary issues, not security advisories.
Beyond security, Maka treats the following as user-facing
privacy commitments:

- **Workspace JSONL stays local.** Session messages, tool
- **Workspace state stays local.** Session messages, tool
results, telemetry, settings are stored under
`app.getPath('userData')`. Cloud sync is not shipped.
- **Tool query strings are NEVER logged.** The WebSearch tool's
Expand All@@ -179,8 +179,7 @@ boundary in §2.3 was crossed. Examples:
looser than the 0o700/0o600 boundary.
- A production OAuth path writes or requires a safeStorage-encrypted
token copy again. `credentials.json` is the single documented token
authority; safeStorage exists only inside the one-shot legacy import
(#1125).
authority; there is no safeStorage legacy import fallback.
- A cleartext secret crosses main→renderer IPC under any
circumstance (including error paths, settings preview, IPC
result envelopes, log lines).
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ The Electron desktop app: `main` (Node/Electron main process) + `preload` (conte

Sub-folders: `browser/` (embedded browser view), `oauth/`, `search/` (thread search), `web-search/`, `types/`. The browser IPC handler itself (`browser-ipc-main.ts`) is flat in `src/main/`, not under `browser/`.

`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (credential migration, connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.
`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.

## IPC contract

Expand Down
5 changes: 2 additions & 3 deletions apps/desktop/e2e/fixtures.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -356,9 +356,8 @@ export const test = base.extend<{
);
},
// Stale sessions: boots the e2e-fixture `stale-sessions` fixture — one
// healthy session (zai-live, secret seeded), one unlocked fake session
// (opened active), and one locked legacy session whose connection is
// gone. Exercises the #1038 health-notice authority against real IPC
// healthy session (zai-live, secret seeded) and one locked fake-backend session
// (opened active). Exercises the #1038 health-notice authority against real IPC
// (connection list, hasSecret probe, connectionLocked summaries).
// Readiness = turns on screen: the fake session is open.
staleSessionsWindow: async ({}, use) => {
Expand Down
25 changes: 14 additions & 11 deletions apps/desktop/e2e/session-health-notice.spec.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,14 +3,15 @@
// "will the next send fail?" from the same facts as the send gate
// (connection list, hasSecret probe, connectionLocked on the summary).
//
// The stale-sessions e2e-fixture seeds the exact on-disk states, and
// both stale sessions carry user messages, so storage self-heals them to
// `connectionLocked: true` on first read — the send can neither use
// their connections nor silently rebind, even though a healthy default
// The stale-sessions e2e-fixture seeds the exact on-disk states: one
// locked fake-backend session and one healthy ai-sdk session. The stale
// session carries user messages, so storage self-heals it to
// `connectionLocked: true` on first read — the send can neither use its
// connection nor silently rebind, even though a healthy default
// connection exists. The old "default exists && enabled" proxy hid the
// notice in exactly this state (#1038 case 1); it must now show.
// The silent-rebind counterpart (unlocked empty stale session) is
// covered by the projection and notice unit tests.
// The deleted-connection and legacy-backend notice variants are covered
// by deriveSessionHealthNotice unit tests.

import { test, expect } from './fixtures';

Expand DownExpand Up@@ -39,7 +40,7 @@ async function probeNoticeAlignment(page: import('@playwright/test').Page) {
});
}

test('locked stale sessions show the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
test('a locked stale session shows the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
// Active = stale fake session (locked by its history): notice shows.
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();

Expand All@@ -48,12 +49,14 @@ test('locked stale sessions show the health notice even with a ready default', a
expect(alignment.leftDelta).toBeLessThanOrEqual(1);
expect(alignment.rightDelta).toBeLessThanOrEqual(1);

// Switch to the locked legacy session → its deleted-connection notice.
// A healthy session must not show the notice.
await page.getByRole('button', { name: '展开侧边栏' }).click();
await page.getByText('旧的 Claude 连接会话').first().click();
await expect(page.getByText('连接已删除')).toBeVisible();
await page.getByText('正常会话(Z.ai Live)').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeHidden();

// Click-through lands in Settings · 模型.
// Back on the stale session, click-through lands in Settings · 模型.
await page.getByText('旧的本地模拟会话').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();
await page.getByRole('button', { name: '去模型' }).click();
await expect(page.getByLabel('设置内容')).toBeVisible();
// The connection list itself, not just the settings shell: `add-connection`
Expand Down
27 changes: 12 additions & 15 deletions apps/desktop/src/main/__tests__/automation-persistence-e2e.test.ts
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
/**
* End-to-end: durable cron persistence + cross-session query/management.
*
* Exercises the REAL host wiring (createMainAutomationWiring) against a REAL
* FileAutomationStore on a real temp workspace, simulating an app restart:
* Exercises the real host wiring against the operational SQLite authority,
* simulating an app restart:
*
* session A creates a durable cron ──sync──► <workspace>/automations.json
* │
* (restart: a fresh wiring loads it) ◄──loadAll────────┘
* │
* session A creates a durable cron ──sync──► runtime.sqlite
* (restart: a fresh wiring loads it) ◄──loadAll───────┘
* session B (never saw it) lists / pauses / resumes / deletes it
*
* This is the query-and-persistence loop the reviewer asked for: a persisted
Expand All@@ -18,10 +16,11 @@

import { strict as assert } from 'node:assert';
import { describe, it, before, after } from 'node:test';
import { mkdtemp, rm, readFile } from 'node:fs/promises';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { MakaToolContext, MakaTool } from '@maka/runtime';
import { createAutomationStore } from '@maka/storage';
import { createMainAutomationWiring } from '../automation-wiring.js';

function ctx(sessionId: string): MakaToolContext {
Expand DownExpand Up@@ -60,15 +59,13 @@ function automationTool(wiring: ReturnType<typeof makeWiring>): MakaTool {
}

async function readStore(workspaceRoot: string): Promise<Array<{ id: string; name: string }>> {
try {
const raw = await readFile(join(workspaceRoot, 'automations.json'), 'utf8');
return (JSON.parse(raw) as { automations: Array<{ id: string; name: string }> }).automations;
} catch {
return [];
}
return (await createAutomationStore(workspaceRoot).loadAll()).map(({ id, name }) => ({
id,
name,
}));
}

/** The store sync is fire-and-forget; poll the file until it settles. */
/** Store sync is fire-and-forget; poll the authority until it settles. */
async function waitForStore(
workspaceRoot: string,
predicate: (rows: Array<{ id: string; name: string }>) => boolean,
Expand DownExpand Up@@ -175,7 +172,7 @@ describe('E2E: durable cron persistence + cross-session query/management', () =>
});

describe('E2E: a cron-disabled host (CLI) sharing the workspace never clobbers durable crons', () => {
it('a heartbeat-only wiring neither loads nor overwrites the owner\'s automations.json', async () => {
it('a heartbeat-only wiring neither loads nor overwrites durable Automations', async () => {
const ws = await mkdtemp(join(tmpdir(), 'maka-automation-clobber-'));
try {
// ── owner (cron-enabled, desktop) creates a durable cron ──────────────
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); refactor(storage): make SQLite the sole operational authority by jackwener · Pull Request #1994 · apache/maka · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 29 additions & 78 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -152,7 +152,7 @@ Start with [ARCHITECTURE.md](./ARCHITECTURE.md). It provides the system map, cod
apps/desktop/ Electron main / preload / React renderer

packages/core/ Pure contracts for Sessions, Events, Permissions, and Connections
packages/storage/ SQLite operational state, legacy importers, and payload stores
packages/storage/ SQLite operational state, configuration, and payload stores
packages/runtime/ AgentRun, model adapters, tools, context, and recovery
packages/headless/ TaskRun, Autonomous Loop, Self-check, eval, and AHE
packages/cli/ TUI and non-interactive CLI
Expand All@@ -168,83 +168,49 @@ Maka stores workspace data under Electron `userData` by default:

```text
<Electron userData>/workspaces/default/
runtime.sqlite
llm-connections.json
credentials.json
settings.json
sessions/
artifacts/
```

Current boundaries that matter:

- Sessionsand connection metadata live in the local filesystem;
- Sessions, messages, execution ledgers, workflows, usage, Automations, Daily Review, and Headless TaskRuns live in `runtime.sqlite`;
- Runtime credentials such as API keys, bot tokens, and proxy passwords currently live in local plaintext `credentials.json`, behind the OS account boundary, with POSIX directory mode `0700` and file mode `0600` enforced;
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless; Electron `safeStorage` only decrypts pre-existing legacy token files once at desktop startup (#1125);
- Subscription OAuth tokens (Claude, Codex, GitHub Copilot, and the Cursor/Antigravity previews) live in the same `credentials.json` — the single authority for desktop, TUI, and headless. Pre-existing Electron `safeStorage` credential/token files are not imported; affected users must re-authenticate;
- Renderer does not receive plaintext credentials. File writes, Shell, and dangerous tool calls pass through the permission engine;
- Headless real-model evaluation fails closed by default and requires an explicit external isolation boundary.

Read [SECURITY.md](./SECURITY.md) for security reporting and policy, and [docs/README.md](./docs/README.md) for current privacy and sandbox contracts.

## Runtime storage and recovery

RuntimeEvent persistence is always canonical in `runtime.sqlite`. On the first
write, Maka batch-idempotently imports legacy RuntimeEvent JSONL without
rewriting it. Legacy-only workspaces remain available to read-only inspection
until that first write.

Session metadata and Agent Graph control tables now use the same process-local
operational database owner and the same `runtime.sqlite` transaction authority.
The first operational open copies a WAL-consistent `sessions.sqlite` source
into `runtime.sqlite`, validates every source row, and records the source digest
and result in `cutover_journal`. An interrupted copy resumes without partial
rows; a legacy database changed after cutover fails closed. The old database is
retained as migration evidence but is no longer a production writer. Session
transcript bodies remain append-only JSONL.

Core execution state now shares that authority too: AgentRun headers and event
ledgers, event projections, root-turn admissions and source proofs,
Interactions, Host Epoch message receipts, and ShellRun records are canonical
in `runtime.sqlite` across CLI, Desktop, Runtime Host, and Headless. Each legacy
file store is fingerprinted and imported through its own durable
`cutover_journal` entry before the corresponding repository opens. Copy and
validation are one SQLite transaction, retries are idempotent, and a changed
legacy source after cutover fails closed. The legacy files are retained only as
migration evidence; new execution writes do not modify them.

Workflow state is migrating in reviewable slices. Task Ledger events and
projections, Plan events and projections, Deep Research events, and Plan
Reminder records are now canonical in `runtime.sqlite`.
Desktop and Runtime Host production wiring opens these SQLite repositories;
their JSON/JSONL predecessors are read only during a fingerprinted, crash-safe
cutover and are never updated by later mutations.

Usage telemetry and pricing authority now use that same operational database.
Legacy `telemetry.json` and `pricing.json` sources are decoded together and
fingerprinted before their rows and pricing revision are committed atomically.
After cutover, Desktop and Runtime Host write only `runtime.sqlite`; the source
files remain unchanged as migration evidence.

Artifact metadata and lifecycle state now follow the same rule. Payload bytes
remain files, but their records are canonical in `runtime.sqlite` after a
fingerprinted `metadata.jsonl` cutover. Payload publication keeps its durable
staging/link protocol: recovery removes bytes whose metadata transaction did not
commit and preserves committed bytes while cleaning staging residue. Purge
intent recovery likewise completes against the SQLite metadata authority.

Selected-session bundle export now reads that SQLite authority through a
WAL-consistent snapshot, verifies that retained legacy evidence still matches
its completed cutover, and writes only the selected Artifact rows into the
bundle's `runtime.sqlite`. Payload bytes are copied under the Artifact writer
lock, cross-session rows and payloads are excluded, and bundles no longer emit
`artifacts/metadata.jsonl`.

Full operational backup now uses the shared database owner's online SQLite
backup API rather than copying `runtime.sqlite` or its WAL sidecars. A strict
manifest binds the standalone database snapshot to every active session
transcript and canonical Artifact payload by size and SHA-256. Restore verifies
SQLite integrity, foreign keys, supported schema versions, relational identity
sets, transcript decodability, and the exact manifested file tree before
atomically publishing a new state root. Interrupted backup or restore staging
is removed and can be retried without changing either source.
`runtime.sqlite` is the sole operational authority. It owns RuntimeEvents,
session metadata and message history, Agent Graph control, core execution state,
workflow state, usage and pricing, Artifact metadata, Automations, Daily Review,
and Headless TaskRuns. Artifact payload bytes remain regular files under
`artifacts/`; connections, credentials, settings, MCP configuration, skills,
and device identity remain configuration files.

This storage generation does not import earlier File/JSONL authorities. On
upgrade, legacy session titles may still be discoverable through current
metadata, but conversation history that exists only in legacy transcript files
is not copied into `session_messages` and opens as an empty thread. Likewise,
pre-version or `safeStorage`-encrypted credential/token files are not migrated;
users with only those copies must re-authenticate. This data-loss boundary is
intentional for this release and must be considered before upgrading an
existing workspace.

Full operational backup uses the database owner's online SQLite backup API and
copies canonical Artifact payloads under the Artifact writer lock. Its manifest
binds every file by size and SHA-256. Validation checks the standalone SQLite
snapshot's integrity, foreign keys, schema registry and required tables,
decodes canonical session-message and Artifact records, and verifies Artifact
payload sizes against SQLite metadata before restore. Backup and restore use
owner-only file modes, file and directory synchronization, staging, and atomic
publication.

Headless trajectory hydration now consumes a frozen selected-session export
from that SQLite Artifact authority. The cell publishes `trajectory-state`
Expand All@@ -254,21 +220,6 @@ validated snapshot. It does not copy a live WAL or fall back to
`artifacts/metadata.jsonl`. Missing, corrupt, unsupported, or mismatched
evidence fails closed to a summary trajectory instead of mixing authorities.

The remaining storage work is deliberately classified rather than implied
complete:

- Artifact metadata no longer exposes a production JSONL writer;
`artifacts/metadata.jsonl` is accepted only as fingerprinted, read-only
cutover evidence, and can be removed after the migration/cutover matrix is
complete;
- StoredMessage transcript bodies remain append-only JSONL;
- automation, connections, credentials, settings, MCP configuration, skills,
and device identity are configuration state and stay outside this operational
migration;
- Headless TaskRun evaluation ledgers, imported foreign-session caches, Daily
Review archives, and quote-cleanup bookkeeping are separate product/evaluation
domains and are not part of issue #1649.

Runtime continuation remains opt-in:

- `MAKA_RUNTIME_SAFE_BOUNDARY_RESUME=1` enables the Desktop interrupted-turn
Expand Down
11 changes: 5 additions & 6 deletions SECURITY.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,9 +97,9 @@ supply their own boundary. See
is the single authority every surface — Desktop, TUI, headless —
reads and writes, under the same OS-account and 0o700/0o600 boundary
as other runtime credentials. Electron safeStorage is not part of
this boundary anymore; Desktop startup imports pre-existing
safeStorage-encrypted token files into the store once and removes
them (#1125).
this boundary anymore. Pre-existing safeStorage-encrypted credential
or token files are not imported; users with only those copies must
re-authenticate.
3. **Renderer process sandbox + preload IPC bridge.** The
renderer cannot reach files, network, or shell directly. Every
IPC handler in `apps/desktop/src/main/main.ts` is the trust
Expand DownExpand Up@@ -152,7 +152,7 @@ are welcome as ordinary issues, not security advisories.
Beyond security, Maka treats the following as user-facing
privacy commitments:

- **Workspace JSONL stays local.** Session messages, tool
- **Workspace state stays local.** Session messages, tool
results, telemetry, settings are stored under
`app.getPath('userData')`. Cloud sync is not shipped.
- **Tool query strings are NEVER logged.** The WebSearch tool's
Expand All@@ -179,8 +179,7 @@ boundary in §2.3 was crossed. Examples:
looser than the 0o700/0o600 boundary.
- A production OAuth path writes or requires a safeStorage-encrypted
token copy again. `credentials.json` is the single documented token
authority; safeStorage exists only inside the one-shot legacy import
(#1125).
authority; there is no safeStorage legacy import fallback.
- A cleartext secret crosses main→renderer IPC under any
circumstance (including error paths, settings preview, IPC
result envelopes, log lines).
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ The Electron desktop app: `main` (Node/Electron main process) + `preload` (conte

Sub-folders: `browser/` (embedded browser view), `oauth/`, `search/` (thread search), `web-search/`, `types/`. The browser IPC handler itself (`browser-ipc-main.ts`) is flat in `src/main/`, not under `browser/`.

`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (credential migration, connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.
`main.ts` startup order: stores and the runtime/controller are created synchronously at module load; `registerIpc()` runs at top level, **before** `app.whenReady()`; inside `whenReady`, the main window is created **hidden** early and background startup (connection bootstrapping, telemetry, bots, schedulers) runs concurrently without blocking first paint. The window is created hidden and revealed after the renderer's first AppShell paint (the `window:notifyRendererReady` gate in `app.tsx`); a fallback timer reveals it if the renderer never signals, so a fail-soft loading state can show (e.g. if `main.tsx`'s onboarding prefetch times out). The real invariant for IPC: handlers must be registered before the renderer entry runs, because `main.tsx` prefetches the onboarding snapshot before mounting React. Background startup may mutate state after the renderer's first read, so don't assume it has already settled when wiring the UI.

## IPC contract

Expand Down
5 changes: 2 additions & 3 deletions apps/desktop/e2e/fixtures.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -356,9 +356,8 @@ export const test = base.extend<{
);
},
// Stale sessions: boots the e2e-fixture `stale-sessions` fixture — one
// healthy session (zai-live, secret seeded), one unlocked fake session
// (opened active), and one locked legacy session whose connection is
// gone. Exercises the #1038 health-notice authority against real IPC
// healthy session (zai-live, secret seeded) and one locked fake-backend session
// (opened active). Exercises the #1038 health-notice authority against real IPC
// (connection list, hasSecret probe, connectionLocked summaries).
// Readiness = turns on screen: the fake session is open.
staleSessionsWindow: async ({}, use) => {
Expand Down
25 changes: 14 additions & 11 deletions apps/desktop/e2e/session-health-notice.spec.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,14 +3,15 @@
// "will the next send fail?" from the same facts as the send gate
// (connection list, hasSecret probe, connectionLocked on the summary).
//
// The stale-sessions e2e-fixture seeds the exact on-disk states, and
// both stale sessions carry user messages, so storage self-heals them to
// `connectionLocked: true` on first read — the send can neither use
// their connections nor silently rebind, even though a healthy default
// The stale-sessions e2e-fixture seeds the exact on-disk states: one
// locked fake-backend session and one healthy ai-sdk session. The stale
// session carries user messages, so storage self-heals it to
// `connectionLocked: true` on first read — the send can neither use its
// connection nor silently rebind, even though a healthy default
// connection exists. The old "default exists && enabled" proxy hid the
// notice in exactly this state (#1038 case 1); it must now show.
// The silent-rebind counterpart (unlocked empty stale session) is
// covered by the projection and notice unit tests.
// The deleted-connection and legacy-backend notice variants are covered
// by deriveSessionHealthNotice unit tests.

import { test, expect } from './fixtures';

Expand DownExpand Up@@ -39,7 +40,7 @@ async function probeNoticeAlignment(page: import('@playwright/test').Page) {
});
}

test('locked stale sessions show the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
test('a locked stale session shows the health notice even with a ready default', async ({ staleSessionsWindow: page }) => {
// Active = stale fake session (locked by its history): notice shows.
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();

Expand All@@ -48,12 +49,14 @@ test('locked stale sessions show the health notice even with a ready default', a
expect(alignment.leftDelta).toBeLessThanOrEqual(1);
expect(alignment.rightDelta).toBeLessThanOrEqual(1);

// Switch to the locked legacy session → its deleted-connection notice.
// A healthy session must not show the notice.
await page.getByRole('button', { name: '展开侧边栏' }).click();
await page.getByText('旧的 Claude 连接会话').first().click();
await expect(page.getByText('连接已删除')).toBeVisible();
await page.getByText('正常会话(Z.ai Live)').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeHidden();

// Click-through lands in Settings · 模型.
// Back on the stale session, click-through lands in Settings · 模型.
await page.getByText('旧的本地模拟会话').first().click();
await expect(page.getByText('会话已过期 · 请先配置真实模型')).toBeVisible();
await page.getByRole('button', { name: '去模型' }).click();
await expect(page.getByLabel('设置内容')).toBeVisible();
// The connection list itself, not just the settings shell: `add-connection`
Expand Down
27 changes: 12 additions & 15 deletions apps/desktop/src/main/__tests__/automation-persistence-e2e.test.ts
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
/**
* End-to-end: durable cron persistence + cross-session query/management.
*
* Exercises the REAL host wiring (createMainAutomationWiring) against a REAL
* FileAutomationStore on a real temp workspace, simulating an app restart:
* Exercises the real host wiring against the operational SQLite authority,
* simulating an app restart:
*
* session A creates a durable cron ──sync──► <workspace>/automations.json
* │
* (restart: a fresh wiring loads it) ◄──loadAll────────┘
* │
* session A creates a durable cron ──sync──► runtime.sqlite
* (restart: a fresh wiring loads it) ◄──loadAll───────┘
* session B (never saw it) lists / pauses / resumes / deletes it
*
* This is the query-and-persistence loop the reviewer asked for: a persisted
Expand All@@ -18,10 +16,11 @@

import { strict as assert } from 'node:assert';
import { describe, it, before, after } from 'node:test';
import { mkdtemp, rm, readFile } from 'node:fs/promises';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { MakaToolContext, MakaTool } from '@maka/runtime';
import { createAutomationStore } from '@maka/storage';
import { createMainAutomationWiring } from '../automation-wiring.js';

function ctx(sessionId: string): MakaToolContext {
Expand DownExpand Up@@ -60,15 +59,13 @@ function automationTool(wiring: ReturnType<typeof makeWiring>): MakaTool {
}

async function readStore(workspaceRoot: string): Promise<Array<{ id: string; name: string }>> {
try {
const raw = await readFile(join(workspaceRoot, 'automations.json'), 'utf8');
return (JSON.parse(raw) as { automations: Array<{ id: string; name: string }> }).automations;
} catch {
return [];
}
return (await createAutomationStore(workspaceRoot).loadAll()).map(({ id, name }) => ({
id,
name,
}));
}

/** The store sync is fire-and-forget; poll the file until it settles. */
/** Store sync is fire-and-forget; poll the authority until it settles. */
async function waitForStore(
workspaceRoot: string,
predicate: (rows: Array<{ id: string; name: string }>) => boolean,
Expand DownExpand Up@@ -175,7 +172,7 @@ describe('E2E: durable cron persistence + cross-session query/management', () =>
});

describe('E2E: a cron-disabled host (CLI) sharing the workspace never clobbers durable crons', () => {
it('a heartbeat-only wiring neither loads nor overwrites the owner\'s automations.json', async () => {
it('a heartbeat-only wiring neither loads nor overwrites durable Automations', async () => {
const ws = await mkdtemp(join(tmpdir(), 'maka-automation-clobber-'));
try {
// ── owner (cron-enabled, desktop) creates a durable cron ──────────────
Expand Down
Loading
Loading