feat(browser): embedded browser automation (opencli CDP observe→act)

[Astro-Han]

Summary

Embedded browser automation for the desktop agent — ported from PawWork's opencli CDP + numbered-ref observe→act approach. The agent drives a real, per-conversation Chromium view through 6 generic tools, and the page renders live in a right-side panel.

• Sealed CDP bridge (cdp-bridge.ts): wraps webContents.debugger behind a loopback WebSocket (ws://127.0.0.1:<random-port>/<secret>, secret kept in main-process memory only). opencli's CDPBridge client connects to it; its stealth script auto-registers on connect and applies to future documents.
• 6 generic observe→act tools (registered unconditionally, no flag): browser_navigate / snapshot / click / type / wait / extract. Loop: numbered [ref] snapshot (observe) → act by ref (click/type).
• Per-conversation views: each conversation owns a WebContentsView child of the main window — lazily created, stacked, all hidden except the shown one. Views are in-memory (ephemeral across app restart, by design); a shared persistent partition (persist:maka-browser) keeps cookies/login on disk so a login in one conversation is available to all and survives restart.
• Dedicated browser permission category — prompt-on-effect: blocked in explore, prompts in ask and execute (browser effects are treated as irreversible, not auto-allowed like read tools). Takeover-reload is deferred to first effect: observe never reloads, the first mutate reloads once to apply stealth to an already-open page, navigate clears without reloading.
• Visible-conversation lease: the agent touches the browser only for the conversation currently on screen. EVERY action — snapshot / extract / wait (read), navigate, click / type (act) — is rejected when the calling conversation is backgrounded, so a background conversation can't even read a logged-in page the user can't see; a mutate additionally requires a real, non-empty on-screen viewport (opencli's native CDP click hit-tests a composited frame a hidden view lacks). The check runs before the view/connection is acquired, so a vetoed background action creates neither. The lease is continuous, not just a preflight: an action already running when the user switches away is revoked and its connection severed, so a long wait/navigate/extract/delayed mutate can never keep reading or driving a now-hidden page. Because the permission modal hides the native view while it is open, a mutate on the on-screen conversation whose viewport is momentarily absent (the modal just closed) waits briefly for the renderer to restore the strip rather than rejecting — so the first approved click/type lands without a retry. The user always sees the page the agent is acting on — the visible view plus the per-turn permission prompt is the safety net, now enforced rather than assumed.
• Renderer panel (browser-panel.tsx): address bar + nav controls (lucide icons, matching the app's icon set). The panel reserves a strip and mirrors its on-screen rect to main each animation frame, so the native view tracks the strip on resize / sidebar drags. The page is a native view floating above the DOM, not a React child.

Deliberate scope

• Browser views are ephemeral across app restart (login persists, the live page does not) — the simplest split; the agent re-opens on demand. Persisting the live URL across restarts was intentionally skipped.
• Multi-session parallel browsing is out of scope for now (YAGNI), tracked as a follow-up. The browser is single-window: one view is drawn at a time, and the visible-lease rejects mutations on an off-screen conversation. The blocker is the presentation layer, not state management — the runtime already runs sessions concurrently (SessionManager keys active sessions by id) and each conversation already owns its own view + CDP connection + history. A hidden embedded WebContentsView simply can't be driven for clicks: tested directly, a non-displayed view still does snapshot / type / navigate, but a native click silently no-ops because it never composites a frame to hit-test — true even with PawWork's exact 1280×720 default bound, parented or not. PawWork (where this was ported from) gets parallelism by being multi-window (each conversation's view displayed in its own visible window), not by hidden rendering. A follow-up would pick a presentation change — multi-window (à la PawWork), single-window split-pane, or offscreen rendering — and the current lease is just the single-window special case of "a view must be displayed somewhere to be driven", which generalizes when that lands.
• No "open browser from UI" button yet — the only cold entry is the agent (a navigate creates the conversation's panel). The address bar handles manual navigation once a page is open, but it lives inside the panel, which mounts only for a live view — so it is not itself a cold-start entry. A small open-browser affordance is deferred to a follow-up PR.
• No browser_screenshot yet — the app feeds no images to the model at all (attachments are stringified into the prompt), so a screenshot tool would only ever return a byte count the model can't use. Deferred to the phase-2 multimodal PR, which adds image-to-model support and rebuilds the tool against it.
• Cut after review. Two independent reviewers (Codex + a fresh-eye pass, Occam's-razor focus) drove a cleanup commit that removed the dead tool/host surface: the screenshot stub, speculative snapshot knobs, a duplicated URL validator, an unwired probe abstraction, and a few zero-caller exports (−157 lines, no capability lost).
• Hardened after a second review round. Added the visible-conversation lease above — the safety gap the review flagged (the agent could otherwise act on a hidden, backgrounded view). Plus two small fixes: viewportBounds now rejects non-finite rects from the untyped IPC boundary before they reach setBounds, and the shared-partition security backstop (will-download + permission handlers) installs once per session instead of once per view (no listener pile-up across conversations).
• Tightened after a third review round. The browser permission is now a single explicit contract: browser gets its own prompt reason (the dialog names the logged-in session it drives, not a generic “custom” request) and one turn-wide permission scope, so “allow for this turn” actually carries the whole observe→act loop instead of re-prompting on every ref. Plus: browser_extract recovers from an invalid CSS selector (a [12] ref a model echoes) as a clean “no match” instead of a raw DOMException, and @jackwener/opencli is pinned to exact 1.8.4 to match the contract test’s “pinned release” assertion.
• Tightened after a fourth review round. Two safety gaps closed by one model — the agent touches the browser only for the conversation you're watching, and only after you approve it for the turn: (1) the first approved click/type after a permission grant no longer loses the race against the renderer's viewport restore (the modal hides the native view; the mutate now waits briefly for the strip to come back rather than rejecting); (2) the visible lease now gates reads too, so a backgrounded conversation can't snapshot/extract a logged-in page off screen; and (3) the prompt is honest that one browser grant covers the whole turn's reads, navigation, clicks, and typing (rather than splitting into two prompts — the live visible view is the act-phase safety net).
• Tightened after a fifth review round. The whole-turn browser note no longer overstates: it renders only when "本轮记住" is checked (the runtime persists the grant only on allow && rememberForTurn — locked by the rememberForTurn=false does NOT add to set test), and it drops the inaccurate "switching conversations revokes the grant" line. endTurn fires on run completion/abort (closing a conversation aborts its turn), never on a plain switch — a switch just parks the action behind the visible lease and resumes it on return without re-asking. The note now states the grant expires when the turn ends.
• Tightened after a sixth review round. The visible lease is now CONTINUOUS, not a one-time preflight: canDrive() only gated the start, so a browser_wait / navigate / extract / delayed mutate that began while shown kept running after the user switched away. withBrowserPage registers each in-flight action; main's browser:active-session handler calls revokeHiddenBrowserActions(shown) on every switch, severing any action whose conversation just went off screen (same connection-sever path as a timeout/abort) and rejecting with a new BrowserActionRevokedError — so no tool result can carry hidden-page data. And background throttling is now scoped to shown-ness instead of held off for the whole cached connection's life: a hidden conversation's page throttles normally (no off-screen CPU drain), while a shown view stays full-speed so native CDP clicks composite (deleting the override outright was tried and rejected — the smoke proves it's load-bearing for clicks on any view the OS backgrounds, e.g. the app being unfocused). Covered by two new session unit tests + three new smoke checks (live-bridge revoke, both throttle transitions); smoke now 19/19.
• Chat-header badges were hardened so the narrower chat column (browser panel open) no longer collapses short status pills (e.g. 运行中) into vertical text; only the tab title and model label absorb the squeeze.

Verification

Freshly re-run on this branch:

• npm run typecheck — clean across core / storage / runtime / ui / desktop.
• Unit tests:
  • core 618 pass / 0 fail
  • storage 76 pass / 0 fail
  • desktop 1469 pass / 0 fail
  • runtime 396 pass / 1 fail — pre-existing & environment-dependent, not from this PR (which touches no runtime/network code). The failing test network/proxy-test "times out when the proxy accepts TCP but never responds" asserts the error text matches /timeout/i, but in this sandbox fetch to the target returns "fetch failed" before the 100 ms timeout fires — a machine-dependence the test's own comment calls out.

Validated on this branch (CDP path unchanged by the later UI commits):

• npm run smoke:browser — 19/19: real Electron observe→act E2E (sealed ws bridge ↔ webContents.debugger ↔ opencli ↔ live DOM): goto → numbered snapshot → fill-by-ref → click-by-ref → DOM-effect verify → markdown extract, plus a check that the partition security backstop installs once across views, plus the visible-lease driven through the real host/manager/BrowserSession end to end (background-conversation read/navigate/mutate all rejected with no view or connection created; a mutate on the shown conversation waits out a modal-close viewport restore and lands; click/type land once the conversation is shown with a viewport). New this round: an in-flight read is revoked against the live bridge when its conversation goes off screen, and a shown view runs un-throttled while hiding it restores background throttling.
• Manual GUI smoke on real sites: panel render, address bar + nav, agent observe→act, permission prompts (explore/ask/execute), per-conversation view isolation, shared login partition, stealth probe, archive/quit teardown.

[Astro-Han]

Summary

Embedded browser automation for the desktop agent — ported from PawWork's opencli CDP + numbered-ref observe→act approach. The agent drives a real, per-conversation Chromium view through 6 generic tools, and the page renders live in a right-side panel.

• Sealed CDP bridge (cdp-bridge.ts): wraps webContents.debugger behind a loopback WebSocket (ws://127.0.0.1:<random-port>/<secret>, secret kept in main-process memory only). opencli's CDPBridge client connects to it; its stealth script auto-registers on connect and applies to future documents.
• 6 generic observe→act tools (registered unconditionally, no flag): browser_navigate / snapshot / click / type / wait / extract. Loop: numbered [ref] snapshot (observe) → act by ref (click/type).
• Per-conversation views: each conversation owns a WebContentsView child of the main window — lazily created, stacked, all hidden except the shown one. Views are in-memory (ephemeral across app restart, by design); a shared persistent partition (persist:maka-browser) keeps cookies/login on disk so a login in one conversation is available to all and survives restart.
• Dedicated browser permission category — prompt-on-effect: blocked in explore, prompts in ask and execute (browser effects are treated as irreversible, not auto-allowed like read tools). Takeover-reload is deferred to first effect: observe never reloads, the first mutate reloads once to apply stealth to an already-open page, navigate clears without reloading.
• Visible-conversation lease: the agent touches the browser only for the conversation currently on screen. EVERY action — snapshot / extract / wait (read), navigate, click / type (act) — is rejected when the calling conversation is backgrounded, so a background conversation can't even read a logged-in page the user can't see; a mutate additionally requires a real, non-empty on-screen viewport (opencli's native CDP click hit-tests a composited frame a hidden view lacks). The check runs before the view/connection is acquired, so a vetoed background action creates neither. The lease is continuous, not just a preflight: an action already running when the user switches away is revoked and its connection severed, so a long wait/navigate/extract/delayed mutate can never keep reading or driving a now-hidden page. Because the permission modal hides the native view while it is open, a mutate on the on-screen conversation whose viewport is momentarily absent (the modal just closed) waits briefly for the renderer to restore the strip rather than rejecting — so the first approved click/type lands without a retry. The user always sees the page the agent is acting on — the visible view plus the per-turn permission prompt is the safety net, now enforced rather than assumed.
• Renderer panel (browser-panel.tsx): address bar + nav controls (lucide icons, matching the app's icon set). The panel reserves a strip and mirrors its on-screen rect to main each animation frame, so the native view tracks the strip on resize / sidebar drags. The page is a native view floating above the DOM, not a React child.

Deliberate scope

• Browser views are ephemeral across app restart (login persists, the live page does not) — the simplest split; the agent re-opens on demand. Persisting the live URL across restarts was intentionally skipped.
• Multi-session parallel browsing is out of scope for now (YAGNI), tracked as a follow-up. The browser is single-window: one view is drawn at a time, and the visible-lease rejects mutations on an off-screen conversation. The blocker is the presentation layer, not state management — the runtime already runs sessions concurrently (SessionManager keys active sessions by id) and each conversation already owns its own view + CDP connection + history. A hidden embedded WebContentsView simply can't be driven for clicks: tested directly, a non-displayed view still does snapshot / type / navigate, but a native click silently no-ops because it never composites a frame to hit-test — true even with PawWork's exact 1280×720 default bound, parented or not. PawWork (where this was ported from) gets parallelism by being multi-window (each conversation's view displayed in its own visible window), not by hidden rendering. A follow-up would pick a presentation change — multi-window (à la PawWork), single-window split-pane, or offscreen rendering — and the current lease is just the single-window special case of "a view must be displayed somewhere to be driven", which generalizes when that lands.
• No "open browser from UI" button yet — the only cold entry is the agent (a navigate creates the conversation's panel). The address bar handles manual navigation once a page is open, but it lives inside the panel, which mounts only for a live view — so it is not itself a cold-start entry. A small open-browser affordance is deferred to a follow-up PR.
• No browser_screenshot yet — the app feeds no images to the model at all (attachments are stringified into the prompt), so a screenshot tool would only ever return a byte count the model can't use. Deferred to the phase-2 multimodal PR, which adds image-to-model support and rebuilds the tool against it.
• Cut after review. Two independent reviewers (Codex + a fresh-eye pass, Occam's-razor focus) drove a cleanup commit that removed the dead tool/host surface: the screenshot stub, speculative snapshot knobs, a duplicated URL validator, an unwired probe abstraction, and a few zero-caller exports (−157 lines, no capability lost).
• Hardened after a second review round. Added the visible-conversation lease above — the safety gap the review flagged (the agent could otherwise act on a hidden, backgrounded view). Plus two small fixes: viewportBounds now rejects non-finite rects from the untyped IPC boundary before they reach setBounds, and the shared-partition security backstop (will-download + permission handlers) installs once per session instead of once per view (no listener pile-up across conversations).
• Tightened after a third review round. The browser permission is now a single explicit contract: browser gets its own prompt reason (the dialog names the logged-in session it drives, not a generic “custom” request) and one turn-wide permission scope, so “allow for this turn” actually carries the whole observe→act loop instead of re-prompting on every ref. Plus: browser_extract recovers from an invalid CSS selector (a [12] ref a model echoes) as a clean “no match” instead of a raw DOMException, and @jackwener/opencli is pinned to exact 1.8.4 to match the contract test’s “pinned release” assertion.
• Tightened after a fourth review round. Two safety gaps closed by one model — the agent touches the browser only for the conversation you're watching, and only after you approve it for the turn: (1) the first approved click/type after a permission grant no longer loses the race against the renderer's viewport restore (the modal hides the native view; the mutate now waits briefly for the strip to come back rather than rejecting); (2) the visible lease now gates reads too, so a backgrounded conversation can't snapshot/extract a logged-in page off screen; and (3) the prompt is honest that one browser grant covers the whole turn's reads, navigation, clicks, and typing (rather than splitting into two prompts — the live visible view is the act-phase safety net).
• Tightened after a fifth review round. The whole-turn browser note no longer overstates: it renders only when "本轮记住" is checked (the runtime persists the grant only on allow && rememberForTurn — locked by the rememberForTurn=false does NOT add to set test), and it drops the inaccurate "switching conversations revokes the grant" line. endTurn fires on run completion/abort (closing a conversation aborts its turn), never on a plain switch — a switch just parks the action behind the visible lease and resumes it on return without re-asking. The note now states the grant expires when the turn ends.
• Tightened after a sixth review round. The visible lease is now CONTINUOUS, not a one-time preflight: canDrive() only gated the start, so a browser_wait / navigate / extract / delayed mutate that began while shown kept running after the user switched away. withBrowserPage registers each in-flight action; main's browser:active-session handler calls revokeHiddenBrowserActions(shown) on every switch, severing any action whose conversation just went off screen (same connection-sever path as a timeout/abort) and rejecting with a new BrowserActionRevokedError — so no tool result can carry hidden-page data. And background throttling is now scoped to shown-ness instead of held off for the whole cached connection's life: a hidden conversation's page throttles normally (no off-screen CPU drain), while a shown view stays full-speed so native CDP clicks composite (deleting the override outright was tried and rejected — the smoke proves it's load-bearing for clicks on any view the OS backgrounds, e.g. the app being unfocused). Covered by two new session unit tests + three new smoke checks (live-bridge revoke, both throttle transitions); smoke now 19/19.
• Chat-header badges were hardened so the narrower chat column (browser panel open) no longer collapses short status pills (e.g. 运行中) into vertical text; only the tab title and model label absorb the squeeze.

Verification

Freshly re-run on this branch:

• npm run typecheck — clean across core / storage / runtime / ui / desktop.
• Unit tests:
  • core 618 pass / 0 fail
  • storage 76 pass / 0 fail
  • desktop 1469 pass / 0 fail
  • runtime 396 pass / 1 fail — pre-existing & environment-dependent, not from this PR (which touches no runtime/network code). The failing test network/proxy-test "times out when the proxy accepts TCP but never responds" asserts the error text matches /timeout/i, but in this sandbox fetch to the target returns "fetch failed" before the 100 ms timeout fires — a machine-dependence the test's own comment calls out.

Validated on this branch (CDP path unchanged by the later UI commits):

• npm run smoke:browser — 19/19: real Electron observe→act E2E (sealed ws bridge ↔ webContents.debugger ↔ opencli ↔ live DOM): goto → numbered snapshot → fill-by-ref → click-by-ref → DOM-effect verify → markdown extract, plus a check that the partition security backstop installs once across views, plus the visible-lease driven through the real host/manager/BrowserSession end to end (background-conversation read/navigate/mutate all rejected with no view or connection created; a mutate on the shown conversation waits out a modal-close viewport restore and lands; click/type land once the conversation is shown with a viewport). New this round: an in-flight read is revoked against the live bridge when its conversation goes off screen, and a shown view runs un-throttled while hiding it restores background throttling.
• Manual GUI smoke on real sites: panel render, address bar + nav, agent observe→act, permission prompts (explore/ask/execute), per-conversation view isolation, shared login partition, stealth probe, archive/quit teardown.

[Astro-Han]

Summary

Embedded browser automation for the desktop agent — ported from PawWork's opencli CDP + numbered-ref observe→act approach. The agent drives a real, per-conversation Chromium view through 6 generic tools, and the page renders live in a right-side panel.

• Sealed CDP bridge (cdp-bridge.ts): wraps webContents.debugger behind a loopback WebSocket (ws://127.0.0.1:<random-port>/<secret>, secret kept in main-process memory only). opencli's CDPBridge client connects to it; its stealth script auto-registers on connect and applies to future documents.
• 6 generic observe→act tools (registered unconditionally, no flag): browser_navigate / snapshot / click / type / wait / extract. Loop: numbered [ref] snapshot (observe) → act by ref (click/type).
• Per-conversation views: each conversation owns a WebContentsView child of the main window — lazily created, stacked, all hidden except the shown one. Views are in-memory (ephemeral across app restart, by design); a shared persistent partition (persist:maka-browser) keeps cookies/login on disk so a login in one conversation is available to all and survives restart.
• Dedicated browser permission category — prompt-on-effect: blocked in explore, prompts in ask and execute (browser effects are treated as irreversible, not auto-allowed like read tools). Takeover-reload is deferred to first effect: observe never reloads, the first mutate reloads once to apply stealth to an already-open page, navigate clears without reloading.
• Visible-conversation lease: the agent touches the browser only for the conversation currently on screen. EVERY action — snapshot / extract / wait (read), navigate, click / type (act) — is rejected when the calling conversation is backgrounded, so a background conversation can't even read a logged-in page the user can't see; a mutate additionally requires a real, non-empty on-screen viewport (opencli's native CDP click hit-tests a composited frame a hidden view lacks). The check runs before the view/connection is acquired, so a vetoed background action creates neither. The lease is continuous, not just a preflight: an action already running when the user switches away is revoked and its connection severed, so a long wait/navigate/extract/delayed mutate can never keep reading or driving a now-hidden page. Because the permission modal hides the native view while it is open, a mutate on the on-screen conversation whose viewport is momentarily absent (the modal just closed) waits briefly for the renderer to restore the strip rather than rejecting — so the first approved click/type lands without a retry. The user always sees the page the agent is acting on — the visible view plus the per-turn permission prompt is the safety net, now enforced rather than assumed.
• Renderer panel (browser-panel.tsx): address bar + nav controls (lucide icons, matching the app's icon set). The panel reserves a strip and mirrors its on-screen rect to main each animation frame, so the native view tracks the strip on resize / sidebar drags. The page is a native view floating above the DOM, not a React child.

Deliberate scope

• Browser views are ephemeral across app restart (login persists, the live page does not) — the simplest split; the agent re-opens on demand. Persisting the live URL across restarts was intentionally skipped.
• Multi-session parallel browsing is out of scope for now (YAGNI), tracked as a follow-up. The browser is single-window: one view is drawn at a time, and the visible-lease rejects mutations on an off-screen conversation. The blocker is the presentation layer, not state management — the runtime already runs sessions concurrently (SessionManager keys active sessions by id) and each conversation already owns its own view + CDP connection + history. A hidden embedded WebContentsView simply can't be driven for clicks: tested directly, a non-displayed view still does snapshot / type / navigate, but a native click silently no-ops because it never composites a frame to hit-test — true even with PawWork's exact 1280×720 default bound, parented or not. PawWork (where this was ported from) gets parallelism by being multi-window (each conversation's view displayed in its own visible window), not by hidden rendering. A follow-up would pick a presentation change — multi-window (à la PawWork), single-window split-pane, or offscreen rendering — and the current lease is just the single-window special case of "a view must be displayed somewhere to be driven", which generalizes when that lands.
• No "open browser from UI" button yet — the only cold entry is the agent (a navigate creates the conversation's panel). The address bar handles manual navigation once a page is open, but it lives inside the panel, which mounts only for a live view — so it is not itself a cold-start entry. A small open-browser affordance is deferred to a follow-up PR.
• No browser_screenshot yet — the app feeds no images to the model at all (attachments are stringified into the prompt), so a screenshot tool would only ever return a byte count the model can't use. Deferred to the phase-2 multimodal PR, which adds image-to-model support and rebuilds the tool against it.
• Cut after review. Two independent reviewers (Codex + a fresh-eye pass, Occam's-razor focus) drove a cleanup commit that removed the dead tool/host surface: the screenshot stub, speculative snapshot knobs, a duplicated URL validator, an unwired probe abstraction, and a few zero-caller exports (−157 lines, no capability lost).
• Hardened after a second review round. Added the visible-conversation lease above — the safety gap the review flagged (the agent could otherwise act on a hidden, backgrounded view). Plus two small fixes: viewportBounds now rejects non-finite rects from the untyped IPC boundary before they reach setBounds, and the shared-partition security backstop (will-download + permission handlers) installs once per session instead of once per view (no listener pile-up across conversations).
• Tightened after a third review round. The browser permission is now a single explicit contract: browser gets its own prompt reason (the dialog names the logged-in session it drives, not a generic “custom” request) and one turn-wide permission scope, so “allow for this turn” actually carries the whole observe→act loop instead of re-prompting on every ref. Plus: browser_extract recovers from an invalid CSS selector (a [12] ref a model echoes) as a clean “no match” instead of a raw DOMException, and @jackwener/opencli is pinned to exact 1.8.4 to match the contract test’s “pinned release” assertion.
• Tightened after a fourth review round. Two safety gaps closed by one model — the agent touches the browser only for the conversation you're watching, and only after you approve it for the turn: (1) the first approved click/type after a permission grant no longer loses the race against the renderer's viewport restore (the modal hides the native view; the mutate now waits briefly for the strip to come back rather than rejecting); (2) the visible lease now gates reads too, so a backgrounded conversation can't snapshot/extract a logged-in page off screen; and (3) the prompt is honest that one browser grant covers the whole turn's reads, navigation, clicks, and typing (rather than splitting into two prompts — the live visible view is the act-phase safety net).
• Tightened after a fifth review round. The whole-turn browser note no longer overstates: it renders only when "本轮记住" is checked (the runtime persists the grant only on allow && rememberForTurn — locked by the rememberForTurn=false does NOT add to set test), and it drops the inaccurate "switching conversations revokes the grant" line. endTurn fires on run completion/abort (closing a conversation aborts its turn), never on a plain switch — a switch just parks the action behind the visible lease and resumes it on return without re-asking. The note now states the grant expires when the turn ends.
• Tightened after a sixth review round. The visible lease is now CONTINUOUS, not a one-time preflight: canDrive() only gated the start, so a browser_wait / navigate / extract / delayed mutate that began while shown kept running after the user switched away. withBrowserPage registers each in-flight action; main's browser:active-session handler calls revokeHiddenBrowserActions(shown) on every switch, severing any action whose conversation just went off screen (same connection-sever path as a timeout/abort) and rejecting with a new BrowserActionRevokedError — so no tool result can carry hidden-page data. And background throttling is now scoped to shown-ness instead of held off for the whole cached connection's life: a hidden conversation's page throttles normally (no off-screen CPU drain), while a shown view stays full-speed so native CDP clicks composite (deleting the override outright was tried and rejected — the smoke proves it's load-bearing for clicks on any view the OS backgrounds, e.g. the app being unfocused). Covered by two new session unit tests + three new smoke checks (live-bridge revoke, both throttle transitions); smoke now 19/19.
• Chat-header badges were hardened so the narrower chat column (browser panel open) no longer collapses short status pills (e.g. 运行中) into vertical text; only the tab title and model label absorb the squeeze.

Verification

Freshly re-run on this branch:

• npm run typecheck — clean across core / storage / runtime / ui / desktop.
• Unit tests:
  • core 618 pass / 0 fail
  • storage 76 pass / 0 fail
  • desktop 1469 pass / 0 fail
  • runtime 396 pass / 1 fail — pre-existing & environment-dependent, not from this PR (which touches no runtime/network code). The failing test network/proxy-test "times out when the proxy accepts TCP but never responds" asserts the error text matches /timeout/i, but in this sandbox fetch to the target returns "fetch failed" before the 100 ms timeout fires — a machine-dependence the test's own comment calls out.

Validated on this branch (CDP path unchanged by the later UI commits):

• npm run smoke:browser — 19/19: real Electron observe→act E2E (sealed ws bridge ↔ webContents.debugger ↔ opencli ↔ live DOM): goto → numbered snapshot → fill-by-ref → click-by-ref → DOM-effect verify → markdown extract, plus a check that the partition security backstop installs once across views, plus the visible-lease driven through the real host/manager/BrowserSession end to end (background-conversation read/navigate/mutate all rejected with no view or connection created; a mutate on the shown conversation waits out a modal-close viewport restore and lands; click/type land once the conversation is shown with a viewport). New this round: an in-flight read is revoked against the live bridge when its conversation goes off screen, and a shown view runs un-throttled while hiding it restores background throttling.
• Manual GUI smoke on real sites: panel render, address bar + nav, agent observe→act, permission prompts (explore/ask/execute), per-conversation view isolation, shared login partition, stealth probe, archive/quit teardown.

[Astro-Han]

Summary

Embedded browser automation for the desktop agent — ported from PawWork's opencli CDP + numbered-ref observe→act approach. The agent drives a real, per-conversation Chromium view through 6 generic tools, and the page renders live in a right-side panel.

• Sealed CDP bridge (cdp-bridge.ts): wraps webContents.debugger behind a loopback WebSocket (ws://127.0.0.1:<random-port>/<secret>, secret kept in main-process memory only). opencli's CDPBridge client connects to it; its stealth script auto-registers on connect and applies to future documents.
• 6 generic observe→act tools (registered unconditionally, no flag): browser_navigate / snapshot / click / type / wait / extract. Loop: numbered [ref] snapshot (observe) → act by ref (click/type).
• Per-conversation views: each conversation owns a WebContentsView child of the main window — lazily created, stacked, all hidden except the shown one. Views are in-memory (ephemeral across app restart, by design); a shared persistent partition (persist:maka-browser) keeps cookies/login on disk so a login in one conversation is available to all and survives restart.
• Dedicated browser permission category — prompt-on-effect: blocked in explore, prompts in ask and execute (browser effects are treated as irreversible, not auto-allowed like read tools). Takeover-reload is deferred to first effect: observe never reloads, the first mutate reloads once to apply stealth to an already-open page, navigate clears without reloading.
• Visible-conversation lease: the agent touches the browser only for the conversation currently on screen. EVERY action — snapshot / extract / wait (read), navigate, click / type (act) — is rejected when the calling conversation is backgrounded, so a background conversation can't even read a logged-in page the user can't see; a mutate additionally requires a real, non-empty on-screen viewport (opencli's native CDP click hit-tests a composited frame a hidden view lacks). The check runs before the view/connection is acquired, so a vetoed background action creates neither. The lease is continuous, not just a preflight: an action already running when the user switches away is revoked and its connection severed, so a long wait/navigate/extract/delayed mutate can never keep reading or driving a now-hidden page. Because the permission modal hides the native view while it is open, a mutate on the on-screen conversation whose viewport is momentarily absent (the modal just closed) waits briefly for the renderer to restore the strip rather than rejecting — so the first approved click/type lands without a retry. The user always sees the page the agent is acting on — the visible view plus the per-turn permission prompt is the safety net, now enforced rather than assumed.
• Renderer panel (browser-panel.tsx): address bar + nav controls (lucide icons, matching the app's icon set). The panel reserves a strip and mirrors its on-screen rect to main each animation frame, so the native view tracks the strip on resize / sidebar drags. The page is a native view floating above the DOM, not a React child.

Deliberate scope

• Browser views are ephemeral across app restart (login persists, the live page does not) — the simplest split; the agent re-opens on demand. Persisting the live URL across restarts was intentionally skipped.
• Multi-session parallel browsing is out of scope for now (YAGNI), tracked as a follow-up. The browser is single-window: one view is drawn at a time, and the visible-lease rejects mutations on an off-screen conversation. The blocker is the presentation layer, not state management — the runtime already runs sessions concurrently (SessionManager keys active sessions by id) and each conversation already owns its own view + CDP connection + history. A hidden embedded WebContentsView simply can't be driven for clicks: tested directly, a non-displayed view still does snapshot / type / navigate, but a native click silently no-ops because it never composites a frame to hit-test — true even with PawWork's exact 1280×720 default bound, parented or not. PawWork (where this was ported from) gets parallelism by being multi-window (each conversation's view displayed in its own visible window), not by hidden rendering. A follow-up would pick a presentation change — multi-window (à la PawWork), single-window split-pane, or offscreen rendering — and the current lease is just the single-window special case of "a view must be displayed somewhere to be driven", which generalizes when that lands.
• No "open browser from UI" button yet — the only cold entry is the agent (a navigate creates the conversation's panel). The address bar handles manual navigation once a page is open, but it lives inside the panel, which mounts only for a live view — so it is not itself a cold-start entry. A small open-browser affordance is deferred to a follow-up PR.
• No browser_screenshot yet — the app feeds no images to the model at all (attachments are stringified into the prompt), so a screenshot tool would only ever return a byte count the model can't use. Deferred to the phase-2 multimodal PR, which adds image-to-model support and rebuilds the tool against it.
• Cut after review. Two independent reviewers (Codex + a fresh-eye pass, Occam's-razor focus) drove a cleanup commit that removed the dead tool/host surface: the screenshot stub, speculative snapshot knobs, a duplicated URL validator, an unwired probe abstraction, and a few zero-caller exports (−157 lines, no capability lost).
• Hardened after a second review round. Added the visible-conversation lease above — the safety gap the review flagged (the agent could otherwise act on a hidden, backgrounded view). Plus two small fixes: viewportBounds now rejects non-finite rects from the untyped IPC boundary before they reach setBounds, and the shared-partition security backstop (will-download + permission handlers) installs once per session instead of once per view (no listener pile-up across conversations).
• Tightened after a third review round. The browser permission is now a single explicit contract: browser gets its own prompt reason (the dialog names the logged-in session it drives, not a generic “custom” request) and one turn-wide permission scope, so “allow for this turn” actually carries the whole observe→act loop instead of re-prompting on every ref. Plus: browser_extract recovers from an invalid CSS selector (a [12] ref a model echoes) as a clean “no match” instead of a raw DOMException, and @jackwener/opencli is pinned to exact 1.8.4 to match the contract test’s “pinned release” assertion.
• Tightened after a fourth review round. Two safety gaps closed by one model — the agent touches the browser only for the conversation you're watching, and only after you approve it for the turn: (1) the first approved click/type after a permission grant no longer loses the race against the renderer's viewport restore (the modal hides the native view; the mutate now waits briefly for the strip to come back rather than rejecting); (2) the visible lease now gates reads too, so a backgrounded conversation can't snapshot/extract a logged-in page off screen; and (3) the prompt is honest that one browser grant covers the whole turn's reads, navigation, clicks, and typing (rather than splitting into two prompts — the live visible view is the act-phase safety net).
• Tightened after a fifth review round. The whole-turn browser note no longer overstates: it renders only when "本轮记住" is checked (the runtime persists the grant only on allow && rememberForTurn — locked by the rememberForTurn=false does NOT add to set test), and it drops the inaccurate "switching conversations revokes the grant" line. endTurn fires on run completion/abort (closing a conversation aborts its turn), never on a plain switch — a switch just parks the action behind the visible lease and resumes it on return without re-asking. The note now states the grant expires when the turn ends.
• Tightened after a sixth review round. The visible lease is now CONTINUOUS, not a one-time preflight: canDrive() only gated the start, so a browser_wait / navigate / extract / delayed mutate that began while shown kept running after the user switched away. withBrowserPage registers each in-flight action; main's browser:active-session handler calls revokeHiddenBrowserActions(shown) on every switch, severing any action whose conversation just went off screen (same connection-sever path as a timeout/abort) and rejecting with a new BrowserActionRevokedError — so no tool result can carry hidden-page data. And background throttling is now scoped to shown-ness instead of held off for the whole cached connection's life: a hidden conversation's page throttles normally (no off-screen CPU drain), while a shown view stays full-speed so native CDP clicks composite (deleting the override outright was tried and rejected — the smoke proves it's load-bearing for clicks on any view the OS backgrounds, e.g. the app being unfocused). Covered by two new session unit tests + three new smoke checks (live-bridge revoke, both throttle transitions); smoke now 19/19.
• Chat-header badges were hardened so the narrower chat column (browser panel open) no longer collapses short status pills (e.g. 运行中) into vertical text; only the tab title and model label absorb the squeeze.

Verification

Freshly re-run on this branch:

• npm run typecheck — clean across core / storage / runtime / ui / desktop.
• Unit tests:
  • core 618 pass / 0 fail
  • storage 76 pass / 0 fail
  • desktop 1469 pass / 0 fail
  • runtime 396 pass / 1 fail — pre-existing & environment-dependent, not from this PR (which touches no runtime/network code). The failing test network/proxy-test "times out when the proxy accepts TCP but never responds" asserts the error text matches /timeout/i, but in this sandbox fetch to the target returns "fetch failed" before the 100 ms timeout fires — a machine-dependence the test's own comment calls out.

Validated on this branch (CDP path unchanged by the later UI commits):

• npm run smoke:browser — 19/19: real Electron observe→act E2E (sealed ws bridge ↔ webContents.debugger ↔ opencli ↔ live DOM): goto → numbered snapshot → fill-by-ref → click-by-ref → DOM-effect verify → markdown extract, plus a check that the partition security backstop installs once across views, plus the visible-lease driven through the real host/manager/BrowserSession end to end (background-conversation read/navigate/mutate all rejected with no view or connection created; a mutate on the shown conversation waits out a modal-close viewport restore and lands; click/type land once the conversation is shown with a viewport). New this round: an in-flight read is revoked against the live bridge when its conversation goes off screen, and a shown view runs un-throttled while hiding it restores background throttling.
• Manual GUI smoke on real sites: panel render, address bar + nav, agent observe→act, permission prompts (explore/ask/execute), per-conversation view isolation, shared login partition, stealth probe, archive/quit teardown.

[Astro-Han]

Summary

Embedded browser automation for the desktop agent — ported from PawWork's opencli CDP + numbered-ref observe→act approach. The agent drives a real, per-conversation Chromium view through 6 generic tools, and the page renders live in a right-side panel.

• Sealed CDP bridge (cdp-bridge.ts): wraps webContents.debugger behind a loopback WebSocket (ws://127.0.0.1:<random-port>/<secret>, secret kept in main-process memory only). opencli's CDPBridge client connects to it; its stealth script auto-registers on connect and applies to future documents.
• 6 generic observe→act tools (registered unconditionally, no flag): browser_navigate / snapshot / click / type / wait / extract. Loop: numbered [ref] snapshot (observe) → act by ref (click/type).
• Per-conversation views: each conversation owns a WebContentsView child of the main window — lazily created, stacked, all hidden except the shown one. Views are in-memory (ephemeral across app restart, by design); a shared persistent partition (persist:maka-browser) keeps cookies/login on disk so a login in one conversation is available to all and survives restart.
• Dedicated browser permission category — prompt-on-effect: blocked in explore, prompts in ask and execute (browser effects are treated as irreversible, not auto-allowed like read tools). Takeover-reload is deferred to first effect: observe never reloads, the first mutate reloads once to apply stealth to an already-open page, navigate clears without reloading.
• Visible-conversation lease: the agent touches the browser only for the conversation currently on screen. EVERY action — snapshot / extract / wait (read), navigate, click / type (act) — is rejected when the calling conversation is backgrounded, so a background conversation can't even read a logged-in page the user can't see; a mutate additionally requires a real, non-empty on-screen viewport (opencli's native CDP click hit-tests a composited frame a hidden view lacks). The check runs before the view/connection is acquired, so a vetoed background action creates neither. The lease is continuous, not just a preflight: an action already running when the user switches away is revoked and its connection severed, so a long wait/navigate/extract/delayed mutate can never keep reading or driving a now-hidden page. Because the permission modal hides the native view while it is open, a mutate on the on-screen conversation whose viewport is momentarily absent (the modal just closed) waits briefly for the renderer to restore the strip rather than rejecting — so the first approved click/type lands without a retry. The user always sees the page the agent is acting on — the visible view plus the per-turn permission prompt is the safety net, now enforced rather than assumed.
• Renderer panel (browser-panel.tsx): address bar + nav controls (lucide icons, matching the app's icon set). The panel reserves a strip and mirrors its on-screen rect to main each animation frame, so the native view tracks the strip on resize / sidebar drags. The page is a native view floating above the DOM, not a React child.

Deliberate scope

• Browser views are ephemeral across app restart (login persists, the live page does not) — the simplest split; the agent re-opens on demand. Persisting the live URL across restarts was intentionally skipped.
• Multi-session parallel browsing is out of scope for now (YAGNI), tracked as a follow-up. The browser is single-window: one view is drawn at a time, and the visible-lease rejects mutations on an off-screen conversation. The blocker is the presentation layer, not state management — the runtime already runs sessions concurrently (SessionManager keys active sessions by id) and each conversation already owns its own view + CDP connection + history. A hidden embedded WebContentsView simply can't be driven for clicks: tested directly, a non-displayed view still does snapshot / type / navigate, but a native click silently no-ops because it never composites a frame to hit-test — true even with PawWork's exact 1280×720 default bound, parented or not. PawWork (where this was ported from) gets parallelism by being multi-window (each conversation's view displayed in its own visible window), not by hidden rendering. A follow-up would pick a presentation change — multi-window (à la PawWork), single-window split-pane, or offscreen rendering — and the current lease is just the single-window special case of "a view must be displayed somewhere to be driven", which generalizes when that lands.
• No "open browser from UI" button yet — the only cold entry is the agent (a navigate creates the conversation's panel). The address bar handles manual navigation once a page is open, but it lives inside the panel, which mounts only for a live view — so it is not itself a cold-start entry. A small open-browser affordance is deferred to a follow-up PR.
• No browser_screenshot yet — the app feeds no images to the model at all (attachments are stringified into the prompt), so a screenshot tool would only ever return a byte count the model can't use. Deferred to the phase-2 multimodal PR, which adds image-to-model support and rebuilds the tool against it.
• Cut after review. Two independent reviewers (Codex + a fresh-eye pass, Occam's-razor focus) drove a cleanup commit that removed the dead tool/host surface: the screenshot stub, speculative snapshot knobs, a duplicated URL validator, an unwired probe abstraction, and a few zero-caller exports (−157 lines, no capability lost).
• Hardened after a second review round. Added the visible-conversation lease above — the safety gap the review flagged (the agent could otherwise act on a hidden, backgrounded view). Plus two small fixes: viewportBounds now rejects non-finite rects from the untyped IPC boundary before they reach setBounds, and the shared-partition security backstop (will-download + permission handlers) installs once per session instead of once per view (no listener pile-up across conversations).
• Tightened after a third review round. The browser permission is now a single explicit contract: browser gets its own prompt reason (the dialog names the logged-in session it drives, not a generic “custom” request) and one turn-wide permission scope, so “allow for this turn” actually carries the whole observe→act loop instead of re-prompting on every ref. Plus: browser_extract recovers from an invalid CSS selector (a [12] ref a model echoes) as a clean “no match” instead of a raw DOMException, and @jackwener/opencli is pinned to exact 1.8.4 to match the contract test’s “pinned release” assertion.
• Tightened after a fourth review round. Two safety gaps closed by one model — the agent touches the browser only for the conversation you're watching, and only after you approve it for the turn: (1) the first approved click/type after a permission grant no longer loses the race against the renderer's viewport restore (the modal hides the native view; the mutate now waits briefly for the strip to come back rather than rejecting); (2) the visible lease now gates reads too, so a backgrounded conversation can't snapshot/extract a logged-in page off screen; and (3) the prompt is honest that one browser grant covers the whole turn's reads, navigation, clicks, and typing (rather than splitting into two prompts — the live visible view is the act-phase safety net).
• Tightened after a fifth review round. The whole-turn browser note no longer overstates: it renders only when "本轮记住" is checked (the runtime persists the grant only on allow && rememberForTurn — locked by the rememberForTurn=false does NOT add to set test), and it drops the inaccurate "switching conversations revokes the grant" line. endTurn fires on run completion/abort (closing a conversation aborts its turn), never on a plain switch — a switch just parks the action behind the visible lease and resumes it on return without re-asking. The note now states the grant expires when the turn ends.
• Tightened after a sixth review round. The visible lease is now CONTINUOUS, not a one-time preflight: canDrive() only gated the start, so a browser_wait / navigate / extract / delayed mutate that began while shown kept running after the user switched away. withBrowserPage registers each in-flight action; main's browser:active-session handler calls revokeHiddenBrowserActions(shown) on every switch, severing any action whose conversation just went off screen (same connection-sever path as a timeout/abort) and rejecting with a new BrowserActionRevokedError — so no tool result can carry hidden-page data. And background throttling is now scoped to shown-ness instead of held off for the whole cached connection's life: a hidden conversation's page throttles normally (no off-screen CPU drain), while a shown view stays full-speed so native CDP clicks composite (deleting the override outright was tried and rejected — the smoke proves it's load-bearing for clicks on any view the OS backgrounds, e.g. the app being unfocused). Covered by two new session unit tests + three new smoke checks (live-bridge revoke, both throttle transitions); smoke now 19/19.
• Chat-header badges were hardened so the narrower chat column (browser panel open) no longer collapses short status pills (e.g. 运行中) into vertical text; only the tab title and model label absorb the squeeze.

Verification

Freshly re-run on this branch:

• npm run typecheck — clean across core / storage / runtime / ui / desktop.
• Unit tests:
  • core 618 pass / 0 fail
  • storage 76 pass / 0 fail
  • desktop 1469 pass / 0 fail
  • runtime 396 pass / 1 fail — pre-existing & environment-dependent, not from this PR (which touches no runtime/network code). The failing test network/proxy-test "times out when the proxy accepts TCP but never responds" asserts the error text matches /timeout/i, but in this sandbox fetch to the target returns "fetch failed" before the 100 ms timeout fires — a machine-dependence the test's own comment calls out.

Validated on this branch (CDP path unchanged by the later UI commits):

• npm run smoke:browser — 19/19: real Electron observe→act E2E (sealed ws bridge ↔ webContents.debugger ↔ opencli ↔ live DOM): goto → numbered snapshot → fill-by-ref → click-by-ref → DOM-effect verify → markdown extract, plus a check that the partition security backstop installs once across views, plus the visible-lease driven through the real host/manager/BrowserSession end to end (background-conversation read/navigate/mutate all rejected with no view or connection created; a mutate on the shown conversation waits out a modal-close viewport restore and lands; click/type land once the conversation is shown with a viewport). New this round: an in-flight read is revoked against the live bridge when its conversation goes off screen, and a shown view runs un-throttled while hiding it restores background throttling.
• Manual GUI smoke on real sites: panel render, address bar + nav, agent observe→act, permission prompts (explore/ask/execute), per-conversation view isolation, shared login partition, stealth probe, archive/quit teardown.

[Astro-Han]

Summary

Embedded browser automation for the desktop agent — ported from PawWork's opencli CDP + numbered-ref observe→act approach. The agent drives a real, per-conversation Chromium view through 6 generic tools, and the page renders live in a right-side panel.

• Sealed CDP bridge (cdp-bridge.ts): wraps webContents.debugger behind a loopback WebSocket (ws://127.0.0.1:<random-port>/<secret>, secret kept in main-process memory only). opencli's CDPBridge client connects to it; its stealth script auto-registers on connect and applies to future documents.
• 6 generic observe→act tools (registered unconditionally, no flag): browser_navigate / snapshot / click / type / wait / extract. Loop: numbered [ref] snapshot (observe) → act by ref (click/type).
• Per-conversation views: each conversation owns a WebContentsView child of the main window — lazily created, stacked, all hidden except the shown one. Views are in-memory (ephemeral across app restart, by design); a shared persistent partition (persist:maka-browser) keeps cookies/login on disk so a login in one conversation is available to all and survives restart.
• Dedicated browser permission category — prompt-on-effect: blocked in explore, prompts in ask and execute (browser effects are treated as irreversible, not auto-allowed like read tools). Takeover-reload is deferred to first effect: observe never reloads, the first mutate reloads once to apply stealth to an already-open page, navigate clears without reloading.
• Visible-conversation lease: the agent touches the browser only for the conversation currently on screen. EVERY action — snapshot / extract / wait (read), navigate, click / type (act) — is rejected when the calling conversation is backgrounded, so a background conversation can't even read a logged-in page the user can't see; a mutate additionally requires a real, non-empty on-screen viewport (opencli's native CDP click hit-tests a composited frame a hidden view lacks). The check runs before the view/connection is acquired, so a vetoed background action creates neither. The lease is continuous, not just a preflight: an action already running when the user switches away is revoked and its connection severed, so a long wait/navigate/extract/delayed mutate can never keep reading or driving a now-hidden page. Because the permission modal hides the native view while it is open, a mutate on the on-screen conversation whose viewport is momentarily absent (the modal just closed) waits briefly for the renderer to restore the strip rather than rejecting — so the first approved click/type lands without a retry. The user always sees the page the agent is acting on — the visible view plus the per-turn permission prompt is the safety net, now enforced rather than assumed.
• Renderer panel (browser-panel.tsx): address bar + nav controls (lucide icons, matching the app's icon set). The panel reserves a strip and mirrors its on-screen rect to main each animation frame, so the native view tracks the strip on resize / sidebar drags. The page is a native view floating above the DOM, not a React child.

Deliberate scope

• Browser views are ephemeral across app restart (login persists, the live page does not) — the simplest split; the agent re-opens on demand. Persisting the live URL across restarts was intentionally skipped.
• Multi-session parallel browsing is out of scope for now (YAGNI), tracked as a follow-up. The browser is single-window: one view is drawn at a time, and the visible-lease rejects mutations on an off-screen conversation. The blocker is the presentation layer, not state management — the runtime already runs sessions concurrently (SessionManager keys active sessions by id) and each conversation already owns its own view + CDP connection + history. A hidden embedded WebContentsView simply can't be driven for clicks: tested directly, a non-displayed view still does snapshot / type / navigate, but a native click silently no-ops because it never composites a frame to hit-test — true even with PawWork's exact 1280×720 default bound, parented or not. PawWork (where this was ported from) gets parallelism by being multi-window (each conversation's view displayed in its own visible window), not by hidden rendering. A follow-up would pick a presentation change — multi-window (à la PawWork), single-window split-pane, or offscreen rendering — and the current lease is just the single-window special case of "a view must be displayed somewhere to be driven", which generalizes when that lands.
• No "open browser from UI" button yet — the only cold entry is the agent (a navigate creates the conversation's panel). The address bar handles manual navigation once a page is open, but it lives inside the panel, which mounts only for a live view — so it is not itself a cold-start entry. A small open-browser affordance is deferred to a follow-up PR.
• No browser_screenshot yet — the app feeds no images to the model at all (attachments are stringified into the prompt), so a screenshot tool would only ever return a byte count the model can't use. Deferred to the phase-2 multimodal PR, which adds image-to-model support and rebuilds the tool against it.
• Cut after review. Two independent reviewers (Codex + a fresh-eye pass, Occam's-razor focus) drove a cleanup commit that removed the dead tool/host surface: the screenshot stub, speculative snapshot knobs, a duplicated URL validator, an unwired probe abstraction, and a few zero-caller exports (−157 lines, no capability lost).
• Hardened after a second review round. Added the visible-conversation lease above — the safety gap the review flagged (the agent could otherwise act on a hidden, backgrounded view). Plus two small fixes: viewportBounds now rejects non-finite rects from the untyped IPC boundary before they reach setBounds, and the shared-partition security backstop (will-download + permission handlers) installs once per session instead of once per view (no listener pile-up across conversations).
• Tightened after a third review round. The browser permission is now a single explicit contract: browser gets its own prompt reason (the dialog names the logged-in session it drives, not a generic “custom” request) and one turn-wide permission scope, so “allow for this turn” actually carries the whole observe→act loop instead of re-prompting on every ref. Plus: browser_extract recovers from an invalid CSS selector (a [12] ref a model echoes) as a clean “no match” instead of a raw DOMException, and @jackwener/opencli is pinned to exact 1.8.4 to match the contract test’s “pinned release” assertion.
• Tightened after a fourth review round. Two safety gaps closed by one model — the agent touches the browser only for the conversation you're watching, and only after you approve it for the turn: (1) the first approved click/type after a permission grant no longer loses the race against the renderer's viewport restore (the modal hides the native view; the mutate now waits briefly for the strip to come back rather than rejecting); (2) the visible lease now gates reads too, so a backgrounded conversation can't snapshot/extract a logged-in page off screen; and (3) the prompt is honest that one browser grant covers the whole turn's reads, navigation, clicks, and typing (rather than splitting into two prompts — the live visible view is the act-phase safety net).
• Tightened after a fifth review round. The whole-turn browser note no longer overstates: it renders only when "本轮记住" is checked (the runtime persists the grant only on allow && rememberForTurn — locked by the rememberForTurn=false does NOT add to set test), and it drops the inaccurate "switching conversations revokes the grant" line. endTurn fires on run completion/abort (closing a conversation aborts its turn), never on a plain switch — a switch just parks the action behind the visible lease and resumes it on return without re-asking. The note now states the grant expires when the turn ends.
• Tightened after a sixth review round. The visible lease is now CONTINUOUS, not a one-time preflight: canDrive() only gated the start, so a browser_wait / navigate / extract / delayed mutate that began while shown kept running after the user switched away. withBrowserPage registers each in-flight action; main's browser:active-session handler calls revokeHiddenBrowserActions(shown) on every switch, severing any action whose conversation just went off screen (same connection-sever path as a timeout/abort) and rejecting with a new BrowserActionRevokedError — so no tool result can carry hidden-page data. And background throttling is now scoped to shown-ness instead of held off for the whole cached connection's life: a hidden conversation's page throttles normally (no off-screen CPU drain), while a shown view stays full-speed so native CDP clicks composite (deleting the override outright was tried and rejected — the smoke proves it's load-bearing for clicks on any view the OS backgrounds, e.g. the app being unfocused). Covered by two new session unit tests + three new smoke checks (live-bridge revoke, both throttle transitions); smoke now 19/19.
• Chat-header badges were hardened so the narrower chat column (browser panel open) no longer collapses short status pills (e.g. 运行中) into vertical text; only the tab title and model label absorb the squeeze.

Verification

Freshly re-run on this branch:

• npm run typecheck — clean across core / storage / runtime / ui / desktop.
• Unit tests:
  • core 618 pass / 0 fail
  • storage 76 pass / 0 fail
  • desktop 1469 pass / 0 fail
  • runtime 396 pass / 1 fail — pre-existing & environment-dependent, not from this PR (which touches no runtime/network code). The failing test network/proxy-test "times out when the proxy accepts TCP but never responds" asserts the error text matches /timeout/i, but in this sandbox fetch to the target returns "fetch failed" before the 100 ms timeout fires — a machine-dependence the test's own comment calls out.

Validated on this branch (CDP path unchanged by the later UI commits):

• npm run smoke:browser — 19/19: real Electron observe→act E2E (sealed ws bridge ↔ webContents.debugger ↔ opencli ↔ live DOM): goto → numbered snapshot → fill-by-ref → click-by-ref → DOM-effect verify → markdown extract, plus a check that the partition security backstop installs once across views, plus the visible-lease driven through the real host/manager/BrowserSession end to end (background-conversation read/navigate/mutate all rejected with no view or connection created; a mutate on the shown conversation waits out a modal-close viewport restore and lands; click/type land once the conversation is shown with a viewport). New this round: an in-flight read is revoked against the live bridge when its conversation goes off screen, and a shown view runs un-throttled while hiding it restores background throttling.
• Manual GUI smoke on real sites: panel render, address bar + nav, agent observe→act, permission prompts (explore/ask/execute), per-conversation view isolation, shared login partition, stealth probe, archive/quit teardown.

[Astro-Han]

Summary

Embedded browser automation for the desktop agent — ported from PawWork's opencli CDP + numbered-ref observe→act approach. The agent drives a real, per-conversation Chromium view through 6 generic tools, and the page renders live in a right-side panel.

• Sealed CDP bridge (cdp-bridge.ts): wraps webContents.debugger behind a loopback WebSocket (ws://127.0.0.1:<random-port>/<secret>, secret kept in main-process memory only). opencli's CDPBridge client connects to it; its stealth script auto-registers on connect and applies to future documents.
• 6 generic observe→act tools (registered unconditionally, no flag): browser_navigate / snapshot / click / type / wait / extract. Loop: numbered [ref] snapshot (observe) → act by ref (click/type).
• Per-conversation views: each conversation owns a WebContentsView child of the main window — lazily created, stacked, all hidden except the shown one. Views are in-memory (ephemeral across app restart, by design); a shared persistent partition (persist:maka-browser) keeps cookies/login on disk so a login in one conversation is available to all and survives restart.
• Dedicated browser permission category — prompt-on-effect: blocked in explore, prompts in ask and execute (browser effects are treated as irreversible, not auto-allowed like read tools). Takeover-reload is deferred to first effect: observe never reloads, the first mutate reloads once to apply stealth to an already-open page, navigate clears without reloading.
• Visible-conversation lease: the agent touches the browser only for the conversation currently on screen. EVERY action — snapshot / extract / wait (read), navigate, click / type (act) — is rejected when the calling conversation is backgrounded, so a background conversation can't even read a logged-in page the user can't see; a mutate additionally requires a real, non-empty on-screen viewport (opencli's native CDP click hit-tests a composited frame a hidden view lacks). The check runs before the view/connection is acquired, so a vetoed background action creates neither. The lease is continuous, not just a preflight: an action already running when the user switches away is revoked and its connection severed, so a long wait/navigate/extract/delayed mutate can never keep reading or driving a now-hidden page. Because the permission modal hides the native view while it is open, a mutate on the on-screen conversation whose viewport is momentarily absent (the modal just closed) waits briefly for the renderer to restore the strip rather than rejecting — so the first approved click/type lands without a retry. The user always sees the page the agent is acting on — the visible view plus the per-turn permission prompt is the safety net, now enforced rather than assumed.
• Renderer panel (browser-panel.tsx): address bar + nav controls (lucide icons, matching the app's icon set). The panel reserves a strip and mirrors its on-screen rect to main each animation frame, so the native view tracks the strip on resize / sidebar drags. The page is a native view floating above the DOM, not a React child.

Deliberate scope

• Browser views are ephemeral across app restart (login persists, the live page does not) — the simplest split; the agent re-opens on demand. Persisting the live URL across restarts was intentionally skipped.
• Multi-session parallel browsing is out of scope for now (YAGNI), tracked as a follow-up. The browser is single-window: one view is drawn at a time, and the visible-lease rejects mutations on an off-screen conversation. The blocker is the presentation layer, not state management — the runtime already runs sessions concurrently (SessionManager keys active sessions by id) and each conversation already owns its own view + CDP connection + history. A hidden embedded WebContentsView simply can't be driven for clicks: tested directly, a non-displayed view still does snapshot / type / navigate, but a native click silently no-ops because it never composites a frame to hit-test — true even with PawWork's exact 1280×720 default bound, parented or not. PawWork (where this was ported from) gets parallelism by being multi-window (each conversation's view displayed in its own visible window), not by hidden rendering. A follow-up would pick a presentation change — multi-window (à la PawWork), single-window split-pane, or offscreen rendering — and the current lease is just the single-window special case of "a view must be displayed somewhere to be driven", which generalizes when that lands.
• No "open browser from UI" button yet — the only cold entry is the agent (a navigate creates the conversation's panel). The address bar handles manual navigation once a page is open, but it lives inside the panel, which mounts only for a live view — so it is not itself a cold-start entry. A small open-browser affordance is deferred to a follow-up PR.
• No browser_screenshot yet — the app feeds no images to the model at all (attachments are stringified into the prompt), so a screenshot tool would only ever return a byte count the model can't use. Deferred to the phase-2 multimodal PR, which adds image-to-model support and rebuilds the tool against it.
• Cut after review. Two independent reviewers (Codex + a fresh-eye pass, Occam's-razor focus) drove a cleanup commit that removed the dead tool/host surface: the screenshot stub, speculative snapshot knobs, a duplicated URL validator, an unwired probe abstraction, and a few zero-caller exports (−157 lines, no capability lost).
• Hardened after a second review round. Added the visible-conversation lease above — the safety gap the review flagged (the agent could otherwise act on a hidden, backgrounded view). Plus two small fixes: viewportBounds now rejects non-finite rects from the untyped IPC boundary before they reach setBounds, and the shared-partition security backstop (will-download + permission handlers) installs once per session instead of once per view (no listener pile-up across conversations).
• Tightened after a third review round. The browser permission is now a single explicit contract: browser gets its own prompt reason (the dialog names the logged-in session it drives, not a generic “custom” request) and one turn-wide permission scope, so “allow for this turn” actually carries the whole observe→act loop instead of re-prompting on every ref. Plus: browser_extract recovers from an invalid CSS selector (a [12] ref a model echoes) as a clean “no match” instead of a raw DOMException, and @jackwener/opencli is pinned to exact 1.8.4 to match the contract test’s “pinned release” assertion.
• Tightened after a fourth review round. Two safety gaps closed by one model — the agent touches the browser only for the conversation you're watching, and only after you approve it for the turn: (1) the first approved click/type after a permission grant no longer loses the race against the renderer's viewport restore (the modal hides the native view; the mutate now waits briefly for the strip to come back rather than rejecting); (2) the visible lease now gates reads too, so a backgrounded conversation can't snapshot/extract a logged-in page off screen; and (3) the prompt is honest that one browser grant covers the whole turn's reads, navigation, clicks, and typing (rather than splitting into two prompts — the live visible view is the act-phase safety net).
• Tightened after a fifth review round. The whole-turn browser note no longer overstates: it renders only when "本轮记住" is checked (the runtime persists the grant only on allow && rememberForTurn — locked by the rememberForTurn=false does NOT add to set test), and it drops the inaccurate "switching conversations revokes the grant" line. endTurn fires on run completion/abort (closing a conversation aborts its turn), never on a plain switch — a switch just parks the action behind the visible lease and resumes it on return without re-asking. The note now states the grant expires when the turn ends.
• Tightened after a sixth review round. The visible lease is now CONTINUOUS, not a one-time preflight: canDrive() only gated the start, so a browser_wait / navigate / extract / delayed mutate that began while shown kept running after the user switched away. withBrowserPage registers each in-flight action; main's browser:active-session handler calls revokeHiddenBrowserActions(shown) on every switch, severing any action whose conversation just went off screen (same connection-sever path as a timeout/abort) and rejecting with a new BrowserActionRevokedError — so no tool result can carry hidden-page data. And background throttling is now scoped to shown-ness instead of held off for the whole cached connection's life: a hidden conversation's page throttles normally (no off-screen CPU drain), while a shown view stays full-speed so native CDP clicks composite (deleting the override outright was tried and rejected — the smoke proves it's load-bearing for clicks on any view the OS backgrounds, e.g. the app being unfocused). Covered by two new session unit tests + three new smoke checks (live-bridge revoke, both throttle transitions); smoke now 19/19.
• Chat-header badges were hardened so the narrower chat column (browser panel open) no longer collapses short status pills (e.g. 运行中) into vertical text; only the tab title and model label absorb the squeeze.

Verification

Freshly re-run on this branch:

• npm run typecheck — clean across core / storage / runtime / ui / desktop.
• Unit tests:
  • core 618 pass / 0 fail
  • storage 76 pass / 0 fail
  • desktop 1469 pass / 0 fail
  • runtime 396 pass / 1 fail — pre-existing & environment-dependent, not from this PR (which touches no runtime/network code). The failing test network/proxy-test "times out when the proxy accepts TCP but never responds" asserts the error text matches /timeout/i, but in this sandbox fetch to the target returns "fetch failed" before the 100 ms timeout fires — a machine-dependence the test's own comment calls out.

Validated on this branch (CDP path unchanged by the later UI commits):

• npm run smoke:browser — 19/19: real Electron observe→act E2E (sealed ws bridge ↔ webContents.debugger ↔ opencli ↔ live DOM): goto → numbered snapshot → fill-by-ref → click-by-ref → DOM-effect verify → markdown extract, plus a check that the partition security backstop installs once across views, plus the visible-lease driven through the real host/manager/BrowserSession end to end (background-conversation read/navigate/mutate all rejected with no view or connection created; a mutate on the shown conversation waits out a modal-close viewport restore and lands; click/type land once the conversation is shown with a viewport). New this round: an in-flight read is revoked against the live bridge when its conversation goes off screen, and a shown view runs un-throttled while hiding it restores background throttling.
• Manual GUI smoke on real sites: panel render, address bar + nav, agent observe→act, permission prompts (explore/ask/execute), per-conversation view isolation, shared login partition, stealth probe, archive/quit teardown.

[Astro-Han]

Summary

Embedded browser automation for the desktop agent — ported from PawWork's opencli CDP + numbered-ref observe→act approach. The agent drives a real, per-conversation Chromium view through 6 generic tools, and the page renders live in a right-side panel.

• Sealed CDP bridge (cdp-bridge.ts): wraps webContents.debugger behind a loopback WebSocket (ws://127.0.0.1:<random-port>/<secret>, secret kept in main-process memory only). opencli's CDPBridge client connects to it; its stealth script auto-registers on connect and applies to future documents.
• 6 generic observe→act tools (registered unconditionally, no flag): browser_navigate / snapshot / click / type / wait / extract. Loop: numbered [ref] snapshot (observe) → act by ref (click/type).
• Per-conversation views: each conversation owns a WebContentsView child of the main window — lazily created, stacked, all hidden except the shown one. Views are in-memory (ephemeral across app restart, by design); a shared persistent partition (persist:maka-browser) keeps cookies/login on disk so a login in one conversation is available to all and survives restart.
• Dedicated browser permission category — prompt-on-effect: blocked in explore, prompts in ask and execute (browser effects are treated as irreversible, not auto-allowed like read tools). Takeover-reload is deferred to first effect: observe never reloads, the first mutate reloads once to apply stealth to an already-open page, navigate clears without reloading.
• Visible-conversation lease: the agent touches the browser only for the conversation currently on screen. EVERY action — snapshot / extract / wait (read), navigate, click / type (act) — is rejected when the calling conversation is backgrounded, so a background conversation can't even read a logged-in page the user can't see; a mutate additionally requires a real, non-empty on-screen viewport (opencli's native CDP click hit-tests a composited frame a hidden view lacks). The check runs before the view/connection is acquired, so a vetoed background action creates neither. The lease is continuous, not just a preflight: an action already running when the user switches away is revoked and its connection severed, so a long wait/navigate/extract/delayed mutate can never keep reading or driving a now-hidden page. Because the permission modal hides the native view while it is open, a mutate on the on-screen conversation whose viewport is momentarily absent (the modal just closed) waits briefly for the renderer to restore the strip rather than rejecting — so the first approved click/type lands without a retry. The user always sees the page the agent is acting on — the visible view plus the per-turn permission prompt is the safety net, now enforced rather than assumed.
• Renderer panel (browser-panel.tsx): address bar + nav controls (lucide icons, matching the app's icon set). The panel reserves a strip and mirrors its on-screen rect to main each animation frame, so the native view tracks the strip on resize / sidebar drags. The page is a native view floating above the DOM, not a React child.

Deliberate scope

• Browser views are ephemeral across app restart (login persists, the live page does not) — the simplest split; the agent re-opens on demand. Persisting the live URL across restarts was intentionally skipped.
• Multi-session parallel browsing is out of scope for now (YAGNI), tracked as a follow-up. The browser is single-window: one view is drawn at a time, and the visible-lease rejects mutations on an off-screen conversation. The blocker is the presentation layer, not state management — the runtime already runs sessions concurrently (SessionManager keys active sessions by id) and each conversation already owns its own view + CDP connection + history. A hidden embedded WebContentsView simply can't be driven for clicks: tested directly, a non-displayed view still does snapshot / type / navigate, but a native click silently no-ops because it never composites a frame to hit-test — true even with PawWork's exact 1280×720 default bound, parented or not. PawWork (where this was ported from) gets parallelism by being multi-window (each conversation's view displayed in its own visible window), not by hidden rendering. A follow-up would pick a presentation change — multi-window (à la PawWork), single-window split-pane, or offscreen rendering — and the current lease is just the single-window special case of "a view must be displayed somewhere to be driven", which generalizes when that lands.
• No "open browser from UI" button yet — the only cold entry is the agent (a navigate creates the conversation's panel). The address bar handles manual navigation once a page is open, but it lives inside the panel, which mounts only for a live view — so it is not itself a cold-start entry. A small open-browser affordance is deferred to a follow-up PR.
• No browser_screenshot yet — the app feeds no images to the model at all (attachments are stringified into the prompt), so a screenshot tool would only ever return a byte count the model can't use. Deferred to the phase-2 multimodal PR, which adds image-to-model support and rebuilds the tool against it.
• Cut after review. Two independent reviewers (Codex + a fresh-eye pass, Occam's-razor focus) drove a cleanup commit that removed the dead tool/host surface: the screenshot stub, speculative snapshot knobs, a duplicated URL validator, an unwired probe abstraction, and a few zero-caller exports (−157 lines, no capability lost).
• Hardened after a second review round. Added the visible-conversation lease above — the safety gap the review flagged (the agent could otherwise act on a hidden, backgrounded view). Plus two small fixes: viewportBounds now rejects non-finite rects from the untyped IPC boundary before they reach setBounds, and the shared-partition security backstop (will-download + permission handlers) installs once per session instead of once per view (no listener pile-up across conversations).
• Tightened after a third review round. The browser permission is now a single explicit contract: browser gets its own prompt reason (the dialog names the logged-in session it drives, not a generic “custom” request) and one turn-wide permission scope, so “allow for this turn” actually carries the whole observe→act loop instead of re-prompting on every ref. Plus: browser_extract recovers from an invalid CSS selector (a [12] ref a model echoes) as a clean “no match” instead of a raw DOMException, and @jackwener/opencli is pinned to exact 1.8.4 to match the contract test’s “pinned release” assertion.
• Tightened after a fourth review round. Two safety gaps closed by one model — the agent touches the browser only for the conversation you're watching, and only after you approve it for the turn: (1) the first approved click/type after a permission grant no longer loses the race against the renderer's viewport restore (the modal hides the native view; the mutate now waits briefly for the strip to come back rather than rejecting); (2) the visible lease now gates reads too, so a backgrounded conversation can't snapshot/extract a logged-in page off screen; and (3) the prompt is honest that one browser grant covers the whole turn's reads, navigation, clicks, and typing (rather than splitting into two prompts — the live visible view is the act-phase safety net).
• Tightened after a fifth review round. The whole-turn browser note no longer overstates: it renders only when "本轮记住" is checked (the runtime persists the grant only on allow && rememberForTurn — locked by the rememberForTurn=false does NOT add to set test), and it drops the inaccurate "switching conversations revokes the grant" line. endTurn fires on run completion/abort (closing a conversation aborts its turn), never on a plain switch — a switch just parks the action behind the visible lease and resumes it on return without re-asking. The note now states the grant expires when the turn ends.
• Tightened after a sixth review round. The visible lease is now CONTINUOUS, not a one-time preflight: canDrive() only gated the start, so a browser_wait / navigate / extract / delayed mutate that began while shown kept running after the user switched away. withBrowserPage registers each in-flight action; main's browser:active-session handler calls revokeHiddenBrowserActions(shown) on every switch, severing any action whose conversation just went off screen (same connection-sever path as a timeout/abort) and rejecting with a new BrowserActionRevokedError — so no tool result can carry hidden-page data. And background throttling is now scoped to shown-ness instead of held off for the whole cached connection's life: a hidden conversation's page throttles normally (no off-screen CPU drain), while a shown view stays full-speed so native CDP clicks composite (deleting the override outright was tried and rejected — the smoke proves it's load-bearing for clicks on any view the OS backgrounds, e.g. the app being unfocused). Covered by two new session unit tests + three new smoke checks (live-bridge revoke, both throttle transitions); smoke now 19/19.
• Chat-header badges were hardened so the narrower chat column (browser panel open) no longer collapses short status pills (e.g. 运行中) into vertical text; only the tab title and model label absorb the squeeze.

Verification

Freshly re-run on this branch:

• npm run typecheck — clean across core / storage / runtime / ui / desktop.
• Unit tests:
  • core 618 pass / 0 fail
  • storage 76 pass / 0 fail
  • desktop 1469 pass / 0 fail
  • runtime 396 pass / 1 fail — pre-existing & environment-dependent, not from this PR (which touches no runtime/network code). The failing test network/proxy-test "times out when the proxy accepts TCP but never responds" asserts the error text matches /timeout/i, but in this sandbox fetch to the target returns "fetch failed" before the 100 ms timeout fires — a machine-dependence the test's own comment calls out.

Validated on this branch (CDP path unchanged by the later UI commits):

• npm run smoke:browser — 19/19: real Electron observe→act E2E (sealed ws bridge ↔ webContents.debugger ↔ opencli ↔ live DOM): goto → numbered snapshot → fill-by-ref → click-by-ref → DOM-effect verify → markdown extract, plus a check that the partition security backstop installs once across views, plus the visible-lease driven through the real host/manager/BrowserSession end to end (background-conversation read/navigate/mutate all rejected with no view or connection created; a mutate on the shown conversation waits out a modal-close viewport restore and lands; click/type land once the conversation is shown with a viewport). New this round: an in-flight read is revoked against the live bridge when its conversation goes off screen, and a shown view runs un-throttled while hiding it restores background throttling.
• Manual GUI smoke on real sites: panel render, address bar + nav, agent observe→act, permission prompts (explore/ask/execute), per-conversation view isolation, shared login partition, stealth probe, archive/quit teardown.