fix(pi-plugin): support pi-web multi-session and RPC hosts

[elrond298]

Summary

This PR adapts the Pi plugin for pi-web, where multiple Pi sessions share one persistent Node.js process and commands run through Pi’s RPC mode.

It addresses shared-process session isolation, duplicate startup work, unsafe subagent CLI detection, RPC command feedback, and /ctx-dream failures before the first model turn.

Changes

Session isolation

• Replace the process-global child-session flag with an AsyncLocalStorage<boolean> marker.
• Update the marker through the public subagents:child:session-created and subagents:child:disposed events.
• Suppress initialization only for in-process child subagents.
• Allow independent sessions in the same pi-web process to initialize normally.
• Remove child lifecycle listeners during session_shutdown to prevent stale handlers after reloads.

Process-wide startup maintenance

• Claim deferred startup maintenance once per process.
• Schedule the legacy-memory identity rekey introduced by database migration v22 only once per process.
• Schedule the session-to-project mapping backfill only once per process.
• Load session history only after acquiring the durable backfill lease, avoiding redundant JSONL scans.

Safer subagent CLI detection

• Reuse process.argv[1] only when it identifies a supported Pi CLI; otherwise use the packaged executable, bundled CLI, or PATH fallback as appropriate.
• This prevents embedded hosts such as Next.js from accidentally launching another web server when creating a subagent.

RPC command presentation

• Preserve model-invisible command entries while presenting them through the RPC UI.
• Show short progress updates through ctx.ui.notify.
• Show formatted results in modal dialogs through ctx.ui.custom.
• Apply this behavior to /ctx-status, /ctx-embed, /ctx-recomp, /ctx-session-upgrade, and /ctx-dream.

Dreamer registration

• Track Dreamer registration ownership per Pi extension instance.
• Prevent one session’s shutdown from deregistering a same-project sibling.
• Transfer the active registration to a remaining worktree owner when necessary.
• Synchronize the current project’s Dreamer registration immediately before a manual /ctx-dream run.
• This allows /ctx-dream to work before the first before_agent_start event.

Verification

• bun run --cwd packages/pi-plugin build
• bun run --cwd packages/pi-plugin test — 804 passed, 0 failed
• bun test packages/plugin/src/features/magic-context/session-project-backfill.test.ts — 9 passed, 0 failed
• git diff --check

Regression coverage includes:

• independent same-process sessions versus child-session suppression
• lifecycle listener cleanup during shutdown
• process-wide startup maintenance claimed through full runtime initialization
• lazy backfill lease gating without session-history reads
• owner-aware Dreamer registration, shutdown isolation, and ownership transfer
• single-timer Dreamer handoff when remaining owners repeat a worktree directory
• embedded-host Pi CLI discrimination
• RPC notification and modal-dialog routing
• pre-execution /ctx-dream registration synchronization

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

---

Summary by cubic

Supports pi-web multi-session and RPC hosts in packages/pi-plugin, isolating work by session and owner. Old behavior suppressed all same‑process inits and drained all jobs on shutdown; new behavior suppresses only in‑process child subagents, drains only the shutting‑down session’s historian/recomp and the active owner’s Dreamer jobs (cancel after ~5s), and presents command output through the live RPC context.

• Scope child‑session detection to lifecycle AsyncLocalStorage; remove lifecycle listeners on session_shutdown; keep agent_end synchronous.
• Claim process‑wide startup tasks once and defer session‑history reads until the backfill lease is acquired.
• Dreamer registration is shared per project and owner‑aware: require an active registration owner for manual runs, ensure registration before /ctx-dream, resolve manual runs through the active owner’s options, reject ownerless/deregistered requests, transfer/handoff ownership on exit, notify all owners after adjunct updates, preserve process‑shared argument order across extension reloads, prevent stale runs across worktrees, and track complete manual runs (including lease waits) during shutdown.
• Route command output through the live RPC context: short progress via ctx.ui.notify; detailed results or rpcDisplay: "dialog" via ctx.ui.custom, with a fallback to notifications when dialogs are unavailable. Applied to /ctx-status, /ctx-embed, /ctx-recomp, /ctx-session-upgrade, /ctx-wrapup, /ctx-flush, and /ctx-dream. Capture session snapshots at command start and block late UI updates.
• Await only the matching session’s historian and recomp during session_shutdown; propagate cancellation to recomp and memory migration and abort remaining work after a short grace period.
• Resolve subagent CLI safely: reuse process.argv[1] only when the host is the Pi CLI; otherwise prefer the packaged binary, bundled CLI, or pi on PATH. Never spawn with a shell.

^{Written for commit 329dddf. Summary will update on new commits.}

Greptile Summary

This PR adapts the Pi plugin to persistent multi-session and RPC hosts while isolating session-owned lifecycle and background work.

• Uses async-local child-session detection and removes lifecycle listeners during shutdown.
• Makes startup maintenance process-wide and gates session-history reads behind the durable backfill lease.
• Adds owner-aware Dreamer registration, transfer, manual-run synchronization, and shutdown draining.
• Routes command progress and results through each invocation’s live RPC context.
• Safely selects the Pi executable when running inside embedded hosts.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
packages/pi-plugin/src/index.ts	Introduces async-local child-session suppression, process-wide maintenance claims, project-aware Dreamer synchronization, and session-scoped shutdown cleanup.
packages/pi-plugin/src/dreamer/index.ts	Adds owner-aware project registration, generation-fenced timer handoff, owner-bound manual execution, and per-owner in-flight tracking.
packages/pi-plugin/src/pi-recomp-runner.ts	Tracks detached recomp jobs by session and fences late completion after shutdown cancellation.
packages/pi-plugin/src/commands/pi-command-utils.ts	Adds invocation-scoped RPC notifications and dialogs while preserving model-invisible command entries.
packages/pi-plugin/src/subagent-runner.ts	Restricts argv reuse to supported Pi CLI paths and preserves safe executable fallbacks for embedded hosts.
packages/pi-plugin/src/read-session-pi.ts	Defers session-history loading until the durable backfill lease is acquired.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
H[Persistent Pi host] --> S1[Session A extension]
H --> S2[Session B extension]
S1 --> C1[Session-scoped commands and jobs]
S2 --> C2[Session-scoped commands and jobs]
S1 --> D[Process-shared Dreamer registry]
S2 --> D
D --> O[Active project owner]
O --> T[Scheduled and manual Dreamer work]
S1 --> M[Process-wide startup maintenance]
S2 --> M
C1 --> U1[Live RPC UI context A]
C2 --> U2[Live RPC UI context B]

Loading

_{Reviews (7): Last reviewed commit: "fix(pi-plugin): harden embedded host lif..." | Re-trigger Greptile}

Context used (3)

• Knowledge Base — Pi plugin runtime
• Knowledge Base — Historian and dreamer workflows
• Knowledge Base — Dreamer orchestration

[cubic-dev-ai]

All reported issues were addressed across 17 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 17 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[alfonso-magic-context]

Thanks for this — especially as a first contribution. The diagnosis is right: the #247 process-global latch is what makes the second pi-web session skip Magic Context entirely, and routing child suppression through subagents:child:session-created/disposed plus AsyncLocalStorage is the correct seam. Once-per-process startup maintenance, not reusing a non-Pi argv[1], Dreamer sibling ownership, and keeping ctx-status entries model-invisible while presenting them in RPC are all the right instincts. And noted that you already pushed the Dreamer owner-handoff stabilization mid-review — that resolves one of the items we had flagged, and that kind of responsiveness makes this easy to shepherd.

Two clarifications so we don't talk past each other:

1. "RPC hosts" here is Pi RPC mode (ctx.ui.notify/ctx.ui.custom). It does not change Magic Context's RPC server, which must stay on 127.0.0.1 with a bearer token. We checked; this PR does not touch that.
2. The old "second init in the process is a no-op" test should change — that contract is the bug for pi-web. Please keep the child-only skip test (you did).

Before we can merge:

• Dreamer registeredProjects on globalThis (same jiti moduleCache:false reason as the child marker), so two sessions in one repo don't start two timers.
• session_shutdown draining only that session's in-flight work — in pi-web, shutdown is not process exit.
• RPC presentation using the command's live ctx, not a session_start closure.
• The #177 "never spawn bare pi" test kept alongside the new embedded-host test.
• packages/pi-plugin/PARITY.md updated for RPC dialogs, the multi-session process model, and the latch → ALS change.

We've approved CI for this PR so your next push gets the full check suite. Really solid work — happy to re-review quickly.

[cubic-dev-ai]

All reported issues were addressed across 21 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 4 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[elrond298]

Rebased onto the latest master (52e650c6) and force-pushed the updated branch. No functional changes were made as part of the rebase.

[alfonso-magic-context]

Thanks for the rework — the quality jump since the last round is real, and most of it verified clean under a full re-review (all five prior items confirmed addressed at source; ALS suppression held up under a four-concurrent-child probe with an async hop; both suites green on the PR merged onto current master; no public adapter surface rides in, which matters because the API question is deliberately deferred to #353).

Four items block the merge, two of them reproduced by probe rather than read off the diff:

1. Stale sibling-worktree dreamer timer.registerPiDreamerProject keeps the old owner in the shared owners map when a different owner re-registers the same project identity from another directory (src/dreamer/index.ts:157-185), and the retired timer's predicate checks only that old owner's entry and directory (:190-194) — not the current registration generation/activeOwner. A probe with owner A/worktree A then owner B/worktree B showed the old A client still prompts. The committed test (dreamer/index.test.ts:428-470) covers one owner changing directories and misses this. Bind scheduled clients to the current registration generation (or activeOwner+projectDir) and add the sibling-owner regression.

2. Shutdown timeout abandons rather than cancels.src/index.ts:2337-2373 stops waiting after 5s while timeout.ts:1-14 leaves the underlying promise alive; detached recomp/upgrade retains ctx (commands/ctx-recomp.ts:177-278) and pi-recomp-runner.ts:69-84 calls onStatusChange from finally without a stale-context guard. Pi 0.83 invalidates command contexts after disposal, so long recomp work can outlive shutdown and throw on dead ctx/UI. Capture immutable session data up front, fence or cancel on shutdown, and guard post-await presentation.

3. Native Pi 0.83 RPC dialogs are silently vacuous.pi-command-utils.ts:180-190 assumes ctx.ui.custom renders or rejects, but Pi 0.83 rpc-mode implements it as Promise.resolve(undefined) — the catch fallback never runs, so native/external RPC hosts lose the detailed results entirely. It works in your host because pi-web supplies its own custom UI. Add a capability check with a portable notification fallback (and a test against the real 0.83 shape, not a functional fake), or scope the PARITY.md claim explicitly to hosts that provide ui.custom.

4. Missing #247 storm regression. The process-global latch you removed originally existed for the four-child in-process init storm. Our probe of your ALS scoping passed it — the implementation looks right — but no committed test reproduces the storm. Please add the parallel multi-child regression asserting every child registers no tools/events/background scans.

Nothing else stands between this and merge — the internals-first direction is settled on our side, and #353 tracks the public API question separately.

[elrond298]

Thanks for the detailed review. I addressed the four remaining items, rebased the branch onto master at 7d6fda35, and force-pushed the updated commits.

1. Dreamer timers and results from an old registration

   Each Dreamer registration now has a version number. A scheduled Dreamer job checks that version before creating a session, sending a prompt, reading messages, and accepting the final result. If another worktree or a reloaded plugin has replaced the registration, the old job stops and its result is ignored.

   I also covered the asynchronous A → B → A case. If the first A timer finishes starting after the final A timer has already been registered, cleanup from the first timer cannot remove the final timer entry stored for that project directory.

2. Recompilation and upgrade tasks after session shutdown

   Recompilation and session upgrade now copy the required session, directory, branch, and model data when the command starts. Their background tasks no longer read from the command context after it may have become invalid.

   During session_shutdown, the plugin waits up to five seconds for that session’s background tasks to finish. Tasks that are still running then receive a cancellation signal. After cancellation or shutdown, their results cannot update compaction markers, migration state, memories, or the session’s UI.

   agent_end still returns immediately. Waiting and cancellation happen only during session_shutdown.

3. Dialog fallback in native Pi 0.83

   Native Pi 0.83 can resolve ui.custom() with undefined without displaying anything or throwing an error. The plugin now records whether the rendering callback passed to ui.custom() was actually called.

   pi-web continues to display the custom dialog. If native Pi does not call the rendering callback, the command displays the same information as a notification instead.

   All commands that display status also receive the session shutdown signal. If ui.custom() resolves or rejects after shutdown, the command does not use the old UI context to display a notification.

4. Four child sessions initializing concurrently

   I added a behavioral test that starts four child sessions at the same time and closes them in different orders, including while other child sessions are still initializing.

   The test verifies that:

   • each child session keeps its own child-session state;
   • child sessions do not register primary-session tools, event handlers, entry renderers, or startup maintenance;
   • closing one child session does not change how the other child sessions are handled;
   • the parent session still performs its normal initialization.

   This keeps the current AsyncLocalStorage design and verifies that concurrently running child sessions do not overwrite one another’s state.

Additional points from the earlier review remain covered:

• All plugin instances in the same process use the same Dreamer registration map, preventing duplicate timers for the same project.
• Shutdown waits only for work owned by the session and Dreamer registration being closed.
• RPC commands receive the current command context when invoked instead of retaining a context from plugin startup. Their status output is displayed in the UI and is not added to the model conversation.
• The [pi] Historian error spawning pi (Windows Powershell) #177 regression remains a separate test. It creates a valid Pi CLI path and verifies that the child process uses process.execPath with that cli.js, does not execute bare pi, and does not use a shell. The adjacent embedded-host test separately verifies that the plugin does not re-run an unrelated host program from process.argv[1].
• PARITY.md documents the shared process behavior, RPC presentation behavior, and how AsyncLocalStorage keeps concurrently initializing child sessions separate.
• No implementation of the Host Adapter API proposed in [pi-plugin] pi-web embedded multi-session: field data + proposal for a public Host Adapter API (coordination with #350) #353, and no dashboard changes, are included in this PR.

After rebasing onto 7d6fda35, the Pi-plugin suite passes 848 tests. Another 59 tests covering the changed Dreamer and shared code also pass. The build, workspace type checking, formatting check, and Git whitespace check pass.

The new CI and Smoke runs are currently waiting for workflow approval. Cubic reports that it skipped automatic review because the force-push rewrote the branch history, so it will need to be triggered manually.