Add Bestie realtime voice through the Buzz agent

[tlongwell-block]

Adds a headphones control to Bestie for duplex voice conversations through an OpenAI Realtime compatible endpoint. One shared call follows the Bestie panel across the app, with transcripts, microphone mute, thinking level and tool approval controls. Changing community or account, disconnecting, or closing the panel ends the call and revokes pending approvals.

The development host reuses buzz-agent and buzz-dev-mcp for tools. Bestie receives its own persistent Nostr keypair and owner-signed NIP-OA attestation, and each call is bound to the selected community. Tool approval defaults to Automatic; users can select Ask each time before connecting. Provider credentials stay on the host. Browser notifications use native EventSource with short-lived, single-use authorization so affected WebKit versions do not stall sparse updates.

Configure BUZZ_REALTIME_ENDPOINT in .env.local, with optional provider token, model and binary paths; see the setup guide. This works through the live development broker in just web or just desktop. Packaged applications do not yet include the agent host.

Validation: hosted CI is green, including TypeScript/build, unit tests, Rust/tool integration, browser measurements and Chromium/WebKit journeys. Local checks also cover 46 targeted host/media tests, 1,115 unit tests, signed identity/profile writes against an isolated relay, and repeated sparse-stream regression tests on Linux WebKit.

All four Frankie combinations—llama.cpp and MTPLX, each with Qwen TTS or Breeze TTS—passed the real browser acceptance harness on Apple Silicon using the published source revisions. Each run exercised repeated spoken turns, a short pause inside an utterance, an automatically approved MCP shell tool, barge-in with a distinct new answer, and reconnect. Both MTPLX runs had no playback gaps. Each native run recorded one brief gap (91–96 ms) during long-story playback; normal replies had none. The model runtimes and model artifacts are supplied separately; no model weights or reference recordings are included here.

[wesbillman]

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Reviewed head a3eda453e713bd2ef0dbf79b8c5fcb1d6310545c against base/merge-base 17f90c18fff6b86bc029e710401fb2b60bc385ea. Changes required for the defects below. The development-host-only scope and Automatic tool approval default are treated as intentional, not findings.

[P1] Allow cooperative cancellation before terminating the agent

dev/bestie-realtime.mjs:143–153 closes agent stdin and immediately sends SIGTERM to its process group. The pinned buzz-agent handles stdin EOF by cancelling sessions, but does not install a SIGTERM handler. Immediate termination can prevent that cleanup. The comment that the detached agent group contains the MCP children is incorrect for the pinned implementation: MCP starts a new group (crates/buzz-agent/src/mcp.rs:758), and shell tools start another (crates/buzz-dev-mcp/src/shell.rs:677).

Closing MCP's pipes is not immediate tool cancellation: its pinned rmcp 1.8.0 implementation drains in-flight handlers for up to five seconds. In an isolated probe with the real release MCP binary, a shell command wrote its harmless scratch marker 2.02 seconds after both parent pipes were closed. The host can already release call capacity when the agent group exits, allowing an old approved command to execute after disconnect/community change and overlap a new call.

Give agent EOF cancellation a bounded cooperative window before termination/escalation, and retain call capacity until the owned work is stopped. Add a real agent/MCP topology regression using a delayed scratch write. This is a bounded post-disconnect execution defect, not an indefinite orphan claim.

[P1] Declare native microphone usage before advertising desktop voice

src/bundled/bestie/media/voice.mjs:468–475 adds microphone capture, and docs/bestie-voice.md:13–14 advertises just desktop, but the native app has no NSMicrophoneUsageDescription or src-tauri/Info.plist. Inspection of the macOS dev executable built at this exact head confirms its embedded plist contains only the bundle name and version keys. Tauri dev codegen reads a supplied Info.plist; it does not synthesize this privacy declaration.

The native path therefore lacks the required macOS microphone purpose declaration. Supply it for the dev executable and validate the actual native permission grant/denial path. The new Playwright fixture replaces getUserMedia with synthetic audio, so its green WebKit runs do not cover this boundary. This request does not expand scope to a packaged agent host or sandboxed release entitlements.

[P2] Flush the final coalesced playback position after the in-flight ACK

src/bundled/bestie/media/voice.mjs:118–130 returns when a playback report is already pending. A newer worklet position overwrites latestPlayback, but the finally only clears playbackPending; it never sends that retained tail. The worklet suppresses unchanged positions (audio-worklet.mjs:90–97), so after audio finishes there is no further event to rescue the final report.

A focused probe running the unchanged production voice client and worklet emitted positions [2400, 4800], held the first ACP playback ACK until the second arrived, and then acknowledged it. The only RPC position sent was [2400], even after additional worklet ticks. With an audio response plus a tool call, the pinned agent sets pending_input for the tool continuation but refuses to create the response while played < emitted (realtime_session/media.rs:138–149, 679). Bestie therefore falls silent after the tool until another utterance interrupts and resets the state.

Re-drain a retained latest position when the pending report settles, preserving the existing closed/interruption guards. Add a delayed-ACK regression that verifies the final sample count reaches the agent without additional speech.

Validation

All hosted checks were successful at the reviewed head. Review included the cross-module call/approval lifecycle, scoped broker routes, identity persistence, actual pinned ACP producer, dependency shutdown behavior, native executable metadata, the isolated real-MCP probe, and the production-client/worklet delayed-ACK probe. No live community mutations, microphone capture, or TCC changes were performed. Broad CI suites were not rerun locally by the review lead.