Uh oh!
There was an error while loading. Please reload this page.
feat: add cross-platform voice supervisor - #6206
Conversation
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| const upstreamRetryAfterSeconds = (value: string | undefined): number => { | ||
| if (value === undefined) return 1; | ||
| const parsed = Number.parseFloat(value); | ||
| return Number.isFinite(parsed) && parsed > 0 ? Math.min(120, Math.ceil(parsed)) : 1; | ||
| }; |
There was a problem hiding this comment.
🟡 Mediumvoice/OpenAiRealtime.ts:107
When a 429 response includes an HTTP-date in the Retry-After header (e.g. Tue, 11 Aug 2026 12:00:00 GMT), upstreamRetryAfterSeconds parses it with Number.parseFloat, which returns NaN. The function then falls back to 1, so callers receive retryAfterSeconds: 1 and retry before the upstream-specified time. HTTP permits Retry-After to be either a delta-seconds value or an HTTP-date; consider parsing both forms (e.g. falling back to Date.parse when the float parse fails).
const upstreamRetryAfterSeconds = (value: string | undefined): number => {
if (value === undefined) return 1;
const parsed = Number.parseFloat(value);
- return Number.isFinite(parsed) && parsed > 0 ? Math.min(120, Math.ceil(parsed)) : 1;+ if (Number.isFinite(parsed) && parsed > 0) return Math.min(120, Math.ceil(parsed));+ const dateMs = Date.parse(value);+ if (Number.isFinite(dateMs)) {+ const delta = Math.ceil((dateMs - Date.now()) / 1_000);+ return delta > 0 ? Math.min(120, delta) : 1;+ }+ return 1;
};🤖 Copy this AI Prompt to have your agent fix this:
In file @apps/server/src/voice/OpenAiRealtime.ts around lines 107-111:
When a 429 response includes an HTTP-date in the `Retry-After` header (e.g. `Tue, 11 Aug 2026 12:00:00 GMT`), `upstreamRetryAfterSeconds` parses it with `Number.parseFloat`, which returns `NaN`. The function then falls back to `1`, so callers receive `retryAfterSeconds: 1` and retry before the upstream-specified time. HTTP permits `Retry-After` to be either a delta-seconds value or an HTTP-date; consider parsing both forms (e.g. falling back to `Date.parse` when the float parse fails).
| projectModelSelection: project.defaultModelSelection, | ||
| draft, | ||
| }); | ||
| const primaryDefaults = dependencies.readPrimaryThreadDefaults(); |
There was a problem hiding this comment.
🟠 Highvoice/voiceStartDefaults.ts:253
primaryDefaults is read from readPrimaryThreadDefaults(), which returns the primary environment's settings. When project.environmentId targets a different environment, defaultThreadEnvMode (line 264) and newWorktreesStartFromOrigin (line 307) are resolved against the wrong environment. This can cause a voice command on a remote project to create a local checkout instead of a worktree (or vice versa), or to apply the wrong start-from-origin policy. The target environment's settings are already available in environment.settings — use that instead.
🤖 Copy this AI Prompt to have your agent fix this:
In file @apps/web/src/voice/voiceStartDefaults.ts around line 253:
`primaryDefaults` is read from `readPrimaryThreadDefaults()`, which returns the *primary* environment's settings. When `project.environmentId` targets a different environment, `defaultThreadEnvMode` (line 264) and `newWorktreesStartFromOrigin` (line 307) are resolved against the wrong environment. This can cause a voice command on a remote project to create a local checkout instead of a worktree (or vice versa), or to apply the wrong start-from-origin policy. The target environment's settings are already available in `environment.settings` — use that instead.
There was a problem hiding this comment.
Reviewed the new/changed Effect service code in this PR (apps/server/src/voice/*, apps/desktop/src/electron/ElectronSystemPreferences.ts, packages/client-runtime/src/state/voiceHttp.ts, packages/contracts/src/voice.ts, and the web/mobile voice adapters).
Most of it follows the conventions: ElectronSystemPreferences.ts, OpenAiRealtime.ts, and OpenAiRealtimeCredential.ts all use the canonical imports/Context.Service/make/layer shape with inline interfaces, Schema.TaggedErrorClass failures, structured attributes, exported Schema.is predicates, and dependencies acquired from the environment; ManagedRuntime/runPromise stay at React, Expo, and app-runtime boundaries.
Four convention violations in the changed scope are noted inline: Effect.catchTag in the two new server voice modules, two pure compatibility re-export shims left behind by the web-to-client-runtime move, and a non-canonical layer export name for a new Context.Service.
Posted via Macroscope — Effect Service Conventions
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
t3dotgg
commented
Aug 23, 2026
Note 🤖 GPT-5.6 Sol responding on behalf of Theo Closing this PR after an automated pass over open pull requests. Duplicates the maintainer-owned voice implementation in #5213. |
What changed
@t3tools/client-runtime.Why
T3 can coordinate multiple coding agents and environments, but doing so currently requires staying at the keyboard. Voice Supervisor adds an explicit, reviewable speech interface for listing work, checking status, opening threads, and proposing confirmed commands while preserving T3's existing environment authority and receipt-backed execution model.
This is a full Realtime conversation rather than dictation. It uses the host environment's OpenAI API credential or
OPENAI_API_KEY; it does not reuse ChatGPT subscription entitlements.Behavior and safety
Verification
vp i --frozen-lockfile --force— dependency and supply-chain verification passed; the installed WebRTC package contains the exact Android and iOS native pins.vp test run <54 changed test files>— 54 files, 682 tests passed.git diff --check origin/main...HEADpassed.Draft status
The source and focused verification gates are green. A rebuilt forked desktop/mobile client, real OpenAI Realtime smoke, and final UI screenshots are intentionally still pending and will be added before requesting maintainer review.
Related prior Realtime panel exploration: #3997.
Implemented with GPT-5.6 Sol through the Codex harness in T3 Code.
Note
Add cross-platform Voice Supervisor for real-time AI voice conversations
/api/voice/openai/credential,/api/voice/realtime/client-secret) for credential management and ephemeral client-secret minting, with rate limiting, scope enforcement, and no-store headers.client-runtimemodules for realtime session transport, event decoding, supervisor state reduction, tool schemas, and thread/voice supervisor repository operations.react-native-webrtc-backed adapter for mobile, each with audio session lifecycle management.T3VoiceAudioSession) for iOS and Android to observe audio interruptions, route loss, and media services reset events without taking audio session ownership./settings/voicepage on web and a Voice Supervisor settings row on mobile for configuring the host environment and OpenAI API key.voice.toggleas a built-in keybinding command, wired into the command palette and sidebar on web.react-native-webrtc@124.0.7is added as a new mobile dependency and patched viapnpm-workspace.yaml; the Android manifest explicitly removesMediaProjectionServicefrom the WebRTC module.📊 Macroscope summarized 90f076e. 86 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted
🗂️ Filtered Issues
No issues evaluated.