feat(mobile): add offline iPhone voice input - #8614

Merged
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription
Aug 31, 2026
Merged

feat(mobile): add offline iPhone voice input#8614
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription

Conversation

@t3dotgg

@t3dotggt3dotgg commented Aug 29, 2026

Copy link
Copy Markdown
Member

The mobile composer could not use Apple's newer on-device transcription model.

This adds local dictation for supported iPhones on iOS 26 or later. The microphone is available in existing threads and new task drafts, including queued task editing. T3 Code prepares Apple's speech assets for the device language and transcribes up to five minutes of audio on the phone.

The transcript is inserted at the selection where recording started. T3 Code keeps the keyboard open and lets the user review the text before sending it. Canceling keeps the draft unchanged. Temporary audio files are removed when the operation ends.

Tests:

  • All 42 focused tests passed after the Expo SDK 57 rebase.
  • Mobile typecheck and targeted lint passed after the Expo SDK 57 rebase.
  • The pre-rebase Expo SDK 56 version passed an iOS release build and a physical iPhone test.

This change needs a new native mobile build.

UI screenshots are pending permission to capture them.

Built with GPT-5.6 Sol in the Codex harness through T3 Code.


Note

Medium Risk
Touches microphone permissions, audio session lifecycle, and draft mutation timing with new native dependencies; incorrect cancellation or late transcript handling could corrupt composer text or leave sessions locked.

Overview
Adds on-device dictation for supported iPhones (iOS 26+) in the new task and thread composers. Users tap the mic, see recording status and a live waveform, then confirm to transcribe locally and insert text at the caret where recording started—send stays blocked until transcription finishes or is canceled.

A new @t3tools/client-runtime/voice-inputVoiceInputController owns the full lifecycle: single app-wide session, permission checks, up to five minutes of capture, cooperative cancellation, temp file cleanup, and draft owner/revision guards so late transcripts cannot overwrite a changed draft. Mobile wires expo-audio recording and metering through useVoiceInputController, with @react-native-ai/apple on iOS for SpeechTranscriber preparation and transcription.

Composer UX swaps the toolbar for dictation controls (cancel, status, confirm) with Reanimated layout transitions; collapsed composers can show a compact dictation strip. The native T3ComposerEditor gains a readOnly mode that blocks edits, paste, and drops while keeping focus and the keyboard. useComposerCommandMenu now keys selection off ownerKey so draft/thread switches reset caret position.

expo-audio and @react-native-ai/apple are added with patches (recording failure reporting, locale resolution, transcription cleanup). app.config.ts registers microphone permission for expo-audio. User and maintainer docs describe behavior and future remote-transcription boundaries. Requires a new native mobile build.

Reviewed by Cursor Bugbot for commit 50fc0ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add offline iPhone voice dictation to mobile composer

  • Adds VoiceInputController in controller.ts to orchestrate the full dictation lifecycle: permission checks, audio session setup, recording, on-device transcription via Apple's Speech framework, transcript commit, and cancellation semantics.
  • Integrates dictation UI into ThreadComposer.tsx and NewTaskDraftScreen.tsx with an animated dictation toolbar, live waveform metering, elapsed-time status, and confirm/cancel controls. Submission, attachments, and pickers are blocked while dictation is busy.
  • Adds a readOnly mode to the native iOS composer editor (T3ComposerEditorView.swift) so the text view blocks edits, paste, and drops while preserving focus and selection during dictation.
  • Patches @react-native-ai/apple@0.12.0 and expo-audio@57.0.4 to fix locale resolution, temp-file cleanup, and recording-failure error reporting.
  • Risk: resolveTranscriptCommit in controller.ts contains stray + tokens in the leftNeedsBoundary/rightNeedsBoundary computation that break TypeScript syntax and will prevent the module from compiling.

Macroscope summarized 50fc0ed.

@coderabbitai

coderabbitaiBot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d020e7f5-6801-48a2-9440-06dce8facce2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Aug 29, 2026
Comment threadpackages/client-runtime/src/voice-input/controller.ts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

disabled={isIncomingShareTransferPending}

Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from NewTaskDraftScreen, whose focus-effect cleanup calls controller.dispose() and discards the recording; disable the control or guard its handler when voiceInput.isBusy is true.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/NewTaskDraftScreen.tsx around line 997:
Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from `NewTaskDraftScreen`, whose focus-effect cleanup calls `controller.dispose()` and discards the recording; disable the control or guard its handler when `voiceInput.isBusy` is true.

onChangeDraftMessage: props.onChangeDraftMessage,
onUpdateInteractionMode: props.onUpdateInteractionMode,
});
const voiceInput = useVoiceInputController({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highthreads/ThreadComposer.tsx:293

When a pending user-input request hides ThreadComposer, an active useVoiceInputController session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from ThreadDetailScreen.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/ThreadComposer.tsx around line 293:
When a pending user-input request hides `ThreadComposer`, an active `useVoiceInputController` session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from `ThreadDetailScreen`.

@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire13.3 KiB13.3 KiB+11 B (+0.1%)15.1 KiB
CodexThread snapshot wire6.9 KiB6.9 KiB−7 B (−0.1%)7.3 KiB
CodexLive turn WebSocket wire6.4 KiB6.5 KiB+18 B (+0.3%)7.8 KiB
CodexLive turn WebSocket decoded55.6 KiB55.6 KiB0 B (0.0%)66.4 KiB
CodexLive turn messages11110 (0.0%)21
ClaudeTotal thread wire13.3 KiB13.3 KiB+32 B (+0.2%)15.1 KiB
ClaudeThread snapshot wire6.9 KiB6.9 KiB+3 B (+0.0%)7.3 KiB
ClaudeLive turn WebSocket wire6.4 KiB6.4 KiB+29 B (+0.4%)7.8 KiB
ClaudeLive turn WebSocket decoded56.4 KiB56.4 KiB+44 B (+0.1%)66.4 KiB
ClaudeLive turn messages1011+1 (+10.0%)21

Baseline: e09b88b · PR result: 50fc0ed · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.4 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
right !== undefined &&
/[A-Za-z0-9([{'"]/.test(right) &&
(left === undefined || /\s/.test(left));
insertion = `${leftNeedsBoundary ? " " : ""}${replacement}${rightNeedsBoundary ? " " : ""}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emoji breaks English transcript spacing

Low Severity

English boundary spacing reads captured.text[index], which is a single UTF-16 code unit. A caret beside an emoji sees a surrogate that fails the letter/punctuation check, so the transcript is inserted without the intended leading or trailing space.

Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 384d7fb. Configure here.

Comment threadapps/mobile/src/features/threads/NewTaskDraftScreen.tsx
@macroscopeapp

macroscopeappBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial user-facing iPhone voice-input workflow spanning native audio capture, local transcription, draft mutation, and composer lifecycle changes, along with new app-level audio defaults. Unresolved lifecycle and emoji-spacing concerns indicate that the behavior needs human review before merging.

Not approved because:

  • 2 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarmingeforce-pushed the t3code/ios-local-voice-transcription branch from 384d7fb to b4c5578CompareAugust 30, 2026 22:08
Comment threadpackages/client-runtime/src/voice-input/controller.ts
Comment threadapps/mobile/src/native/voiceTranscription.ios.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad0c05f. Configure here.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
Comment threadpackages/client-runtime/src/voice-input/controller.ts Outdated
@juliusmarminge
juliusmarminge merged commit 352710d into mainAug 31, 2026
27 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/ios-local-voice-transcription branch August 31, 2026 01:08
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
**Full Changelog**: pingdotgg/t3code@v0.0.37-nightly.20260830.1232...v0.0.37-nightly.20260831.1233
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37-nightly.20260831.1233
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* Remove Messages Glass Lab experiment by @juliusmarminge in pingdotgg/t3code#8599
* Require human review for pull requests changing product defaults by @juliusmarminge in pingdotgg/t3code#8603
* fix(codex): avoid quadratic app-server input buffering by @juliusmarminge in pingdotgg/t3code#8605
* fix(mobile): stabilize iOS header item transitions by @juliusmarminge in pingdotgg/t3code#8607
* chore(mobile): upgrade to Expo SDK 57 by @juliusmarminge in pingdotgg/t3code#8609
* fix(mobile): harden native header toolbar items by @juliusmarminge in pingdotgg/t3code#8611
* fix(server): stop querying Claude context usage after turns by @t3dotgg in pingdotgg/t3code#8610
* chore: vouch ryanrhughes by @t3dotgg in pingdotgg/t3code#8613
* feat(web): attach PDFs, ZIPs, and other files to a turn by @t3dotgg in pingdotgg/t3code#8236
* feat(web): keybinding settings as settings rows by @StiensWout in pingdotgg/t3code#8532
* feat: let an environment publish themes as a file by @ryanrhughes in pingdotgg/t3code#8569
* fix(web): clean up provider settings list and editor by @StiensWout in pingdotgg/t3code#8504
* fix(web): keep project picker popup inside the sidebar by @SunkenInTime in pingdotgg/t3code#8627
* fix(mobile): prevent header overflow and back-button artifacts by @juliusmarminge in pingdotgg/t3code#8624
* fix(server): retry automatic thread title generation by @Bil0000 in pingdotgg/t3code#8087
* fix(client-runtime): refresh edited pull request comments by @Bil0000 in pingdotgg/t3code#8094
* fix(web): four composer spacing defects by @Bil0000 in pingdotgg/t3code#8090
* perf(desktop): skip duplicate browser updates by @Bil0000 in pingdotgg/t3code#8018
* fix(web): render nested markdown images correctly by @flamboh in pingdotgg/t3code#8501
* fix(web): unify activity logs and composer banners by @juliusmarminge in pingdotgg/t3code#8693
* fix(mobile): reduce dev-client reload and Metro startup cost by @juliusmarminge in pingdotgg/t3code#8694
* revert(web): restore previous composer banners by @t3dotgg in pingdotgg/t3code#8733
* test(web): remove tests for unreachable helpers by @t3-code[bot] in pingdotgg/t3code#8738
* feat(mobile): update tool summaries and chat transitions by @juliusmarminge in pingdotgg/t3code#8793
* feat(web): play video attachments in chat by @Bil0000 in pingdotgg/t3code#8688
* fix(web,mobile): snooze menu no longer offers the same wake time twice by @vitalyiegorov in pingdotgg/t3code#8741
* fix(grok): allow model changes in existing threads by @ahmed-besic in pingdotgg/t3code#8392
* feat(mobile): pick, share, and receive files in threads by @t3dotgg in pingdotgg/t3code#8237
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
* fix(web): prevent pull request metadata overlap by @MatthewFeroz in pingdotgg/t3code#8790
## New Contributors
* @ryanrhughes made their first contribution in pingdotgg/t3code#8569
* @ahmed-besic made their first contribution in pingdotgg/t3code#8392
* @MatthewFeroz made their first contribution in pingdotgg/t3code#8790
**Full Changelog**: pingdotgg/t3code@v0.0.36...v0.0.37
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native ChangeChanges the native fingerprint; merging blocks production OTAs until a new store build ships.size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@t3dotgg@juliusmarminge
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat(mobile): add offline iPhone voice input - #8614

Merged
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription
Aug 31, 2026
Merged

feat(mobile): add offline iPhone voice input#8614
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription

Conversation

@t3dotgg

@t3dotggt3dotgg commented Aug 29, 2026

Copy link
Copy Markdown
Member

The mobile composer could not use Apple's newer on-device transcription model.

This adds local dictation for supported iPhones on iOS 26 or later. The microphone is available in existing threads and new task drafts, including queued task editing. T3 Code prepares Apple's speech assets for the device language and transcribes up to five minutes of audio on the phone.

The transcript is inserted at the selection where recording started. T3 Code keeps the keyboard open and lets the user review the text before sending it. Canceling keeps the draft unchanged. Temporary audio files are removed when the operation ends.

Tests:

  • All 42 focused tests passed after the Expo SDK 57 rebase.
  • Mobile typecheck and targeted lint passed after the Expo SDK 57 rebase.
  • The pre-rebase Expo SDK 56 version passed an iOS release build and a physical iPhone test.

This change needs a new native mobile build.

UI screenshots are pending permission to capture them.

Built with GPT-5.6 Sol in the Codex harness through T3 Code.


Note

Medium Risk
Touches microphone permissions, audio session lifecycle, and draft mutation timing with new native dependencies; incorrect cancellation or late transcript handling could corrupt composer text or leave sessions locked.

Overview
Adds on-device dictation for supported iPhones (iOS 26+) in the new task and thread composers. Users tap the mic, see recording status and a live waveform, then confirm to transcribe locally and insert text at the caret where recording started—send stays blocked until transcription finishes or is canceled.

A new @t3tools/client-runtime/voice-inputVoiceInputController owns the full lifecycle: single app-wide session, permission checks, up to five minutes of capture, cooperative cancellation, temp file cleanup, and draft owner/revision guards so late transcripts cannot overwrite a changed draft. Mobile wires expo-audio recording and metering through useVoiceInputController, with @react-native-ai/apple on iOS for SpeechTranscriber preparation and transcription.

Composer UX swaps the toolbar for dictation controls (cancel, status, confirm) with Reanimated layout transitions; collapsed composers can show a compact dictation strip. The native T3ComposerEditor gains a readOnly mode that blocks edits, paste, and drops while keeping focus and the keyboard. useComposerCommandMenu now keys selection off ownerKey so draft/thread switches reset caret position.

expo-audio and @react-native-ai/apple are added with patches (recording failure reporting, locale resolution, transcription cleanup). app.config.ts registers microphone permission for expo-audio. User and maintainer docs describe behavior and future remote-transcription boundaries. Requires a new native mobile build.

Reviewed by Cursor Bugbot for commit 50fc0ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add offline iPhone voice dictation to mobile composer

  • Adds VoiceInputController in controller.ts to orchestrate the full dictation lifecycle: permission checks, audio session setup, recording, on-device transcription via Apple's Speech framework, transcript commit, and cancellation semantics.
  • Integrates dictation UI into ThreadComposer.tsx and NewTaskDraftScreen.tsx with an animated dictation toolbar, live waveform metering, elapsed-time status, and confirm/cancel controls. Submission, attachments, and pickers are blocked while dictation is busy.
  • Adds a readOnly mode to the native iOS composer editor (T3ComposerEditorView.swift) so the text view blocks edits, paste, and drops while preserving focus and selection during dictation.
  • Patches @react-native-ai/apple@0.12.0 and expo-audio@57.0.4 to fix locale resolution, temp-file cleanup, and recording-failure error reporting.
  • Risk: resolveTranscriptCommit in controller.ts contains stray + tokens in the leftNeedsBoundary/rightNeedsBoundary computation that break TypeScript syntax and will prevent the module from compiling.

Macroscope summarized 50fc0ed.

@coderabbitai

coderabbitaiBot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d020e7f5-6801-48a2-9440-06dce8facce2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Aug 29, 2026
Comment threadpackages/client-runtime/src/voice-input/controller.ts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

disabled={isIncomingShareTransferPending}

Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from NewTaskDraftScreen, whose focus-effect cleanup calls controller.dispose() and discards the recording; disable the control or guard its handler when voiceInput.isBusy is true.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/NewTaskDraftScreen.tsx around line 997:
Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from `NewTaskDraftScreen`, whose focus-effect cleanup calls `controller.dispose()` and discards the recording; disable the control or guard its handler when `voiceInput.isBusy` is true.

onChangeDraftMessage: props.onChangeDraftMessage,
onUpdateInteractionMode: props.onUpdateInteractionMode,
});
const voiceInput = useVoiceInputController({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highthreads/ThreadComposer.tsx:293

When a pending user-input request hides ThreadComposer, an active useVoiceInputController session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from ThreadDetailScreen.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/ThreadComposer.tsx around line 293:
When a pending user-input request hides `ThreadComposer`, an active `useVoiceInputController` session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from `ThreadDetailScreen`.

@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire13.3 KiB13.3 KiB+11 B (+0.1%)15.1 KiB
CodexThread snapshot wire6.9 KiB6.9 KiB−7 B (−0.1%)7.3 KiB
CodexLive turn WebSocket wire6.4 KiB6.5 KiB+18 B (+0.3%)7.8 KiB
CodexLive turn WebSocket decoded55.6 KiB55.6 KiB0 B (0.0%)66.4 KiB
CodexLive turn messages11110 (0.0%)21
ClaudeTotal thread wire13.3 KiB13.3 KiB+32 B (+0.2%)15.1 KiB
ClaudeThread snapshot wire6.9 KiB6.9 KiB+3 B (+0.0%)7.3 KiB
ClaudeLive turn WebSocket wire6.4 KiB6.4 KiB+29 B (+0.4%)7.8 KiB
ClaudeLive turn WebSocket decoded56.4 KiB56.4 KiB+44 B (+0.1%)66.4 KiB
ClaudeLive turn messages1011+1 (+10.0%)21

Baseline: e09b88b · PR result: 50fc0ed · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.4 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
right !== undefined &&
/[A-Za-z0-9([{'"]/.test(right) &&
(left === undefined || /\s/.test(left));
insertion = `${leftNeedsBoundary ? " " : ""}${replacement}${rightNeedsBoundary ? " " : ""}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emoji breaks English transcript spacing

Low Severity

English boundary spacing reads captured.text[index], which is a single UTF-16 code unit. A caret beside an emoji sees a surrogate that fails the letter/punctuation check, so the transcript is inserted without the intended leading or trailing space.

Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 384d7fb. Configure here.

Comment threadapps/mobile/src/features/threads/NewTaskDraftScreen.tsx
@macroscopeapp

macroscopeappBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial user-facing iPhone voice-input workflow spanning native audio capture, local transcription, draft mutation, and composer lifecycle changes, along with new app-level audio defaults. Unresolved lifecycle and emoji-spacing concerns indicate that the behavior needs human review before merging.

Not approved because:

  • 2 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarmingeforce-pushed the t3code/ios-local-voice-transcription branch from 384d7fb to b4c5578CompareAugust 30, 2026 22:08
Comment threadpackages/client-runtime/src/voice-input/controller.ts
Comment threadapps/mobile/src/native/voiceTranscription.ios.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad0c05f. Configure here.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
Comment threadpackages/client-runtime/src/voice-input/controller.ts Outdated
@juliusmarminge
juliusmarminge merged commit 352710d into mainAug 31, 2026
27 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/ios-local-voice-transcription branch August 31, 2026 01:08
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
**Full Changelog**: pingdotgg/t3code@v0.0.37-nightly.20260830.1232...v0.0.37-nightly.20260831.1233
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37-nightly.20260831.1233
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* Remove Messages Glass Lab experiment by @juliusmarminge in pingdotgg/t3code#8599
* Require human review for pull requests changing product defaults by @juliusmarminge in pingdotgg/t3code#8603
* fix(codex): avoid quadratic app-server input buffering by @juliusmarminge in pingdotgg/t3code#8605
* fix(mobile): stabilize iOS header item transitions by @juliusmarminge in pingdotgg/t3code#8607
* chore(mobile): upgrade to Expo SDK 57 by @juliusmarminge in pingdotgg/t3code#8609
* fix(mobile): harden native header toolbar items by @juliusmarminge in pingdotgg/t3code#8611
* fix(server): stop querying Claude context usage after turns by @t3dotgg in pingdotgg/t3code#8610
* chore: vouch ryanrhughes by @t3dotgg in pingdotgg/t3code#8613
* feat(web): attach PDFs, ZIPs, and other files to a turn by @t3dotgg in pingdotgg/t3code#8236
* feat(web): keybinding settings as settings rows by @StiensWout in pingdotgg/t3code#8532
* feat: let an environment publish themes as a file by @ryanrhughes in pingdotgg/t3code#8569
* fix(web): clean up provider settings list and editor by @StiensWout in pingdotgg/t3code#8504
* fix(web): keep project picker popup inside the sidebar by @SunkenInTime in pingdotgg/t3code#8627
* fix(mobile): prevent header overflow and back-button artifacts by @juliusmarminge in pingdotgg/t3code#8624
* fix(server): retry automatic thread title generation by @Bil0000 in pingdotgg/t3code#8087
* fix(client-runtime): refresh edited pull request comments by @Bil0000 in pingdotgg/t3code#8094
* fix(web): four composer spacing defects by @Bil0000 in pingdotgg/t3code#8090
* perf(desktop): skip duplicate browser updates by @Bil0000 in pingdotgg/t3code#8018
* fix(web): render nested markdown images correctly by @flamboh in pingdotgg/t3code#8501
* fix(web): unify activity logs and composer banners by @juliusmarminge in pingdotgg/t3code#8693
* fix(mobile): reduce dev-client reload and Metro startup cost by @juliusmarminge in pingdotgg/t3code#8694
* revert(web): restore previous composer banners by @t3dotgg in pingdotgg/t3code#8733
* test(web): remove tests for unreachable helpers by @t3-code[bot] in pingdotgg/t3code#8738
* feat(mobile): update tool summaries and chat transitions by @juliusmarminge in pingdotgg/t3code#8793
* feat(web): play video attachments in chat by @Bil0000 in pingdotgg/t3code#8688
* fix(web,mobile): snooze menu no longer offers the same wake time twice by @vitalyiegorov in pingdotgg/t3code#8741
* fix(grok): allow model changes in existing threads by @ahmed-besic in pingdotgg/t3code#8392
* feat(mobile): pick, share, and receive files in threads by @t3dotgg in pingdotgg/t3code#8237
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
* fix(web): prevent pull request metadata overlap by @MatthewFeroz in pingdotgg/t3code#8790
## New Contributors
* @ryanrhughes made their first contribution in pingdotgg/t3code#8569
* @ahmed-besic made their first contribution in pingdotgg/t3code#8392
* @MatthewFeroz made their first contribution in pingdotgg/t3code#8790
**Full Changelog**: pingdotgg/t3code@v0.0.36...v0.0.37
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native ChangeChanges the native fingerprint; merging blocks production OTAs until a new store build ships.size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@t3dotgg@juliusmarminge
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(mobile): add offline iPhone voice input - #8614

Merged
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription
Aug 31, 2026
Merged

feat(mobile): add offline iPhone voice input#8614
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription

Conversation

@t3dotgg

@t3dotggt3dotgg commented Aug 29, 2026

Copy link
Copy Markdown
Member

The mobile composer could not use Apple's newer on-device transcription model.

This adds local dictation for supported iPhones on iOS 26 or later. The microphone is available in existing threads and new task drafts, including queued task editing. T3 Code prepares Apple's speech assets for the device language and transcribes up to five minutes of audio on the phone.

The transcript is inserted at the selection where recording started. T3 Code keeps the keyboard open and lets the user review the text before sending it. Canceling keeps the draft unchanged. Temporary audio files are removed when the operation ends.

Tests:

  • All 42 focused tests passed after the Expo SDK 57 rebase.
  • Mobile typecheck and targeted lint passed after the Expo SDK 57 rebase.
  • The pre-rebase Expo SDK 56 version passed an iOS release build and a physical iPhone test.

This change needs a new native mobile build.

UI screenshots are pending permission to capture them.

Built with GPT-5.6 Sol in the Codex harness through T3 Code.


Note

Medium Risk
Touches microphone permissions, audio session lifecycle, and draft mutation timing with new native dependencies; incorrect cancellation or late transcript handling could corrupt composer text or leave sessions locked.

Overview
Adds on-device dictation for supported iPhones (iOS 26+) in the new task and thread composers. Users tap the mic, see recording status and a live waveform, then confirm to transcribe locally and insert text at the caret where recording started—send stays blocked until transcription finishes or is canceled.

A new @t3tools/client-runtime/voice-inputVoiceInputController owns the full lifecycle: single app-wide session, permission checks, up to five minutes of capture, cooperative cancellation, temp file cleanup, and draft owner/revision guards so late transcripts cannot overwrite a changed draft. Mobile wires expo-audio recording and metering through useVoiceInputController, with @react-native-ai/apple on iOS for SpeechTranscriber preparation and transcription.

Composer UX swaps the toolbar for dictation controls (cancel, status, confirm) with Reanimated layout transitions; collapsed composers can show a compact dictation strip. The native T3ComposerEditor gains a readOnly mode that blocks edits, paste, and drops while keeping focus and the keyboard. useComposerCommandMenu now keys selection off ownerKey so draft/thread switches reset caret position.

expo-audio and @react-native-ai/apple are added with patches (recording failure reporting, locale resolution, transcription cleanup). app.config.ts registers microphone permission for expo-audio. User and maintainer docs describe behavior and future remote-transcription boundaries. Requires a new native mobile build.

Reviewed by Cursor Bugbot for commit 50fc0ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add offline iPhone voice dictation to mobile composer

  • Adds VoiceInputController in controller.ts to orchestrate the full dictation lifecycle: permission checks, audio session setup, recording, on-device transcription via Apple's Speech framework, transcript commit, and cancellation semantics.
  • Integrates dictation UI into ThreadComposer.tsx and NewTaskDraftScreen.tsx with an animated dictation toolbar, live waveform metering, elapsed-time status, and confirm/cancel controls. Submission, attachments, and pickers are blocked while dictation is busy.
  • Adds a readOnly mode to the native iOS composer editor (T3ComposerEditorView.swift) so the text view blocks edits, paste, and drops while preserving focus and selection during dictation.
  • Patches @react-native-ai/apple@0.12.0 and expo-audio@57.0.4 to fix locale resolution, temp-file cleanup, and recording-failure error reporting.
  • Risk: resolveTranscriptCommit in controller.ts contains stray + tokens in the leftNeedsBoundary/rightNeedsBoundary computation that break TypeScript syntax and will prevent the module from compiling.

Macroscope summarized 50fc0ed.

@coderabbitai

coderabbitaiBot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d020e7f5-6801-48a2-9440-06dce8facce2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Aug 29, 2026
Comment threadpackages/client-runtime/src/voice-input/controller.ts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

disabled={isIncomingShareTransferPending}

Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from NewTaskDraftScreen, whose focus-effect cleanup calls controller.dispose() and discards the recording; disable the control or guard its handler when voiceInput.isBusy is true.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/NewTaskDraftScreen.tsx around line 997:
Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from `NewTaskDraftScreen`, whose focus-effect cleanup calls `controller.dispose()` and discards the recording; disable the control or guard its handler when `voiceInput.isBusy` is true.

onChangeDraftMessage: props.onChangeDraftMessage,
onUpdateInteractionMode: props.onUpdateInteractionMode,
});
const voiceInput = useVoiceInputController({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highthreads/ThreadComposer.tsx:293

When a pending user-input request hides ThreadComposer, an active useVoiceInputController session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from ThreadDetailScreen.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/ThreadComposer.tsx around line 293:
When a pending user-input request hides `ThreadComposer`, an active `useVoiceInputController` session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from `ThreadDetailScreen`.

@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire13.3 KiB13.3 KiB+11 B (+0.1%)15.1 KiB
CodexThread snapshot wire6.9 KiB6.9 KiB−7 B (−0.1%)7.3 KiB
CodexLive turn WebSocket wire6.4 KiB6.5 KiB+18 B (+0.3%)7.8 KiB
CodexLive turn WebSocket decoded55.6 KiB55.6 KiB0 B (0.0%)66.4 KiB
CodexLive turn messages11110 (0.0%)21
ClaudeTotal thread wire13.3 KiB13.3 KiB+32 B (+0.2%)15.1 KiB
ClaudeThread snapshot wire6.9 KiB6.9 KiB+3 B (+0.0%)7.3 KiB
ClaudeLive turn WebSocket wire6.4 KiB6.4 KiB+29 B (+0.4%)7.8 KiB
ClaudeLive turn WebSocket decoded56.4 KiB56.4 KiB+44 B (+0.1%)66.4 KiB
ClaudeLive turn messages1011+1 (+10.0%)21

Baseline: e09b88b · PR result: 50fc0ed · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.4 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
right !== undefined &&
/[A-Za-z0-9([{'"]/.test(right) &&
(left === undefined || /\s/.test(left));
insertion = `${leftNeedsBoundary ? " " : ""}${replacement}${rightNeedsBoundary ? " " : ""}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emoji breaks English transcript spacing

Low Severity

English boundary spacing reads captured.text[index], which is a single UTF-16 code unit. A caret beside an emoji sees a surrogate that fails the letter/punctuation check, so the transcript is inserted without the intended leading or trailing space.

Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 384d7fb. Configure here.

Comment threadapps/mobile/src/features/threads/NewTaskDraftScreen.tsx
@macroscopeapp

macroscopeappBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial user-facing iPhone voice-input workflow spanning native audio capture, local transcription, draft mutation, and composer lifecycle changes, along with new app-level audio defaults. Unresolved lifecycle and emoji-spacing concerns indicate that the behavior needs human review before merging.

Not approved because:

  • 2 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarmingeforce-pushed the t3code/ios-local-voice-transcription branch from 384d7fb to b4c5578CompareAugust 30, 2026 22:08
Comment threadpackages/client-runtime/src/voice-input/controller.ts
Comment threadapps/mobile/src/native/voiceTranscription.ios.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad0c05f. Configure here.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
Comment threadpackages/client-runtime/src/voice-input/controller.ts Outdated
@juliusmarminge
juliusmarminge merged commit 352710d into mainAug 31, 2026
27 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/ios-local-voice-transcription branch August 31, 2026 01:08
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
**Full Changelog**: pingdotgg/t3code@v0.0.37-nightly.20260830.1232...v0.0.37-nightly.20260831.1233
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37-nightly.20260831.1233
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* Remove Messages Glass Lab experiment by @juliusmarminge in pingdotgg/t3code#8599
* Require human review for pull requests changing product defaults by @juliusmarminge in pingdotgg/t3code#8603
* fix(codex): avoid quadratic app-server input buffering by @juliusmarminge in pingdotgg/t3code#8605
* fix(mobile): stabilize iOS header item transitions by @juliusmarminge in pingdotgg/t3code#8607
* chore(mobile): upgrade to Expo SDK 57 by @juliusmarminge in pingdotgg/t3code#8609
* fix(mobile): harden native header toolbar items by @juliusmarminge in pingdotgg/t3code#8611
* fix(server): stop querying Claude context usage after turns by @t3dotgg in pingdotgg/t3code#8610
* chore: vouch ryanrhughes by @t3dotgg in pingdotgg/t3code#8613
* feat(web): attach PDFs, ZIPs, and other files to a turn by @t3dotgg in pingdotgg/t3code#8236
* feat(web): keybinding settings as settings rows by @StiensWout in pingdotgg/t3code#8532
* feat: let an environment publish themes as a file by @ryanrhughes in pingdotgg/t3code#8569
* fix(web): clean up provider settings list and editor by @StiensWout in pingdotgg/t3code#8504
* fix(web): keep project picker popup inside the sidebar by @SunkenInTime in pingdotgg/t3code#8627
* fix(mobile): prevent header overflow and back-button artifacts by @juliusmarminge in pingdotgg/t3code#8624
* fix(server): retry automatic thread title generation by @Bil0000 in pingdotgg/t3code#8087
* fix(client-runtime): refresh edited pull request comments by @Bil0000 in pingdotgg/t3code#8094
* fix(web): four composer spacing defects by @Bil0000 in pingdotgg/t3code#8090
* perf(desktop): skip duplicate browser updates by @Bil0000 in pingdotgg/t3code#8018
* fix(web): render nested markdown images correctly by @flamboh in pingdotgg/t3code#8501
* fix(web): unify activity logs and composer banners by @juliusmarminge in pingdotgg/t3code#8693
* fix(mobile): reduce dev-client reload and Metro startup cost by @juliusmarminge in pingdotgg/t3code#8694
* revert(web): restore previous composer banners by @t3dotgg in pingdotgg/t3code#8733
* test(web): remove tests for unreachable helpers by @t3-code[bot] in pingdotgg/t3code#8738
* feat(mobile): update tool summaries and chat transitions by @juliusmarminge in pingdotgg/t3code#8793
* feat(web): play video attachments in chat by @Bil0000 in pingdotgg/t3code#8688
* fix(web,mobile): snooze menu no longer offers the same wake time twice by @vitalyiegorov in pingdotgg/t3code#8741
* fix(grok): allow model changes in existing threads by @ahmed-besic in pingdotgg/t3code#8392
* feat(mobile): pick, share, and receive files in threads by @t3dotgg in pingdotgg/t3code#8237
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
* fix(web): prevent pull request metadata overlap by @MatthewFeroz in pingdotgg/t3code#8790
## New Contributors
* @ryanrhughes made their first contribution in pingdotgg/t3code#8569
* @ahmed-besic made their first contribution in pingdotgg/t3code#8392
* @MatthewFeroz made their first contribution in pingdotgg/t3code#8790
**Full Changelog**: pingdotgg/t3code@v0.0.36...v0.0.37
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native ChangeChanges the native fingerprint; merging blocks production OTAs until a new store build ships.size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@t3dotgg@juliusmarminge
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(mobile): add offline iPhone voice input - #8614

Merged
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription
Aug 31, 2026
Merged

feat(mobile): add offline iPhone voice input#8614
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription

Conversation

@t3dotgg

@t3dotggt3dotgg commented Aug 29, 2026

Copy link
Copy Markdown
Member

The mobile composer could not use Apple's newer on-device transcription model.

This adds local dictation for supported iPhones on iOS 26 or later. The microphone is available in existing threads and new task drafts, including queued task editing. T3 Code prepares Apple's speech assets for the device language and transcribes up to five minutes of audio on the phone.

The transcript is inserted at the selection where recording started. T3 Code keeps the keyboard open and lets the user review the text before sending it. Canceling keeps the draft unchanged. Temporary audio files are removed when the operation ends.

Tests:

  • All 42 focused tests passed after the Expo SDK 57 rebase.
  • Mobile typecheck and targeted lint passed after the Expo SDK 57 rebase.
  • The pre-rebase Expo SDK 56 version passed an iOS release build and a physical iPhone test.

This change needs a new native mobile build.

UI screenshots are pending permission to capture them.

Built with GPT-5.6 Sol in the Codex harness through T3 Code.


Note

Medium Risk
Touches microphone permissions, audio session lifecycle, and draft mutation timing with new native dependencies; incorrect cancellation or late transcript handling could corrupt composer text or leave sessions locked.

Overview
Adds on-device dictation for supported iPhones (iOS 26+) in the new task and thread composers. Users tap the mic, see recording status and a live waveform, then confirm to transcribe locally and insert text at the caret where recording started—send stays blocked until transcription finishes or is canceled.

A new @t3tools/client-runtime/voice-inputVoiceInputController owns the full lifecycle: single app-wide session, permission checks, up to five minutes of capture, cooperative cancellation, temp file cleanup, and draft owner/revision guards so late transcripts cannot overwrite a changed draft. Mobile wires expo-audio recording and metering through useVoiceInputController, with @react-native-ai/apple on iOS for SpeechTranscriber preparation and transcription.

Composer UX swaps the toolbar for dictation controls (cancel, status, confirm) with Reanimated layout transitions; collapsed composers can show a compact dictation strip. The native T3ComposerEditor gains a readOnly mode that blocks edits, paste, and drops while keeping focus and the keyboard. useComposerCommandMenu now keys selection off ownerKey so draft/thread switches reset caret position.

expo-audio and @react-native-ai/apple are added with patches (recording failure reporting, locale resolution, transcription cleanup). app.config.ts registers microphone permission for expo-audio. User and maintainer docs describe behavior and future remote-transcription boundaries. Requires a new native mobile build.

Reviewed by Cursor Bugbot for commit 50fc0ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add offline iPhone voice dictation to mobile composer

  • Adds VoiceInputController in controller.ts to orchestrate the full dictation lifecycle: permission checks, audio session setup, recording, on-device transcription via Apple's Speech framework, transcript commit, and cancellation semantics.
  • Integrates dictation UI into ThreadComposer.tsx and NewTaskDraftScreen.tsx with an animated dictation toolbar, live waveform metering, elapsed-time status, and confirm/cancel controls. Submission, attachments, and pickers are blocked while dictation is busy.
  • Adds a readOnly mode to the native iOS composer editor (T3ComposerEditorView.swift) so the text view blocks edits, paste, and drops while preserving focus and selection during dictation.
  • Patches @react-native-ai/apple@0.12.0 and expo-audio@57.0.4 to fix locale resolution, temp-file cleanup, and recording-failure error reporting.
  • Risk: resolveTranscriptCommit in controller.ts contains stray + tokens in the leftNeedsBoundary/rightNeedsBoundary computation that break TypeScript syntax and will prevent the module from compiling.

Macroscope summarized 50fc0ed.

@coderabbitai

coderabbitaiBot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d020e7f5-6801-48a2-9440-06dce8facce2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Aug 29, 2026
Comment threadpackages/client-runtime/src/voice-input/controller.ts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

disabled={isIncomingShareTransferPending}

Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from NewTaskDraftScreen, whose focus-effect cleanup calls controller.dispose() and discards the recording; disable the control or guard its handler when voiceInput.isBusy is true.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/NewTaskDraftScreen.tsx around line 997:
Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from `NewTaskDraftScreen`, whose focus-effect cleanup calls `controller.dispose()` and discards the recording; disable the control or guard its handler when `voiceInput.isBusy` is true.

onChangeDraftMessage: props.onChangeDraftMessage,
onUpdateInteractionMode: props.onUpdateInteractionMode,
});
const voiceInput = useVoiceInputController({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highthreads/ThreadComposer.tsx:293

When a pending user-input request hides ThreadComposer, an active useVoiceInputController session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from ThreadDetailScreen.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/ThreadComposer.tsx around line 293:
When a pending user-input request hides `ThreadComposer`, an active `useVoiceInputController` session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from `ThreadDetailScreen`.

@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire13.3 KiB13.3 KiB+11 B (+0.1%)15.1 KiB
CodexThread snapshot wire6.9 KiB6.9 KiB−7 B (−0.1%)7.3 KiB
CodexLive turn WebSocket wire6.4 KiB6.5 KiB+18 B (+0.3%)7.8 KiB
CodexLive turn WebSocket decoded55.6 KiB55.6 KiB0 B (0.0%)66.4 KiB
CodexLive turn messages11110 (0.0%)21
ClaudeTotal thread wire13.3 KiB13.3 KiB+32 B (+0.2%)15.1 KiB
ClaudeThread snapshot wire6.9 KiB6.9 KiB+3 B (+0.0%)7.3 KiB
ClaudeLive turn WebSocket wire6.4 KiB6.4 KiB+29 B (+0.4%)7.8 KiB
ClaudeLive turn WebSocket decoded56.4 KiB56.4 KiB+44 B (+0.1%)66.4 KiB
ClaudeLive turn messages1011+1 (+10.0%)21

Baseline: e09b88b · PR result: 50fc0ed · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.4 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
right !== undefined &&
/[A-Za-z0-9([{'"]/.test(right) &&
(left === undefined || /\s/.test(left));
insertion = `${leftNeedsBoundary ? " " : ""}${replacement}${rightNeedsBoundary ? " " : ""}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emoji breaks English transcript spacing

Low Severity

English boundary spacing reads captured.text[index], which is a single UTF-16 code unit. A caret beside an emoji sees a surrogate that fails the letter/punctuation check, so the transcript is inserted without the intended leading or trailing space.

Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 384d7fb. Configure here.

Comment threadapps/mobile/src/features/threads/NewTaskDraftScreen.tsx
@macroscopeapp

macroscopeappBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial user-facing iPhone voice-input workflow spanning native audio capture, local transcription, draft mutation, and composer lifecycle changes, along with new app-level audio defaults. Unresolved lifecycle and emoji-spacing concerns indicate that the behavior needs human review before merging.

Not approved because:

  • 2 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarmingeforce-pushed the t3code/ios-local-voice-transcription branch from 384d7fb to b4c5578CompareAugust 30, 2026 22:08
Comment threadpackages/client-runtime/src/voice-input/controller.ts
Comment threadapps/mobile/src/native/voiceTranscription.ios.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad0c05f. Configure here.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
Comment threadpackages/client-runtime/src/voice-input/controller.ts Outdated
@juliusmarminge
juliusmarminge merged commit 352710d into mainAug 31, 2026
27 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/ios-local-voice-transcription branch August 31, 2026 01:08
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
**Full Changelog**: pingdotgg/t3code@v0.0.37-nightly.20260830.1232...v0.0.37-nightly.20260831.1233
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37-nightly.20260831.1233
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* Remove Messages Glass Lab experiment by @juliusmarminge in pingdotgg/t3code#8599
* Require human review for pull requests changing product defaults by @juliusmarminge in pingdotgg/t3code#8603
* fix(codex): avoid quadratic app-server input buffering by @juliusmarminge in pingdotgg/t3code#8605
* fix(mobile): stabilize iOS header item transitions by @juliusmarminge in pingdotgg/t3code#8607
* chore(mobile): upgrade to Expo SDK 57 by @juliusmarminge in pingdotgg/t3code#8609
* fix(mobile): harden native header toolbar items by @juliusmarminge in pingdotgg/t3code#8611
* fix(server): stop querying Claude context usage after turns by @t3dotgg in pingdotgg/t3code#8610
* chore: vouch ryanrhughes by @t3dotgg in pingdotgg/t3code#8613
* feat(web): attach PDFs, ZIPs, and other files to a turn by @t3dotgg in pingdotgg/t3code#8236
* feat(web): keybinding settings as settings rows by @StiensWout in pingdotgg/t3code#8532
* feat: let an environment publish themes as a file by @ryanrhughes in pingdotgg/t3code#8569
* fix(web): clean up provider settings list and editor by @StiensWout in pingdotgg/t3code#8504
* fix(web): keep project picker popup inside the sidebar by @SunkenInTime in pingdotgg/t3code#8627
* fix(mobile): prevent header overflow and back-button artifacts by @juliusmarminge in pingdotgg/t3code#8624
* fix(server): retry automatic thread title generation by @Bil0000 in pingdotgg/t3code#8087
* fix(client-runtime): refresh edited pull request comments by @Bil0000 in pingdotgg/t3code#8094
* fix(web): four composer spacing defects by @Bil0000 in pingdotgg/t3code#8090
* perf(desktop): skip duplicate browser updates by @Bil0000 in pingdotgg/t3code#8018
* fix(web): render nested markdown images correctly by @flamboh in pingdotgg/t3code#8501
* fix(web): unify activity logs and composer banners by @juliusmarminge in pingdotgg/t3code#8693
* fix(mobile): reduce dev-client reload and Metro startup cost by @juliusmarminge in pingdotgg/t3code#8694
* revert(web): restore previous composer banners by @t3dotgg in pingdotgg/t3code#8733
* test(web): remove tests for unreachable helpers by @t3-code[bot] in pingdotgg/t3code#8738
* feat(mobile): update tool summaries and chat transitions by @juliusmarminge in pingdotgg/t3code#8793
* feat(web): play video attachments in chat by @Bil0000 in pingdotgg/t3code#8688
* fix(web,mobile): snooze menu no longer offers the same wake time twice by @vitalyiegorov in pingdotgg/t3code#8741
* fix(grok): allow model changes in existing threads by @ahmed-besic in pingdotgg/t3code#8392
* feat(mobile): pick, share, and receive files in threads by @t3dotgg in pingdotgg/t3code#8237
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
* fix(web): prevent pull request metadata overlap by @MatthewFeroz in pingdotgg/t3code#8790
## New Contributors
* @ryanrhughes made their first contribution in pingdotgg/t3code#8569
* @ahmed-besic made their first contribution in pingdotgg/t3code#8392
* @MatthewFeroz made their first contribution in pingdotgg/t3code#8790
**Full Changelog**: pingdotgg/t3code@v0.0.36...v0.0.37
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native ChangeChanges the native fingerprint; merging blocks production OTAs until a new store build ships.size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@t3dotgg@juliusmarminge
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat(mobile): add offline iPhone voice input - #8614

Merged
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription
Aug 31, 2026
Merged

feat(mobile): add offline iPhone voice input#8614
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription

Conversation

@t3dotgg

@t3dotggt3dotgg commented Aug 29, 2026

Copy link
Copy Markdown
Member

The mobile composer could not use Apple's newer on-device transcription model.

This adds local dictation for supported iPhones on iOS 26 or later. The microphone is available in existing threads and new task drafts, including queued task editing. T3 Code prepares Apple's speech assets for the device language and transcribes up to five minutes of audio on the phone.

The transcript is inserted at the selection where recording started. T3 Code keeps the keyboard open and lets the user review the text before sending it. Canceling keeps the draft unchanged. Temporary audio files are removed when the operation ends.

Tests:

  • All 42 focused tests passed after the Expo SDK 57 rebase.
  • Mobile typecheck and targeted lint passed after the Expo SDK 57 rebase.
  • The pre-rebase Expo SDK 56 version passed an iOS release build and a physical iPhone test.

This change needs a new native mobile build.

UI screenshots are pending permission to capture them.

Built with GPT-5.6 Sol in the Codex harness through T3 Code.


Note

Medium Risk
Touches microphone permissions, audio session lifecycle, and draft mutation timing with new native dependencies; incorrect cancellation or late transcript handling could corrupt composer text or leave sessions locked.

Overview
Adds on-device dictation for supported iPhones (iOS 26+) in the new task and thread composers. Users tap the mic, see recording status and a live waveform, then confirm to transcribe locally and insert text at the caret where recording started—send stays blocked until transcription finishes or is canceled.

A new @t3tools/client-runtime/voice-inputVoiceInputController owns the full lifecycle: single app-wide session, permission checks, up to five minutes of capture, cooperative cancellation, temp file cleanup, and draft owner/revision guards so late transcripts cannot overwrite a changed draft. Mobile wires expo-audio recording and metering through useVoiceInputController, with @react-native-ai/apple on iOS for SpeechTranscriber preparation and transcription.

Composer UX swaps the toolbar for dictation controls (cancel, status, confirm) with Reanimated layout transitions; collapsed composers can show a compact dictation strip. The native T3ComposerEditor gains a readOnly mode that blocks edits, paste, and drops while keeping focus and the keyboard. useComposerCommandMenu now keys selection off ownerKey so draft/thread switches reset caret position.

expo-audio and @react-native-ai/apple are added with patches (recording failure reporting, locale resolution, transcription cleanup). app.config.ts registers microphone permission for expo-audio. User and maintainer docs describe behavior and future remote-transcription boundaries. Requires a new native mobile build.

Reviewed by Cursor Bugbot for commit 50fc0ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add offline iPhone voice dictation to mobile composer

  • Adds VoiceInputController in controller.ts to orchestrate the full dictation lifecycle: permission checks, audio session setup, recording, on-device transcription via Apple's Speech framework, transcript commit, and cancellation semantics.
  • Integrates dictation UI into ThreadComposer.tsx and NewTaskDraftScreen.tsx with an animated dictation toolbar, live waveform metering, elapsed-time status, and confirm/cancel controls. Submission, attachments, and pickers are blocked while dictation is busy.
  • Adds a readOnly mode to the native iOS composer editor (T3ComposerEditorView.swift) so the text view blocks edits, paste, and drops while preserving focus and selection during dictation.
  • Patches @react-native-ai/apple@0.12.0 and expo-audio@57.0.4 to fix locale resolution, temp-file cleanup, and recording-failure error reporting.
  • Risk: resolveTranscriptCommit in controller.ts contains stray + tokens in the leftNeedsBoundary/rightNeedsBoundary computation that break TypeScript syntax and will prevent the module from compiling.

Macroscope summarized 50fc0ed.

@coderabbitai

coderabbitaiBot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d020e7f5-6801-48a2-9440-06dce8facce2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Aug 29, 2026
Comment threadpackages/client-runtime/src/voice-input/controller.ts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

disabled={isIncomingShareTransferPending}

Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from NewTaskDraftScreen, whose focus-effect cleanup calls controller.dispose() and discards the recording; disable the control or guard its handler when voiceInput.isBusy is true.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/NewTaskDraftScreen.tsx around line 997:
Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from `NewTaskDraftScreen`, whose focus-effect cleanup calls `controller.dispose()` and discards the recording; disable the control or guard its handler when `voiceInput.isBusy` is true.

onChangeDraftMessage: props.onChangeDraftMessage,
onUpdateInteractionMode: props.onUpdateInteractionMode,
});
const voiceInput = useVoiceInputController({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highthreads/ThreadComposer.tsx:293

When a pending user-input request hides ThreadComposer, an active useVoiceInputController session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from ThreadDetailScreen.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/ThreadComposer.tsx around line 293:
When a pending user-input request hides `ThreadComposer`, an active `useVoiceInputController` session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from `ThreadDetailScreen`.

@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire13.3 KiB13.3 KiB+11 B (+0.1%)15.1 KiB
CodexThread snapshot wire6.9 KiB6.9 KiB−7 B (−0.1%)7.3 KiB
CodexLive turn WebSocket wire6.4 KiB6.5 KiB+18 B (+0.3%)7.8 KiB
CodexLive turn WebSocket decoded55.6 KiB55.6 KiB0 B (0.0%)66.4 KiB
CodexLive turn messages11110 (0.0%)21
ClaudeTotal thread wire13.3 KiB13.3 KiB+32 B (+0.2%)15.1 KiB
ClaudeThread snapshot wire6.9 KiB6.9 KiB+3 B (+0.0%)7.3 KiB
ClaudeLive turn WebSocket wire6.4 KiB6.4 KiB+29 B (+0.4%)7.8 KiB
ClaudeLive turn WebSocket decoded56.4 KiB56.4 KiB+44 B (+0.1%)66.4 KiB
ClaudeLive turn messages1011+1 (+10.0%)21

Baseline: e09b88b · PR result: 50fc0ed · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.4 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
right !== undefined &&
/[A-Za-z0-9([{'"]/.test(right) &&
(left === undefined || /\s/.test(left));
insertion = `${leftNeedsBoundary ? " " : ""}${replacement}${rightNeedsBoundary ? " " : ""}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emoji breaks English transcript spacing

Low Severity

English boundary spacing reads captured.text[index], which is a single UTF-16 code unit. A caret beside an emoji sees a surrogate that fails the letter/punctuation check, so the transcript is inserted without the intended leading or trailing space.

Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 384d7fb. Configure here.

Comment threadapps/mobile/src/features/threads/NewTaskDraftScreen.tsx
@macroscopeapp

macroscopeappBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial user-facing iPhone voice-input workflow spanning native audio capture, local transcription, draft mutation, and composer lifecycle changes, along with new app-level audio defaults. Unresolved lifecycle and emoji-spacing concerns indicate that the behavior needs human review before merging.

Not approved because:

  • 2 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarmingeforce-pushed the t3code/ios-local-voice-transcription branch from 384d7fb to b4c5578CompareAugust 30, 2026 22:08
Comment threadpackages/client-runtime/src/voice-input/controller.ts
Comment threadapps/mobile/src/native/voiceTranscription.ios.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad0c05f. Configure here.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
Comment threadpackages/client-runtime/src/voice-input/controller.ts Outdated
@juliusmarminge
juliusmarminge merged commit 352710d into mainAug 31, 2026
27 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/ios-local-voice-transcription branch August 31, 2026 01:08
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
**Full Changelog**: pingdotgg/t3code@v0.0.37-nightly.20260830.1232...v0.0.37-nightly.20260831.1233
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37-nightly.20260831.1233
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* Remove Messages Glass Lab experiment by @juliusmarminge in pingdotgg/t3code#8599
* Require human review for pull requests changing product defaults by @juliusmarminge in pingdotgg/t3code#8603
* fix(codex): avoid quadratic app-server input buffering by @juliusmarminge in pingdotgg/t3code#8605
* fix(mobile): stabilize iOS header item transitions by @juliusmarminge in pingdotgg/t3code#8607
* chore(mobile): upgrade to Expo SDK 57 by @juliusmarminge in pingdotgg/t3code#8609
* fix(mobile): harden native header toolbar items by @juliusmarminge in pingdotgg/t3code#8611
* fix(server): stop querying Claude context usage after turns by @t3dotgg in pingdotgg/t3code#8610
* chore: vouch ryanrhughes by @t3dotgg in pingdotgg/t3code#8613
* feat(web): attach PDFs, ZIPs, and other files to a turn by @t3dotgg in pingdotgg/t3code#8236
* feat(web): keybinding settings as settings rows by @StiensWout in pingdotgg/t3code#8532
* feat: let an environment publish themes as a file by @ryanrhughes in pingdotgg/t3code#8569
* fix(web): clean up provider settings list and editor by @StiensWout in pingdotgg/t3code#8504
* fix(web): keep project picker popup inside the sidebar by @SunkenInTime in pingdotgg/t3code#8627
* fix(mobile): prevent header overflow and back-button artifacts by @juliusmarminge in pingdotgg/t3code#8624
* fix(server): retry automatic thread title generation by @Bil0000 in pingdotgg/t3code#8087
* fix(client-runtime): refresh edited pull request comments by @Bil0000 in pingdotgg/t3code#8094
* fix(web): four composer spacing defects by @Bil0000 in pingdotgg/t3code#8090
* perf(desktop): skip duplicate browser updates by @Bil0000 in pingdotgg/t3code#8018
* fix(web): render nested markdown images correctly by @flamboh in pingdotgg/t3code#8501
* fix(web): unify activity logs and composer banners by @juliusmarminge in pingdotgg/t3code#8693
* fix(mobile): reduce dev-client reload and Metro startup cost by @juliusmarminge in pingdotgg/t3code#8694
* revert(web): restore previous composer banners by @t3dotgg in pingdotgg/t3code#8733
* test(web): remove tests for unreachable helpers by @t3-code[bot] in pingdotgg/t3code#8738
* feat(mobile): update tool summaries and chat transitions by @juliusmarminge in pingdotgg/t3code#8793
* feat(web): play video attachments in chat by @Bil0000 in pingdotgg/t3code#8688
* fix(web,mobile): snooze menu no longer offers the same wake time twice by @vitalyiegorov in pingdotgg/t3code#8741
* fix(grok): allow model changes in existing threads by @ahmed-besic in pingdotgg/t3code#8392
* feat(mobile): pick, share, and receive files in threads by @t3dotgg in pingdotgg/t3code#8237
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
* fix(web): prevent pull request metadata overlap by @MatthewFeroz in pingdotgg/t3code#8790
## New Contributors
* @ryanrhughes made their first contribution in pingdotgg/t3code#8569
* @ahmed-besic made their first contribution in pingdotgg/t3code#8392
* @MatthewFeroz made their first contribution in pingdotgg/t3code#8790
**Full Changelog**: pingdotgg/t3code@v0.0.36...v0.0.37
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native ChangeChanges the native fingerprint; merging blocks production OTAs until a new store build ships.size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@t3dotgg@juliusmarminge
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(mobile): add offline iPhone voice input - #8614

Merged
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription
Aug 31, 2026
Merged

feat(mobile): add offline iPhone voice input#8614
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription

Conversation

@t3dotgg

@t3dotggt3dotgg commented Aug 29, 2026

Copy link
Copy Markdown
Member

The mobile composer could not use Apple's newer on-device transcription model.

This adds local dictation for supported iPhones on iOS 26 or later. The microphone is available in existing threads and new task drafts, including queued task editing. T3 Code prepares Apple's speech assets for the device language and transcribes up to five minutes of audio on the phone.

The transcript is inserted at the selection where recording started. T3 Code keeps the keyboard open and lets the user review the text before sending it. Canceling keeps the draft unchanged. Temporary audio files are removed when the operation ends.

Tests:

  • All 42 focused tests passed after the Expo SDK 57 rebase.
  • Mobile typecheck and targeted lint passed after the Expo SDK 57 rebase.
  • The pre-rebase Expo SDK 56 version passed an iOS release build and a physical iPhone test.

This change needs a new native mobile build.

UI screenshots are pending permission to capture them.

Built with GPT-5.6 Sol in the Codex harness through T3 Code.


Note

Medium Risk
Touches microphone permissions, audio session lifecycle, and draft mutation timing with new native dependencies; incorrect cancellation or late transcript handling could corrupt composer text or leave sessions locked.

Overview
Adds on-device dictation for supported iPhones (iOS 26+) in the new task and thread composers. Users tap the mic, see recording status and a live waveform, then confirm to transcribe locally and insert text at the caret where recording started—send stays blocked until transcription finishes or is canceled.

A new @t3tools/client-runtime/voice-inputVoiceInputController owns the full lifecycle: single app-wide session, permission checks, up to five minutes of capture, cooperative cancellation, temp file cleanup, and draft owner/revision guards so late transcripts cannot overwrite a changed draft. Mobile wires expo-audio recording and metering through useVoiceInputController, with @react-native-ai/apple on iOS for SpeechTranscriber preparation and transcription.

Composer UX swaps the toolbar for dictation controls (cancel, status, confirm) with Reanimated layout transitions; collapsed composers can show a compact dictation strip. The native T3ComposerEditor gains a readOnly mode that blocks edits, paste, and drops while keeping focus and the keyboard. useComposerCommandMenu now keys selection off ownerKey so draft/thread switches reset caret position.

expo-audio and @react-native-ai/apple are added with patches (recording failure reporting, locale resolution, transcription cleanup). app.config.ts registers microphone permission for expo-audio. User and maintainer docs describe behavior and future remote-transcription boundaries. Requires a new native mobile build.

Reviewed by Cursor Bugbot for commit 50fc0ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add offline iPhone voice dictation to mobile composer

  • Adds VoiceInputController in controller.ts to orchestrate the full dictation lifecycle: permission checks, audio session setup, recording, on-device transcription via Apple's Speech framework, transcript commit, and cancellation semantics.
  • Integrates dictation UI into ThreadComposer.tsx and NewTaskDraftScreen.tsx with an animated dictation toolbar, live waveform metering, elapsed-time status, and confirm/cancel controls. Submission, attachments, and pickers are blocked while dictation is busy.
  • Adds a readOnly mode to the native iOS composer editor (T3ComposerEditorView.swift) so the text view blocks edits, paste, and drops while preserving focus and selection during dictation.
  • Patches @react-native-ai/apple@0.12.0 and expo-audio@57.0.4 to fix locale resolution, temp-file cleanup, and recording-failure error reporting.
  • Risk: resolveTranscriptCommit in controller.ts contains stray + tokens in the leftNeedsBoundary/rightNeedsBoundary computation that break TypeScript syntax and will prevent the module from compiling.

Macroscope summarized 50fc0ed.

@coderabbitai

coderabbitaiBot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d020e7f5-6801-48a2-9440-06dce8facce2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Aug 29, 2026
Comment threadpackages/client-runtime/src/voice-input/controller.ts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

disabled={isIncomingShareTransferPending}

Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from NewTaskDraftScreen, whose focus-effect cleanup calls controller.dispose() and discards the recording; disable the control or guard its handler when voiceInput.isBusy is true.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/NewTaskDraftScreen.tsx around line 997:
Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from `NewTaskDraftScreen`, whose focus-effect cleanup calls `controller.dispose()` and discards the recording; disable the control or guard its handler when `voiceInput.isBusy` is true.

onChangeDraftMessage: props.onChangeDraftMessage,
onUpdateInteractionMode: props.onUpdateInteractionMode,
});
const voiceInput = useVoiceInputController({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highthreads/ThreadComposer.tsx:293

When a pending user-input request hides ThreadComposer, an active useVoiceInputController session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from ThreadDetailScreen.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/ThreadComposer.tsx around line 293:
When a pending user-input request hides `ThreadComposer`, an active `useVoiceInputController` session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from `ThreadDetailScreen`.

@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire13.3 KiB13.3 KiB+11 B (+0.1%)15.1 KiB
CodexThread snapshot wire6.9 KiB6.9 KiB−7 B (−0.1%)7.3 KiB
CodexLive turn WebSocket wire6.4 KiB6.5 KiB+18 B (+0.3%)7.8 KiB
CodexLive turn WebSocket decoded55.6 KiB55.6 KiB0 B (0.0%)66.4 KiB
CodexLive turn messages11110 (0.0%)21
ClaudeTotal thread wire13.3 KiB13.3 KiB+32 B (+0.2%)15.1 KiB
ClaudeThread snapshot wire6.9 KiB6.9 KiB+3 B (+0.0%)7.3 KiB
ClaudeLive turn WebSocket wire6.4 KiB6.4 KiB+29 B (+0.4%)7.8 KiB
ClaudeLive turn WebSocket decoded56.4 KiB56.4 KiB+44 B (+0.1%)66.4 KiB
ClaudeLive turn messages1011+1 (+10.0%)21

Baseline: e09b88b · PR result: 50fc0ed · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.4 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
right !== undefined &&
/[A-Za-z0-9([{'"]/.test(right) &&
(left === undefined || /\s/.test(left));
insertion = `${leftNeedsBoundary ? " " : ""}${replacement}${rightNeedsBoundary ? " " : ""}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emoji breaks English transcript spacing

Low Severity

English boundary spacing reads captured.text[index], which is a single UTF-16 code unit. A caret beside an emoji sees a surrogate that fails the letter/punctuation check, so the transcript is inserted without the intended leading or trailing space.

Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 384d7fb. Configure here.

Comment threadapps/mobile/src/features/threads/NewTaskDraftScreen.tsx
@macroscopeapp

macroscopeappBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial user-facing iPhone voice-input workflow spanning native audio capture, local transcription, draft mutation, and composer lifecycle changes, along with new app-level audio defaults. Unresolved lifecycle and emoji-spacing concerns indicate that the behavior needs human review before merging.

Not approved because:

  • 2 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarmingeforce-pushed the t3code/ios-local-voice-transcription branch from 384d7fb to b4c5578CompareAugust 30, 2026 22:08
Comment threadpackages/client-runtime/src/voice-input/controller.ts
Comment threadapps/mobile/src/native/voiceTranscription.ios.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad0c05f. Configure here.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
Comment threadpackages/client-runtime/src/voice-input/controller.ts Outdated
@juliusmarminge
juliusmarminge merged commit 352710d into mainAug 31, 2026
27 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/ios-local-voice-transcription branch August 31, 2026 01:08
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
**Full Changelog**: pingdotgg/t3code@v0.0.37-nightly.20260830.1232...v0.0.37-nightly.20260831.1233
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37-nightly.20260831.1233
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* Remove Messages Glass Lab experiment by @juliusmarminge in pingdotgg/t3code#8599
* Require human review for pull requests changing product defaults by @juliusmarminge in pingdotgg/t3code#8603
* fix(codex): avoid quadratic app-server input buffering by @juliusmarminge in pingdotgg/t3code#8605
* fix(mobile): stabilize iOS header item transitions by @juliusmarminge in pingdotgg/t3code#8607
* chore(mobile): upgrade to Expo SDK 57 by @juliusmarminge in pingdotgg/t3code#8609
* fix(mobile): harden native header toolbar items by @juliusmarminge in pingdotgg/t3code#8611
* fix(server): stop querying Claude context usage after turns by @t3dotgg in pingdotgg/t3code#8610
* chore: vouch ryanrhughes by @t3dotgg in pingdotgg/t3code#8613
* feat(web): attach PDFs, ZIPs, and other files to a turn by @t3dotgg in pingdotgg/t3code#8236
* feat(web): keybinding settings as settings rows by @StiensWout in pingdotgg/t3code#8532
* feat: let an environment publish themes as a file by @ryanrhughes in pingdotgg/t3code#8569
* fix(web): clean up provider settings list and editor by @StiensWout in pingdotgg/t3code#8504
* fix(web): keep project picker popup inside the sidebar by @SunkenInTime in pingdotgg/t3code#8627
* fix(mobile): prevent header overflow and back-button artifacts by @juliusmarminge in pingdotgg/t3code#8624
* fix(server): retry automatic thread title generation by @Bil0000 in pingdotgg/t3code#8087
* fix(client-runtime): refresh edited pull request comments by @Bil0000 in pingdotgg/t3code#8094
* fix(web): four composer spacing defects by @Bil0000 in pingdotgg/t3code#8090
* perf(desktop): skip duplicate browser updates by @Bil0000 in pingdotgg/t3code#8018
* fix(web): render nested markdown images correctly by @flamboh in pingdotgg/t3code#8501
* fix(web): unify activity logs and composer banners by @juliusmarminge in pingdotgg/t3code#8693
* fix(mobile): reduce dev-client reload and Metro startup cost by @juliusmarminge in pingdotgg/t3code#8694
* revert(web): restore previous composer banners by @t3dotgg in pingdotgg/t3code#8733
* test(web): remove tests for unreachable helpers by @t3-code[bot] in pingdotgg/t3code#8738
* feat(mobile): update tool summaries and chat transitions by @juliusmarminge in pingdotgg/t3code#8793
* feat(web): play video attachments in chat by @Bil0000 in pingdotgg/t3code#8688
* fix(web,mobile): snooze menu no longer offers the same wake time twice by @vitalyiegorov in pingdotgg/t3code#8741
* fix(grok): allow model changes in existing threads by @ahmed-besic in pingdotgg/t3code#8392
* feat(mobile): pick, share, and receive files in threads by @t3dotgg in pingdotgg/t3code#8237
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
* fix(web): prevent pull request metadata overlap by @MatthewFeroz in pingdotgg/t3code#8790
## New Contributors
* @ryanrhughes made their first contribution in pingdotgg/t3code#8569
* @ahmed-besic made their first contribution in pingdotgg/t3code#8392
* @MatthewFeroz made their first contribution in pingdotgg/t3code#8790
**Full Changelog**: pingdotgg/t3code@v0.0.36...v0.0.37
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native ChangeChanges the native fingerprint; merging blocks production OTAs until a new store build ships.size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@t3dotgg@juliusmarminge
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(mobile): add offline iPhone voice input - #8614

Merged
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription
Aug 31, 2026
Merged

feat(mobile): add offline iPhone voice input#8614
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription

Conversation

@t3dotgg

@t3dotggt3dotgg commented Aug 29, 2026

Copy link
Copy Markdown
Member

The mobile composer could not use Apple's newer on-device transcription model.

This adds local dictation for supported iPhones on iOS 26 or later. The microphone is available in existing threads and new task drafts, including queued task editing. T3 Code prepares Apple's speech assets for the device language and transcribes up to five minutes of audio on the phone.

The transcript is inserted at the selection where recording started. T3 Code keeps the keyboard open and lets the user review the text before sending it. Canceling keeps the draft unchanged. Temporary audio files are removed when the operation ends.

Tests:

  • All 42 focused tests passed after the Expo SDK 57 rebase.
  • Mobile typecheck and targeted lint passed after the Expo SDK 57 rebase.
  • The pre-rebase Expo SDK 56 version passed an iOS release build and a physical iPhone test.

This change needs a new native mobile build.

UI screenshots are pending permission to capture them.

Built with GPT-5.6 Sol in the Codex harness through T3 Code.


Note

Medium Risk
Touches microphone permissions, audio session lifecycle, and draft mutation timing with new native dependencies; incorrect cancellation or late transcript handling could corrupt composer text or leave sessions locked.

Overview
Adds on-device dictation for supported iPhones (iOS 26+) in the new task and thread composers. Users tap the mic, see recording status and a live waveform, then confirm to transcribe locally and insert text at the caret where recording started—send stays blocked until transcription finishes or is canceled.

A new @t3tools/client-runtime/voice-inputVoiceInputController owns the full lifecycle: single app-wide session, permission checks, up to five minutes of capture, cooperative cancellation, temp file cleanup, and draft owner/revision guards so late transcripts cannot overwrite a changed draft. Mobile wires expo-audio recording and metering through useVoiceInputController, with @react-native-ai/apple on iOS for SpeechTranscriber preparation and transcription.

Composer UX swaps the toolbar for dictation controls (cancel, status, confirm) with Reanimated layout transitions; collapsed composers can show a compact dictation strip. The native T3ComposerEditor gains a readOnly mode that blocks edits, paste, and drops while keeping focus and the keyboard. useComposerCommandMenu now keys selection off ownerKey so draft/thread switches reset caret position.

expo-audio and @react-native-ai/apple are added with patches (recording failure reporting, locale resolution, transcription cleanup). app.config.ts registers microphone permission for expo-audio. User and maintainer docs describe behavior and future remote-transcription boundaries. Requires a new native mobile build.

Reviewed by Cursor Bugbot for commit 50fc0ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add offline iPhone voice dictation to mobile composer

  • Adds VoiceInputController in controller.ts to orchestrate the full dictation lifecycle: permission checks, audio session setup, recording, on-device transcription via Apple's Speech framework, transcript commit, and cancellation semantics.
  • Integrates dictation UI into ThreadComposer.tsx and NewTaskDraftScreen.tsx with an animated dictation toolbar, live waveform metering, elapsed-time status, and confirm/cancel controls. Submission, attachments, and pickers are blocked while dictation is busy.
  • Adds a readOnly mode to the native iOS composer editor (T3ComposerEditorView.swift) so the text view blocks edits, paste, and drops while preserving focus and selection during dictation.
  • Patches @react-native-ai/apple@0.12.0 and expo-audio@57.0.4 to fix locale resolution, temp-file cleanup, and recording-failure error reporting.
  • Risk: resolveTranscriptCommit in controller.ts contains stray + tokens in the leftNeedsBoundary/rightNeedsBoundary computation that break TypeScript syntax and will prevent the module from compiling.

Macroscope summarized 50fc0ed.

@coderabbitai

coderabbitaiBot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d020e7f5-6801-48a2-9440-06dce8facce2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Aug 29, 2026
Comment threadpackages/client-runtime/src/voice-input/controller.ts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

disabled={isIncomingShareTransferPending}

Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from NewTaskDraftScreen, whose focus-effect cleanup calls controller.dispose() and discards the recording; disable the control or guard its handler when voiceInput.isBusy is true.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/NewTaskDraftScreen.tsx around line 997:
Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from `NewTaskDraftScreen`, whose focus-effect cleanup calls `controller.dispose()` and discards the recording; disable the control or guard its handler when `voiceInput.isBusy` is true.

onChangeDraftMessage: props.onChangeDraftMessage,
onUpdateInteractionMode: props.onUpdateInteractionMode,
});
const voiceInput = useVoiceInputController({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highthreads/ThreadComposer.tsx:293

When a pending user-input request hides ThreadComposer, an active useVoiceInputController session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from ThreadDetailScreen.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/ThreadComposer.tsx around line 293:
When a pending user-input request hides `ThreadComposer`, an active `useVoiceInputController` session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from `ThreadDetailScreen`.

@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire13.3 KiB13.3 KiB+11 B (+0.1%)15.1 KiB
CodexThread snapshot wire6.9 KiB6.9 KiB−7 B (−0.1%)7.3 KiB
CodexLive turn WebSocket wire6.4 KiB6.5 KiB+18 B (+0.3%)7.8 KiB
CodexLive turn WebSocket decoded55.6 KiB55.6 KiB0 B (0.0%)66.4 KiB
CodexLive turn messages11110 (0.0%)21
ClaudeTotal thread wire13.3 KiB13.3 KiB+32 B (+0.2%)15.1 KiB
ClaudeThread snapshot wire6.9 KiB6.9 KiB+3 B (+0.0%)7.3 KiB
ClaudeLive turn WebSocket wire6.4 KiB6.4 KiB+29 B (+0.4%)7.8 KiB
ClaudeLive turn WebSocket decoded56.4 KiB56.4 KiB+44 B (+0.1%)66.4 KiB
ClaudeLive turn messages1011+1 (+10.0%)21

Baseline: e09b88b · PR result: 50fc0ed · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.4 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
right !== undefined &&
/[A-Za-z0-9([{'"]/.test(right) &&
(left === undefined || /\s/.test(left));
insertion = `${leftNeedsBoundary ? " " : ""}${replacement}${rightNeedsBoundary ? " " : ""}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emoji breaks English transcript spacing

Low Severity

English boundary spacing reads captured.text[index], which is a single UTF-16 code unit. A caret beside an emoji sees a surrogate that fails the letter/punctuation check, so the transcript is inserted without the intended leading or trailing space.

Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 384d7fb. Configure here.

Comment threadapps/mobile/src/features/threads/NewTaskDraftScreen.tsx
@macroscopeapp

macroscopeappBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial user-facing iPhone voice-input workflow spanning native audio capture, local transcription, draft mutation, and composer lifecycle changes, along with new app-level audio defaults. Unresolved lifecycle and emoji-spacing concerns indicate that the behavior needs human review before merging.

Not approved because:

  • 2 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarmingeforce-pushed the t3code/ios-local-voice-transcription branch from 384d7fb to b4c5578CompareAugust 30, 2026 22:08
Comment threadpackages/client-runtime/src/voice-input/controller.ts
Comment threadapps/mobile/src/native/voiceTranscription.ios.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad0c05f. Configure here.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
Comment threadpackages/client-runtime/src/voice-input/controller.ts Outdated
@juliusmarminge
juliusmarminge merged commit 352710d into mainAug 31, 2026
27 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/ios-local-voice-transcription branch August 31, 2026 01:08
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
**Full Changelog**: pingdotgg/t3code@v0.0.37-nightly.20260830.1232...v0.0.37-nightly.20260831.1233
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37-nightly.20260831.1233
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* Remove Messages Glass Lab experiment by @juliusmarminge in pingdotgg/t3code#8599
* Require human review for pull requests changing product defaults by @juliusmarminge in pingdotgg/t3code#8603
* fix(codex): avoid quadratic app-server input buffering by @juliusmarminge in pingdotgg/t3code#8605
* fix(mobile): stabilize iOS header item transitions by @juliusmarminge in pingdotgg/t3code#8607
* chore(mobile): upgrade to Expo SDK 57 by @juliusmarminge in pingdotgg/t3code#8609
* fix(mobile): harden native header toolbar items by @juliusmarminge in pingdotgg/t3code#8611
* fix(server): stop querying Claude context usage after turns by @t3dotgg in pingdotgg/t3code#8610
* chore: vouch ryanrhughes by @t3dotgg in pingdotgg/t3code#8613
* feat(web): attach PDFs, ZIPs, and other files to a turn by @t3dotgg in pingdotgg/t3code#8236
* feat(web): keybinding settings as settings rows by @StiensWout in pingdotgg/t3code#8532
* feat: let an environment publish themes as a file by @ryanrhughes in pingdotgg/t3code#8569
* fix(web): clean up provider settings list and editor by @StiensWout in pingdotgg/t3code#8504
* fix(web): keep project picker popup inside the sidebar by @SunkenInTime in pingdotgg/t3code#8627
* fix(mobile): prevent header overflow and back-button artifacts by @juliusmarminge in pingdotgg/t3code#8624
* fix(server): retry automatic thread title generation by @Bil0000 in pingdotgg/t3code#8087
* fix(client-runtime): refresh edited pull request comments by @Bil0000 in pingdotgg/t3code#8094
* fix(web): four composer spacing defects by @Bil0000 in pingdotgg/t3code#8090
* perf(desktop): skip duplicate browser updates by @Bil0000 in pingdotgg/t3code#8018
* fix(web): render nested markdown images correctly by @flamboh in pingdotgg/t3code#8501
* fix(web): unify activity logs and composer banners by @juliusmarminge in pingdotgg/t3code#8693
* fix(mobile): reduce dev-client reload and Metro startup cost by @juliusmarminge in pingdotgg/t3code#8694
* revert(web): restore previous composer banners by @t3dotgg in pingdotgg/t3code#8733
* test(web): remove tests for unreachable helpers by @t3-code[bot] in pingdotgg/t3code#8738
* feat(mobile): update tool summaries and chat transitions by @juliusmarminge in pingdotgg/t3code#8793
* feat(web): play video attachments in chat by @Bil0000 in pingdotgg/t3code#8688
* fix(web,mobile): snooze menu no longer offers the same wake time twice by @vitalyiegorov in pingdotgg/t3code#8741
* fix(grok): allow model changes in existing threads by @ahmed-besic in pingdotgg/t3code#8392
* feat(mobile): pick, share, and receive files in threads by @t3dotgg in pingdotgg/t3code#8237
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
* fix(web): prevent pull request metadata overlap by @MatthewFeroz in pingdotgg/t3code#8790
## New Contributors
* @ryanrhughes made their first contribution in pingdotgg/t3code#8569
* @ahmed-besic made their first contribution in pingdotgg/t3code#8392
* @MatthewFeroz made their first contribution in pingdotgg/t3code#8790
**Full Changelog**: pingdotgg/t3code@v0.0.36...v0.0.37
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native ChangeChanges the native fingerprint; merging blocks production OTAs until a new store build ships.size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@t3dotgg@juliusmarminge
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat(mobile): add offline iPhone voice input - #8614

Merged
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription
Aug 31, 2026
Merged

feat(mobile): add offline iPhone voice input#8614
juliusmarminge merged 8 commits into
mainfrom
t3code/ios-local-voice-transcription

Conversation

@t3dotgg

@t3dotggt3dotgg commented Aug 29, 2026

Copy link
Copy Markdown
Member

The mobile composer could not use Apple's newer on-device transcription model.

This adds local dictation for supported iPhones on iOS 26 or later. The microphone is available in existing threads and new task drafts, including queued task editing. T3 Code prepares Apple's speech assets for the device language and transcribes up to five minutes of audio on the phone.

The transcript is inserted at the selection where recording started. T3 Code keeps the keyboard open and lets the user review the text before sending it. Canceling keeps the draft unchanged. Temporary audio files are removed when the operation ends.

Tests:

  • All 42 focused tests passed after the Expo SDK 57 rebase.
  • Mobile typecheck and targeted lint passed after the Expo SDK 57 rebase.
  • The pre-rebase Expo SDK 56 version passed an iOS release build and a physical iPhone test.

This change needs a new native mobile build.

UI screenshots are pending permission to capture them.

Built with GPT-5.6 Sol in the Codex harness through T3 Code.


Note

Medium Risk
Touches microphone permissions, audio session lifecycle, and draft mutation timing with new native dependencies; incorrect cancellation or late transcript handling could corrupt composer text or leave sessions locked.

Overview
Adds on-device dictation for supported iPhones (iOS 26+) in the new task and thread composers. Users tap the mic, see recording status and a live waveform, then confirm to transcribe locally and insert text at the caret where recording started—send stays blocked until transcription finishes or is canceled.

A new @t3tools/client-runtime/voice-inputVoiceInputController owns the full lifecycle: single app-wide session, permission checks, up to five minutes of capture, cooperative cancellation, temp file cleanup, and draft owner/revision guards so late transcripts cannot overwrite a changed draft. Mobile wires expo-audio recording and metering through useVoiceInputController, with @react-native-ai/apple on iOS for SpeechTranscriber preparation and transcription.

Composer UX swaps the toolbar for dictation controls (cancel, status, confirm) with Reanimated layout transitions; collapsed composers can show a compact dictation strip. The native T3ComposerEditor gains a readOnly mode that blocks edits, paste, and drops while keeping focus and the keyboard. useComposerCommandMenu now keys selection off ownerKey so draft/thread switches reset caret position.

expo-audio and @react-native-ai/apple are added with patches (recording failure reporting, locale resolution, transcription cleanup). app.config.ts registers microphone permission for expo-audio. User and maintainer docs describe behavior and future remote-transcription boundaries. Requires a new native mobile build.

Reviewed by Cursor Bugbot for commit 50fc0ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add offline iPhone voice dictation to mobile composer

  • Adds VoiceInputController in controller.ts to orchestrate the full dictation lifecycle: permission checks, audio session setup, recording, on-device transcription via Apple's Speech framework, transcript commit, and cancellation semantics.
  • Integrates dictation UI into ThreadComposer.tsx and NewTaskDraftScreen.tsx with an animated dictation toolbar, live waveform metering, elapsed-time status, and confirm/cancel controls. Submission, attachments, and pickers are blocked while dictation is busy.
  • Adds a readOnly mode to the native iOS composer editor (T3ComposerEditorView.swift) so the text view blocks edits, paste, and drops while preserving focus and selection during dictation.
  • Patches @react-native-ai/apple@0.12.0 and expo-audio@57.0.4 to fix locale resolution, temp-file cleanup, and recording-failure error reporting.
  • Risk: resolveTranscriptCommit in controller.ts contains stray + tokens in the leftNeedsBoundary/rightNeedsBoundary computation that break TypeScript syntax and will prevent the module from compiling.

Macroscope summarized 50fc0ed.

@coderabbitai

coderabbitaiBot commented Aug 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d020e7f5-6801-48a2-9440-06dce8facce2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). 📱 Native Change Changes the native fingerprint; merging blocks production OTAs until a new store build ships. labels Aug 29, 2026
Comment threadpackages/client-runtime/src/voice-input/controller.ts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

disabled={isIncomingShareTransferPending}

Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from NewTaskDraftScreen, whose focus-effect cleanup calls controller.dispose() and discards the recording; disable the control or guard its handler when voiceInput.isBusy is true.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/NewTaskDraftScreen.tsx around line 997:
Opening the branch picker while dictation is busy silently cancels the active recording. The branch control remains enabled, so it navigates away from `NewTaskDraftScreen`, whose focus-effect cleanup calls `controller.dispose()` and discards the recording; disable the control or guard its handler when `voiceInput.isBusy` is true.

onChangeDraftMessage: props.onChangeDraftMessage,
onUpdateInteractionMode: props.onUpdateInteractionMode,
});
const voiceInput = useVoiceInputController({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highthreads/ThreadComposer.tsx:293

When a pending user-input request hides ThreadComposer, an active useVoiceInputController session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from ThreadDetailScreen.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/mobile/src/features/threads/ThreadComposer.tsx around line 293:
When a pending user-input request hides `ThreadComposer`, an active `useVoiceInputController` session continues recording or transcribing while its controls are unmounted from view; the transcript can then be inserted into the hidden draft after recording reaches the five-minute limit. Cancel the voice session when the composer becomes unavailable, and pass that availability from `ThreadDetailScreen`.

@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire13.3 KiB13.3 KiB+11 B (+0.1%)15.1 KiB
CodexThread snapshot wire6.9 KiB6.9 KiB−7 B (−0.1%)7.3 KiB
CodexLive turn WebSocket wire6.4 KiB6.5 KiB+18 B (+0.3%)7.8 KiB
CodexLive turn WebSocket decoded55.6 KiB55.6 KiB0 B (0.0%)66.4 KiB
CodexLive turn messages11110 (0.0%)21
ClaudeTotal thread wire13.3 KiB13.3 KiB+32 B (+0.2%)15.1 KiB
ClaudeThread snapshot wire6.9 KiB6.9 KiB+3 B (+0.0%)7.3 KiB
ClaudeLive turn WebSocket wire6.4 KiB6.4 KiB+29 B (+0.4%)7.8 KiB
ClaudeLive turn WebSocket decoded56.4 KiB56.4 KiB+44 B (+0.1%)66.4 KiB
ClaudeLive turn messages1011+1 (+10.0%)21

Baseline: e09b88b · PR result: 50fc0ed · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.4 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
right !== undefined &&
/[A-Za-z0-9([{'"]/.test(right) &&
(left === undefined || /\s/.test(left));
insertion = `${leftNeedsBoundary ? " " : ""}${replacement}${rightNeedsBoundary ? " " : ""}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emoji breaks English transcript spacing

Low Severity

English boundary spacing reads captured.text[index], which is a single UTF-16 code unit. A caret beside an emoji sees a surrogate that fails the letter/punctuation check, so the transcript is inserted without the intended leading or trailing space.

Fix in CursorFix in Web

Reviewed by Cursor Bugbot for commit 384d7fb. Configure here.

Comment threadapps/mobile/src/features/threads/NewTaskDraftScreen.tsx
@macroscopeapp

macroscopeappBot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR introduces a substantial user-facing iPhone voice-input workflow spanning native audio capture, local transcription, draft mutation, and composer lifecycle changes, along with new app-level audio defaults. Unresolved lifecycle and emoji-spacing concerns indicate that the behavior needs human review before merging.

Not approved because:

  • 2 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarmingeforce-pushed the t3code/ios-local-voice-transcription branch from 384d7fb to b4c5578CompareAugust 30, 2026 22:08
Comment threadpackages/client-runtime/src/voice-input/controller.ts
Comment threadapps/mobile/src/native/voiceTranscription.ios.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad0c05f. Configure here.

Comment threadapps/mobile/src/features/threads/ThreadComposer.tsx
Comment threadpackages/client-runtime/src/voice-input/controller.ts Outdated
@juliusmarminge
juliusmarminge merged commit 352710d into mainAug 31, 2026
27 checks passed
@juliusmarminge
juliusmarminge deleted the t3code/ios-local-voice-transcription branch August 31, 2026 01:08
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
**Full Changelog**: pingdotgg/t3code@v0.0.37-nightly.20260830.1232...v0.0.37-nightly.20260831.1233
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37-nightly.20260831.1233
github-actionsBot added a commit to omarcresp/t3code-flake that referenced this pull request Aug 31, 2026
## What's Changed
* Remove Messages Glass Lab experiment by @juliusmarminge in pingdotgg/t3code#8599
* Require human review for pull requests changing product defaults by @juliusmarminge in pingdotgg/t3code#8603
* fix(codex): avoid quadratic app-server input buffering by @juliusmarminge in pingdotgg/t3code#8605
* fix(mobile): stabilize iOS header item transitions by @juliusmarminge in pingdotgg/t3code#8607
* chore(mobile): upgrade to Expo SDK 57 by @juliusmarminge in pingdotgg/t3code#8609
* fix(mobile): harden native header toolbar items by @juliusmarminge in pingdotgg/t3code#8611
* fix(server): stop querying Claude context usage after turns by @t3dotgg in pingdotgg/t3code#8610
* chore: vouch ryanrhughes by @t3dotgg in pingdotgg/t3code#8613
* feat(web): attach PDFs, ZIPs, and other files to a turn by @t3dotgg in pingdotgg/t3code#8236
* feat(web): keybinding settings as settings rows by @StiensWout in pingdotgg/t3code#8532
* feat: let an environment publish themes as a file by @ryanrhughes in pingdotgg/t3code#8569
* fix(web): clean up provider settings list and editor by @StiensWout in pingdotgg/t3code#8504
* fix(web): keep project picker popup inside the sidebar by @SunkenInTime in pingdotgg/t3code#8627
* fix(mobile): prevent header overflow and back-button artifacts by @juliusmarminge in pingdotgg/t3code#8624
* fix(server): retry automatic thread title generation by @Bil0000 in pingdotgg/t3code#8087
* fix(client-runtime): refresh edited pull request comments by @Bil0000 in pingdotgg/t3code#8094
* fix(web): four composer spacing defects by @Bil0000 in pingdotgg/t3code#8090
* perf(desktop): skip duplicate browser updates by @Bil0000 in pingdotgg/t3code#8018
* fix(web): render nested markdown images correctly by @flamboh in pingdotgg/t3code#8501
* fix(web): unify activity logs and composer banners by @juliusmarminge in pingdotgg/t3code#8693
* fix(mobile): reduce dev-client reload and Metro startup cost by @juliusmarminge in pingdotgg/t3code#8694
* revert(web): restore previous composer banners by @t3dotgg in pingdotgg/t3code#8733
* test(web): remove tests for unreachable helpers by @t3-code[bot] in pingdotgg/t3code#8738
* feat(mobile): update tool summaries and chat transitions by @juliusmarminge in pingdotgg/t3code#8793
* feat(web): play video attachments in chat by @Bil0000 in pingdotgg/t3code#8688
* fix(web,mobile): snooze menu no longer offers the same wake time twice by @vitalyiegorov in pingdotgg/t3code#8741
* fix(grok): allow model changes in existing threads by @ahmed-besic in pingdotgg/t3code#8392
* feat(mobile): pick, share, and receive files in threads by @t3dotgg in pingdotgg/t3code#8237
* fix(web): reduce title bar scroll fade height by @maria-rcks in pingdotgg/t3code#8799
* fix(windows): strip quotes from repaired PATH by @UtkarshUsername in pingdotgg/t3code#8746
* fix(web): open agent images in expanded preview by @maria-rcks in pingdotgg/t3code#8807
* fix(git): follow repository instructions in generated source control text by @maria-rcks in pingdotgg/t3code#8804
* fix(server): stop overpricing cached Claude tokens by @SunkenInTime in pingdotgg/t3code#8806
* fix(web): keep image preview above sidebar control by @maria-rcks in pingdotgg/t3code#8811
* fix(web): keep right panel synced with agent edits by @maria-rcks in pingdotgg/t3code#8803
* fix(web,mobile): render Codex citations and artifact templates by @Yash-Singh1 in pingdotgg/t3code#8584
* chore: add Windows setup script to t3.json by @UtkarshUsername in pingdotgg/t3code#8814
* fix(web): fold interim turn responses by @maria-rcks in pingdotgg/t3code#8828
* fix(web): use circle alert for failed tool calls by @maria-rcks in pingdotgg/t3code#8840
* feat(mobile): add offline iPhone voice input by @t3dotgg in pingdotgg/t3code#8614
* fix(web): prevent pull request metadata overlap by @MatthewFeroz in pingdotgg/t3code#8790
## New Contributors
* @ryanrhughes made their first contribution in pingdotgg/t3code#8569
* @ahmed-besic made their first contribution in pingdotgg/t3code#8392
* @MatthewFeroz made their first contribution in pingdotgg/t3code#8790
**Full Changelog**: pingdotgg/t3code@v0.0.36...v0.0.37
Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📱 Native ChangeChanges the native fingerprint; merging blocks production OTAs until a new store build ships.size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@t3dotgg@juliusmarminge