Skip to content

fix(ios): keep Bluetooth voice route active between turns - #660

Closed
dastratakos wants to merge 1 commit into
mainfrom
dastratakos/nouakchott-v1
Closed

fix(ios): keep Bluetooth voice route active between turns#660
dastratakos wants to merge 1 commit into
mainfrom
dastratakos/nouakchott-v1

Conversation

@dastratakos

@dastratakosdastratakos commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Tap-to-talk rebuilt the iOS audio session on every microphone and playback transition. With AirPods connected, that repeatedly tore down and renegotiated the Bluetooth HFP route, so capture could start before the input route settled.

This PR now keeps one voice-chat audio session active for the lifetime of the voice screen:

  • configure playAndRecord + voiceChat + Bluetooth HFP before opening the realtime connection
  • keep the route active between capture and playback
  • reassert activation after interruptions and reconnects without resetting the category
  • return activation failures to an idle, retryable state
  • deactivate only when the voice screen closes

The previous route watchdogs, diagnostics, short-turn handling, send-chain changes, and tap gesture changes have been removed.

Verification

  • rebased on latest main (a8595319 at verification time)
  • ./scripts/check.sh
  • swift test in apps/ios/LukeKit (242 tests)
  • full Luke scheme tests on an iPhone 17 Pro simulator running iOS 26.5
  • installed and launched the app in the simulator; verified tapping Talk to Luke changes the voice screen from Tap or hold to talk to Listening and the control to Tap to send

The iOS Simulator cannot expose AirPods as an iPhone Bluetooth HFP input route. The route configuration follows the documented voiceChat and allowBluetoothHFP behavior; the radio path still needs a physical AirPods smoke check.

Automated visual evidence

Download the deterministic macOS evidence · workflow run

  • Commit: debb3b8dca78d399ade592e0c1f6f2579cb60b36
  • Scenario: smoke
  • Physical-notch check: not performed by CI

@vercel

vercelBot commented Sep 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated
luke-webReadyReadyPreviewSep 4, 2026 1:32am UTC

Request Review

Comment threadapps/ios/Luke/VoiceAudioCapturer.swift Outdated
Comment threadapps/ios/Luke/VoiceAudioCapturer.swift Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security verification: DISCARD

Claim checked: New iOS voice-capture diagnostic logging (os.Logger + lastSummary) is a MEDIUM+ privacy/security issue in RELEASE/App Store builds.

Verdict: DISCARD — not a real MEDIUM+ issue. No off-device exfiltration path for these diagnostics in release builds; logged values are hardware class, not device identity; UI summary is compile-time DEBUG-only.

1. Do Sentry / SessionReplay / analytics capture Logger or lastSummary in release?

PathRelease capture?Why
iOS SentryN/AiOS has no Sentry SDK. PRIVACY.md Sentry section is desktop Electron-only.
PostHog SessionReplayScreenshots of drawn UI onlySessionReplay.swift uses screenshot mode. Release onRecoverableError sets errorMessage = messagewithoutlastSummary (#else branch).
PostHog errorTrackingConfig.autoCaptureUnhandled exceptions onlyDoes not ingest os.Logger / unified-log lines.
Counted product eventsNoAllowlisted vocabulary; voice-capture diagnostics are not wired in.

os.Logger (subsystem: "dev.tryluke.ios", category "voice-capture") writes Apple unified logs on-device. Nothing in this app bridges those lines to PostHog or Luke’s /api/events. Reading them requires local Console / sysdiagnose access — not a remote attacker path.

2. Are Bluetooth port types sensitive in this repo?

No. routeDescription logs portType.rawValue only (e.g. BluetoothA2DPOutput, MicrophoneBuiltIn) — notportName / friendly device names.

That matches the product bar in apps/desktop/native/macos/MicrophoneRoute.swift, which already treats transport class (built-in / bluetooth / other) as first-class diagnostic state, while media/calendar helpers deliberately kill names/titles before they leave the process. Port type is hardware class, not PII.

3. Can lastSummary leak in non-DEBUG builds?

No meaningful leak path:

  • UI append is behind #if DEBUG in VoiceView.swift — stripped from Release/App Store binaries.
  • note() still fills the static in release, but nothing reads it for display, network, or analytics.
  • Residual in-process string is not exploitably reachable without device compromise already past the trust boundary.

4. Logging bar / residual notes (not MEDIUM+)

  • First os.Logger usage on iOS; no conflicting pattern that treats public port-type logs as sensitive.
  • File comments explicitly exclude PCM samples; code does not log buffer contents.
  • error.localizedDescription with privacy: .public is generic AVAudioEngine/session text in practice; even a rare verbose system string would be LOW residual, not MEDIUM+.

Exploitability: None for App Store builds against the claimed privacy boundary (analytics / replay / crash streams). Local unified-log visibility of port types is intentional developer diagnostics, consistent with desktop transport-class reporting.

Open in WebView Automation

Sent by Cursor Security Agent: Security Reviewer

Comment threadapps/ios/Luke/VoiceAudioCapturer.swift Outdated
Comment threadapps/ios/Luke/VoiceAudioCapturer.swift Outdated
Comment threadapps/ios/Luke/VoiceView.swift Outdated
Comment threadapps/ios/Luke/VoiceView.swift Outdated
@dastratakos
dastratakosforce-pushed the dastratakos/nouakchott-v1 branch from c656b35 to b9c8437CompareSeptember 4, 2026 00:57
@dastratakosdastratakos changed the title fix(ios): send every captured chunk before the commit and refuse turns the API wouldfix(ios): keep Bluetooth voice route active between turnsSep 4, 2026
Comment threadapps/ios/Luke/VoiceView.swift
Comment threadapps/ios/Luke/VoiceView.swift
Comment threadapps/ios/Luke/VoiceAudioCapturer.swift

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Security verification: DISCARD

Claim checked: Keeping AVAudioSessionplayAndRecord + voiceChat active for the voice-screen lifetime (new VoiceAudioSession) causes microphone samples to continue being captured or sent after VoiceAudioCapturer.stop() / between turns.

(1) Can samples flow after stop() while the session remains active?

No.

Evidence from the capture → send pipeline:

  1. VoiceAudioCapturer.stop() tears down the only PCM path (apps/ios/Luke/VoiceAudioCapturer.swift): removeTap(onBus:) then engine.stop(). Apple’s tap API stops invoking the tap block after removal. An active AVAudioSession alone does not deliver PCM into the process; samples only enter via the installed tap + running AVAudioEngine.

  2. RealtimeSession.stopCapturing() (called from endTurn / close) cancels the consume task, calls capturer?.stop(), and nils the capturer. Between turns there is no capturer instance and no captureTask.

  3. Send path is status-gated. Chunks are only buffered (PressAudioBuffer) in .connecting or WebSocket-appended in .listening. After endTurn in .listening, status becomes .thinkingafterstopCapturing(), so any stray in-flight chunk hits default and is not sent. PressAudioBuffer is only fed while capture is running during connect; it is drained on channel open or discarded on a superseding press — it is not a between-turns recorder.

  4. Deactivate is correctly deferred to screen exit, not turn end: VoiceAudioSession.activate() in VoiceSessionModel.start(), deactivate() in stop() from VoiceView.onDisappear. That keeps the Bluetooth HFP route warm; it does not keep a tap alive.

(2) MEDIUM+ unauthorized recording / data exfil?

No — same-user product audio routing.

This is intentional push-to-talk route stability while the developer is on the voice screen they opened. No new sink, persistence, or covert capture path. Samples leave the device only during an armed turn over the existing Realtime WebSocket, as before.

Non-vuln privacy UX note (not MEDIUM+): With playAndRecord held active between turns (and after idle close() until disappear), the system mic indicator may remain longer than when each stop() deactivated the session. That is indicator lifetime / route ownership, not continued PCM capture or exfiltration.

(3) Other concrete security issues in the changed lines?

None found. Diff is limited to session activate/reactivate/deactivate lifetime, mode .voiceChat, and removing per-stop setActive(false). No audio logging, no new network endpoints, no credential handling, no weakened auth.

Prior Bugbot threads about route-watchdog / restart budgets / diagnostic logging target code that this PR removed; they are stale relative to the current diff and do not change this discard.

Open in WebView Automation

Sent by Cursor Security Agent: Security Reviewer

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7be37fd. Configure here.

Comment threadapps/ios/Luke/VoiceView.swift
Comment threadapps/ios/Luke/VoiceView.swift
@dastratakos
dastratakosforce-pushed the dastratakos/nouakchott-v1 branch from 7be37fd to 8fdf1d1CompareSeptember 4, 2026 01:18

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{
"candidate": "mic_session_keepalive_capture",
"is_vulnerability": false,
"severity": null,
"exploitability": "Not exploitable as unintended capture. Activating AVAudioSession only reserves/configures the duplex route. Microphone samples enter application code only when VoiceAudioCapturer.start() installs an input-node tap and starts its dedicated AVAudioEngine, which beginTurn() alone invokes. endTurn() synchronously cancels the capture task, removes that tap, stops the capture engine, and clears the capturer. Between turns, VoiceAudioPlayer starts a separate output-only engine whose graph is playerNode -> mainMixerNode; it never accesses, taps, or connects inputNode. RealtimeSession serializes audio only inside the capture stream loop and sends input_audio_buffer.append only in .listening (or buffers it in .connecting for a user-opened reconnect turn). close()/idle also call stopCapturing(), and VoiceView.onDisappear closes the realtime session before deactivating AVAudioSession. An audio append already initiated from a sample captured during an intentional turn may finish around release, but no new between-turn samples can be produced or sent. The orange microphone indicator or an HFP SCO route remaining active while this user-opened voice screen is visible therefore reflects route keepalive, not delivery of microphone buffers to the app or WebSocket.",
"false_positive_risk": "high",
"affected_changed_lines": [
"apps/ios/Luke/VoiceAudioCapturer.swift:4-25",
"apps/ios/Luke/VoiceAudioCapturer.swift:35-38",
"apps/ios/Luke/VoiceAudioCapturer.swift:105-110",
"apps/ios/Luke/VoiceAudioPlayer.swift:12-24",
"apps/ios/Luke/VoiceAudioPlayer.swift:57-60",
"apps/ios/Luke/VoiceView.swift:52-79",
"apps/ios/Luke/VoiceView.swift:157-166"
],
"rationale": "The change extends AVAudioSession route activation, not the lifetime of the only microphone tap or capture engine. No privilege, sandbox, cross-user, credential, or backend boundary is crossed, and the realtime transport has no source of between-turn microphone samples. This is not a MEDIUM+ security vulnerability under the stated threat model."
}
Open in WebView Automation

Sent by Cursor Security Agent: Security Reviewer

Configure one voice-chat audio session before the realtime connection so AirPods can settle onto HFP before tap-to-talk begins. Keep the session active across microphone capture and playback, and only reassert activation on each turn or reconnect so interruptions recover without rebuilding the route. Preserve the reconnect callback before activation so a transient setup failure can be retried by the next press, returning the UI to idle when activation fails.
@dastratakos

Copy link
Copy Markdown
ContributorAuthor

too complex

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@dastratakos