Uh oh!
There was an error while loading. Please reload this page.
feat(mac): on-device transcription via Apple's SpeechTranscriber - #62
Open
z33b0t wants to merge 3 commits into
Open
feat(mac): on-device transcription via Apple's SpeechTranscriber#62z33b0t wants to merge 3 commits into
z33b0t wants to merge 3 commits into
Conversation
Blurt's Mac client was a thin pipe to a Parakeet server: a great experience once you have the GPU box, and no experience at all until you do. macOS 26 ships an on-device streaming ASR stack whose accuracy is in the same league (~2.1% WER clean against Parakeet TDT v3's ~2.5% on LibriSpeech test-clean), runs on the Neural Engine, and ships zero bytes with the app. Settings ▸ Transcription now picks between the server and this Mac. Server mode is unchanged and stays the default. The result models line up almost exactly, so the HUD needed no changes: a volatile result is the `live` half of a partial and a finalized one settles into `committed`. A SpeechDetector rides alongside the transcriber purely so "Hearing you…" keeps meaning what it means in server mode. AppDelegate already talked to DictationClient through a closure surface, so that surface became TranscriptionEngine and DictationClient conforms to it nearly as-is. AudioCapture grows a negotiable output format — the server wants its declared 16 kHz Int16, SpeechAnalyzer names its own — and hands over an AVAudioPCMBuffer rather than packing Data itself. Gating is one runtime check, SpeechTranscriber.isAvailable, which answers both halves of the question: the API only exists on macOS 26, and even there it reports false without a Neural Engine, i.e. on every Intel Mac. Verified on a MacBookPro16,2 running macOS 26.5 — isAvailable false, supportedLocales empty, AssetInventory reporting the modules unsupported. So Intel keeps the server path and loses nothing. The deployment target stays 13.0, the Speech symbols sit behind #available, and because they ship in the x86_64 slice of the SDK too there is no conditional compilation anywhere — one universal binary still runs everywhere from Ventura up. Settings shows the local option greyed out with the reason rather than hiding it, and Settings.engine refuses to return .local on a Mac that cannot run it, so a preference restored from an Apple silicon Mac can't strand this one. CI moves to macos-26: the on-device engine needs the macOS 26 SDK to compile and the Sequoia image doesn't ship it.
The previous comment claimed Sequoia images don't ship the macOS 26 SDK. They do: macos-15-arm64 carries Xcode 26.3 (macOS 26.0-26.2 SDKs) next to the default 16.4, and the workflow's select-newest-stable-Xcode step was already picking it — the last release off main was built against SDK 26.2. So the engine would have compiled on macos-15 unchanged. Pinning is still worth doing, for a different reason: relying on a Sequoia image to bundle a Tahoe Xcode is an accident of image composition, not a guarantee. Say that instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a second transcription engine to the Mac client: macOS 26's on-device
SpeechAnalyzer+SpeechTranscriber, selectable in Settings ▸ Transcriptionalongside the existing blurtd path. Server mode is unchanged and stays the default.
Until now the client was a thin pipe to a Parakeet server — great once you have
the GPU box, nothing until you do. Apple's model is in the same league on
accuracy (~2.1% WER clean vs Parakeet TDT v3's ~2.5% on LibriSpeech test-clean),
runs on the Neural Engine, and ships zero bytes with the app.
Reach
Purely additive —
LSMinimumSystemVersionstays 13.0 and nobody loses anything.Design
AppDelegatealready talked toDictationClientthrough a closure surface, sothat surface became
TranscriptionEngine;DictationClientconforms nearlyas-is. Apple's result model lines up with Blurt's, so the HUD needed no changes:
partialLiveReportingOption.volatileResults)partialCommittedisFinal) accumulatedonVad/ "Hearing you…"SpeechDetector(reportResults: true)onInfo"loading"AssetInventorystatus / install progressAudioCapturegrows a negotiable output format (the server wants its declared16 kHz Int16,
SpeechAnalyzernames its own) and hands over anAVAudioPCMBufferrather than packingDataitself.Why no
#if arch(...)anywhereSpeechTranscriber.isAvailableanswers both halves of "can this Mac do it" —the API only exists on macOS 26, and even there it reports false without a
Neural Engine, i.e. on every Intel Mac. And because the symbols ship in the
x86_64 slice of the SDK too, the universal build compiles unchanged and
Intel simply takes the server path at runtime.
Settings shows the local option greyed out with the reason rather than hiding
it, and
Settings.enginerefuses to return.localon a Mac that can't run it,so a preference restored from an Apple silicon Mac can't strand this one.
Verified
On a MacBookPro16,2 / Intel i7-1068NG7 / macOS 26.5.2 — the negative case,
run against the production source files:
Driving the real
SettingsWindowController:Universal build is intact (
lipo:x86_64 arm64), signs with Developer ID +hardened runtime, and the build is warning-free.
Not yet verified — needs an Apple silicon Mac
I only have the Intel machine, so the positive path is unexercised. Before
merging, on Apple silicon + macOS 26:
privacy proof in one
SpeechDetectorSpeechAnalyzerneeds Speech TCC —NSSpeechRecognitionUsageDescriptionis added defensively; if it's required,
Onboarding.swiftwants a thirdpermission row
CI
mac.ymlmovesmacos-15→macos-26: the engine needs the macOS 26 SDK tocompile and the Sequoia image doesn't ship it.
https://claude.ai/code/session_01SUWiz8CtyyAGuxKgAEVZJu