Skip to content

Add voice dictation support - #99

Open
ekishabasu1 wants to merge 2 commits into
mainfrom
dev/ekishabasu/voice-recognition
Open

ekishabasu1 wants to merge 2 commits into
mainfrom
dev/ekishabasu/voice-recognition

Conversation

@ekishabasu1

@ekishabasu1 ekishabasu1 commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add local automatic speech recognition and microphone dictation in the composer
  • add dictation settings, persistence, IPC/preload plumbing, and packaged runtime support
  • add draft handling and smoke coverage for the voice flow

Validation

  • npm run build
  • npm run smoke:shared (990 checks passed)

Voice dictation settings with fast and accurate local recognition modes and optional transcript polishing

Ekisha Basu added 2 commits September 8, 2026 10:54
Add local speech recognition, dictation controls, voice settings, persistence, IPC plumbing, packaging, and smoke coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9f046cd-30cb-4d7b-8e88-6f7ba3a9f90c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9f046cd-30cb-4d7b-8e88-6f7ba3a9f90c
@FreddyJD

Copy link
Copy Markdown
Collaborator

Thank youu!

@FreddyJD

Copy link
Copy Markdown
Collaborator

@JairEsk do you want to validate on mac and windows and merge approve the PR? :)

@JairEsk
JairEsk self-requested a review September 14, 2026 17:07
@JairEsk

JairEsk commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Hi @ekishabasu1 @FreddyJD ,

Apologies for the delay on this — I was out over the weekend. :(

Thanks for putting this PR together! Really excited about this feature.

While testing the branch locally on macOS (darwin-arm64), I ran into an issue during the initial runtime setup when calling dictation:start:

Error invoking remote method 'dictation:start': Error: The runtime archive didn't contain nemo-speech.

Root Cause

It looks like there is a structural discrepancy in the upstream NeMo-Speech.cpp release archives across targets:

  • Windows (.zip): Extracts directly into root (bin/nemo-speech.exe).
  • macOS / Linux (.tar.gz): Wraps contents inside an enclosing top-level directory (e.g. nemo-speech/bin/nemo-speech on macOS or nemo-speech-0.1.0-*/bin/nemo-speech on Linux).

Since installRuntime() expects staging/bin/nemo-speech at the immediate root, the existence check currently fails on Unix platforms.

Suggested approach

We could dynamically resolve the inner root directory if a wrapper folder is detected after extraction:

const binaryName = process.platform === 'win32' ? 'nemo-speech.exe' : 'nemo-speech'
let contentRoot = staging

if (!(await exists(join(contentRoot, 'bin', binaryName)))) {
  const entries = await fsp.readdir(staging, { withFileTypes: true })
  const nestedDir = entries.find((e) => e.isDirectory())
  if (nestedDir && (await exists(join(staging, nestedDir.name, 'bin', binaryName)))) {
    contentRoot = join(staging, nestedDir.name)
  }
}

const expected = join(contentRoot, 'bin', binaryName)
if (!(await exists(expected))) {
  throw new Error("The runtime archive didn't contain nemo-speech.")
}

await fsp.rm(runtimeDir(), { recursive: true, force: true })
await fsp.rename(contentRoot, runtimeDir())

Let me know what you think, and I'd be happy to re-test this on macOS once updated! :) 💯 

<img width="806" height="173" alt="Captura de pantalla 2026-09-14 a la(s) 11 10 31" src="https://github.com/user-attachments/assets/be61eeb9-2ce0-4a06-8c1a-697cd2027e10" />

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants