Skip to content

feat(asr): 接入 ZenMux 语音转写(JSON + base64)作为可选云端 ASR(#837) - #915

Merged
H-Chris233 merged 2 commits into
Open-Less:betafrom
H-Chris233:codex/zenmux-asr
Aug 4, 2026
Merged

feat(asr): 接入 ZenMux 语音转写(JSON + base64)作为可选云端 ASR(#837)#915
H-Chris233 merged 2 commits into
Open-Less:betafrom
H-Chris233:codex/zenmux-asr

Conversation

@H-Chris233

@H-Chris233H-Chris233 commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

User description

背景

Closes#837

OpenLess 已支持多路云端 ASR,但聚合平台 ZenMux(zenmux.ai)的语音转写接口是 JSON 请求体 + base64 内联音频的私有格式,现有「OpenAI Whisper(兼容)」provider 对不上。已在使用 ZenMux 做 LLM 润色的用户无法复用同一把 Key 做语音识别。

变更

  • 协议WhisperBatchASR 新增 ZenMuxJson 请求格式(POST /audio/transcriptions + application/jsoninput_audio.data 走标准 base64),与 OpenRouter 的 JSON 路径隔离,避免互相污染。
  • 语言跟随language 字段取 working_languages 主语言 → ISO 639-1 映射(zh/en/ja/ko/fr/de/es/it/pt/ru/ar/vi/th/hi),未收录语言省略字段由服务端自动检测。
  • enable_itn 开关:per-provider 高级配置(复用 asr.advanced_config vault),默认开启;设置页对 zenmux 预设显示「数字归一化」开关。
  • 命名预设ASR_PRESETS 新增 zenmux,默认 https://zenmux.ai/api/v1 + qwen/qwen3-asr-flash;模型 slug 自由输入(可填 bytedance/doubao-seed-asr-2.0xiaomi/mimo-v2.5-asr 等)。后端空槽回落同款默认值。
  • 分片:与 OpenRouter 一致按 30s 切分(base64 进 JSON body 体积膨胀约 33%)。
  • i18n/测试:五语言同步新增键;后端新增请求体/语言映射/路由单测,前端单测与 i18n 一致性脚本保持通过。

测试

  • 后端:zenmux 相关单测 5 个、whisper 模块 29 个、coordinator 82 个全部通过;cargo check --lib 无错误。
  • 前端:tsc --noEmit 零错误,完整前端测试套件通过。

说明

ZenMux 官方文档在实现时不可访问,协议以 issue #837 提供的请求/响应形状为准(与已实现的 OpenRouter JSON 格式一致);如联调发现 language/enable_itn 字段校验更严,仅需微调映射或开关发送策略。


PR Type

Enhancement


Description

  • Add ZenMux ASR preset using JSON+base64 transcription

  • Follow working language and expose enable_itn toggle

  • Make test connection match provider real request format

  • Update i18n, docs, and tests across the app


Diagram Walkthrough

flowchart LR
A["ASR request"] --> B{"Request format"}
B -- "zenmux" --> C["ZenMuxJson JSON+base64"]
B -- "openrouter" --> D["OpenRouterJson"]
B -- "others" --> E["Multipart"]
C --> F["ZenMux /audio/transcriptions"]
F --> G["Transcript text"]
Loading

File Walkthrough

Relevant files
Enhancement
8 files
whisper.rs
Add ZenMuxJson request format with language/itn options
+168/-0
providers.rs
Validate ASR connection using provider-specific JSON or multipart
+61/-16
coordinator.rs
Add zenmux provider id, defaults, and enable_itn config
+91/-13
asr_wiring.rs
Wire zenmux format, language mapping, and ASR options
+99/-4
dictation.rs
Apply ZenMux options in dictation session construction
+4/-2
advancedAsrConfig.ts
Add enableItn field to advanced ASR config
+11/-6
ProvidersSection.tsx
Show enable_itn toggle and ZenMux vocabulary note
+54/-29
shared.tsx
Add ZenMux preset with default endpoint and model
+4/-0
I18n
5 files
en.ts
Add English translations for ZenMux and enable_itn
+5/-1
ja.ts
Add Japanese translations for ZenMux and enable_itn
+5/-1
ko.ts
Add Korean translations for ZenMux and enable_itn
+5/-1
zh-CN.ts
Add Simplified Chinese translations for ZenMux and enable_itn
+5/-1
zh-TW.ts
Add Traditional Chinese translations for ZenMux and enable_itn
+5/-1
Tests
2 files
advancedAsrConfig.test.ts
Update tests for enable_itn config parsing
+20/-10
ProvidersSection.test.ts
Add test for ZenMux ASR preset defaults
+14/-0
Documentation
2 files
README.md
Document ZenMux ASR and hotword exception
+2/-2
README.zh.md
Document ZenMux ASR and hotword exception in Chinese
+2/-2

- WhisperBatchASR 新增 ZenMuxJson 请求格式:application/json + base64
input_audio,language 跟随工作语言(ISO 639-1 映射),enable_itn 可配置
- 新增 zenmux 命名预设:默认 https://zenmux.ai/api/v1 + qwen/qwen3-asr-flash,
复用 Whisper 兼容路由与 30s 分片;模型 slug 自由输入
- 设置页新增 enable_itn 开关(per-provider 高级配置,默认开启)
- 补齐五语言 i18n、前端单测与 README
@github-actions

github-actionsBot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

(Review updated until commit 7331f7f)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅

837 - PR Code Verified

Compliant requirements:

  • ZenMux preset added with default base URL and model; API Key and model slug are user-configurable.
  • ZenMuxJson request format implements POST /audio/transcriptions with application/json and base64 input_audio; language is derived from the primary working language; the response text is handled by the existing Whisper transcription pipeline.
  • Failures propagate through the existing Whisper transcription error paths.

Requires further human verification:

  • Live verification against the actual ZenMux endpoint with a real API key (request/response shape and 4xx error surfacing).
  • Manual UI verification of the new ZenMux preset and the enable_itn toggle in Settings.
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected

- validate_asr_transcription 按 provider 真实请求格式发探针:OpenRouter /
ZenMux 走 JSON+base64,其余 whisper 兼容厂商走 multipart,避免假阴性
- 设置页对 zenmux 预设增加说明:JSON 协议不发送词典热词(prompt/hotwords),
词典仍进入润色链路但不参与识别偏置;README 同步标注例外
- coordinator 再导出 whisper_request_format 供验证路径复用
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 7331f7f

@H-Chris233
H-Chris233 merged commit 5f9586f into Open-Less:betaAug 4, 2026
5 checks passed
bigsongeth added a commit to bigsongeth/openless that referenced this pull request Aug 4, 2026
beta 合入 Open-Less#915(ZenMux 语音转写)后与本分支在 asr_wiring.rs 的 QA 构造点相撞:
两边改的是同一条语句的不同部分。
本分支 词表来源换成 asr_vocab_phrases(按命中排序 + 保底席位)
beta 外面包一层 apply_zenmux_asr_options
互不冲突,取并集。闭合括号那几行 beta 已经写成 apply_zenmux_asr_options 的形状,
所以只需保留自己那半个词表调用。听写路径那处 git 自动合并正确(asr_vocab_phrases
保住了),已核对。
cargo check 干净;cargo test --lib 1000 passed;tsc --noEmit 干净;npm test 通过。
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[asr] 接入 ZenMux 语音转写接口(JSON + base64 私有格式)作为可选云端 ASR

1 participant

@H-Chris233