Uh oh!
There was an error while loading. Please reload this page.
www: put the two engines and the 4-bit options on the site - #73
Merged
Conversation
The site still described a Parakeet-only, bf16-only server. Four model features have shipped since its last content change: the fp16 checkpoint for pre-Ampere cards, nvfp4 for Parakeet, the Whisper engine, and nf4 for Whisper. None of them appeared anywhere, including the two that are the strongest things we can say — ~100 languages, and a model that fits in half a gigabyte. Adds an engines section (Parakeet default / Whisper via --engine), makes the pipeline diagram engine-agnostic, and turns the naming joke into the reveal: we didn't need to name ourselves after Whisper, we just run it when you want it. Every number is re-measured on an RTX 5090 over the repo's 208-clip corpus rather than carried over, and two were wrong: * "~70ms to transcribe a 35-second clip" conflated two things. 75 ms is the median dictation; a 35.6 s clip takes ~277 ms. * "~2.3 GB VRAM" is now 1.4 GB peak in bf16, 0.5 GB in 4-bit. Measuring those turned up a bug in scripts/compare_precision.py worth recording: run in one process, its VRAM figures are contaminated — Parakeet nvfp4 reports 1.82 GB of weights, more than bf16's 1.28, because NeMo keeps model instances alive across the harness's reloads and the previous precision is still resident when the next is measured. Measured one precision per process the numbers match the committed table exactly (bf16 peak 1.43 GB, nvfp4 weights 0.51 GB). The WER figures from that script are unaffected and reproduce. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01376qJiterkkQwTWmN1AC4Q
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The site still described a Parakeet-only, bf16-only server. Its last content change was
907ac19; everywww/commit since is dependabot. Four model features shipped in that window and none of them appear anywhere:~2.3 GB VRAM25 languages; pipeline hard-codedParakeetThe two most valuable things we can now say — ~100 languages and a model that fits in half a gigabyte — were both missing.
What changed
--engine whisper) side by side, with languages, WER, latency, and VRAM for each, plus a note that both take a 4-bit option and why the formats differ.systemdblock re-numbered.Numbers were re-measured, and two were wrong
Everything is measured on an RTX 5090 over the repo's own 208-clip / 25.8-min corpus, not carried over:
~70ms to transcribe a 35-second clipconflated two figures. 75 ms is the median dictation; a 35.6 s clip takes ~277 ms. The stat is now labelled for what it measures.~2.3 GB VRAM→ 1.4 GB peak in bf16, 0.5 GB in 4-bit.A bug this turned up in
scripts/compare_precision.pyWorth recording separately: run in one process, that script's VRAM figures are contaminated. Measuring bf16 and nvfp4 together, it reports Parakeet nvfp4 at 1.82 GB of weights — more than bf16's 1.28 GB, and bf16 peak at 3.84 GB against the committed table's 1.43 GB. 1.28 + 0.51 ≈ 1.82 gives it away: NeMo keeps model instances alive across the harness's reloads, so the previous precision is still resident when the next is measured.
Measured one precision per process, the numbers match the committed README table exactly (bf16 peak 1.43 GB, nvfp4 weights 0.51 GB), which is what this PR uses. The WER figures are unaffected and reproduce (2.99% / 2.86%, p=0.69).
compare_whisper_quant.pyin #72 already isolates per variant for this reason;compare_precision.pyhas not been fixed here — happy to do that separately.Notes
main; land that first.curlbut Chrome showed an error page for it, so I stopped rather than keep poking.next build, TypeScript, and static generation all pass, and the new section reuses the card/grid classes the existing "why bother" section already uses.🤖 Generated with Claude Code
https://claude.ai/code/session_01376qJiterkkQwTWmN1AC4Q