Skip to content

www: put the two engines and the 4-bit options on the site - #73

Merged
z33b0t merged 1 commit into
mainfrom
www-model-lineup
Aug 19, 2026
Merged

www: put the two engines and the 4-bit options on the site#73
z33b0t merged 1 commit into
mainfrom
www-model-lineup

Conversation

@z33b0t

Copy link
Copy Markdown
Contributor

The site still described a Parakeet-only, bf16-only server. Its last content change was 907ac19; every www/ commit since is dependabot. Four model features shipped in that window and none of them appear anywhere:

shippedthe site said
fp16 checkpoint (#68)"any GPU from the 20-series up" — accidentally made true by it, never explained
nvfp4 for Parakeet (#69)~2.3 GB VRAM
Whisper engine (#71)no mention at all; 25 languages; pipeline hard-coded Parakeet
nf4 for Whisper (#72)

The two most valuable things we can now say — ~100 languages and a model that fits in half a gigabyte — were both missing.

What changed

  • New "Two models. Pick one." section — Parakeet (default) and Whisper (--engine whisper) side by side, with languages, WER, latency, and VRAM for each, plus a note that both take a 4-bit option and why the formats differ.
  • The naming joke becomes the reveal. The page's whole gag is that the field is "the word Whisper with the vowels moved around." Rather than fight that, it now lands as: we didn't need to name ourselves after Whisper — we just run it when you want it.
  • Pipeline diagram no longer hard-codes Parakeet.
  • Stats, card 02, and the systemd block re-numbered.

Numbers were re-measured, and two were wrong

Everything is measured on an RTX 5090 over the repo's own 208-clip / 25.8-min corpus, not carried over:

  • ~70ms to transcribe a 35-second clipconflated two figures. 75 ms is the median dictation; a 35.6 s clip takes ~277 ms. The stat is now labelled for what it measures.
  • ~2.3 GB VRAM1.4 GB peak in bf16, 0.5 GB in 4-bit.
Parakeet (measured, isolated)weightspeakmedian clip35.6 s clip
bf161.28 GB1.43 GB74.9 ms276.6 ms
nvfp40.51 GB0.66 GB176.4 ms428.5 ms

A bug this turned up in scripts/compare_precision.py

Worth recording separately: run in one process, that script's VRAM figures are contaminated. Measuring bf16 and nvfp4 together, it reports Parakeet nvfp4 at 1.82 GB of weights — more than bf16's 1.28 GB, and bf16 peak at 3.84 GB against the committed table's 1.43 GB. 1.28 + 0.51 ≈ 1.82 gives it away: NeMo keeps model instances alive across the harness's reloads, so the previous precision is still resident when the next is measured.

Measured one precision per process, the numbers match the committed README table exactly (bf16 peak 1.43 GB, nvfp4 weights 0.51 GB), which is what this PR uses. The WER figures are unaffected and reproduce (2.99% / 2.86%, p=0.69). compare_whisper_quant.py in #72 already isolates per variant for this reason; compare_precision.py has not been fixed here — happy to do that separately.

Notes

  • Depends on asr: add a 4-bit nf4 option for the Whisper engine #72 for the nf4 numbers to be true of main; land that first.
  • Layout not visually verified — the local preview server was reachable by curl but Chrome showed an error page for it, so I stopped rather than keep poking. next build, TypeScript, and static generation all pass, and the new section reuses the card/grid classes the existing "why bother" section already uses.

🤖 Generated with Claude Code

https://claude.ai/code/session_01376qJiterkkQwTWmN1AC4Q

The site still described a Parakeet-only, bf16-only server. Four model
features have shipped since its last content change: the fp16 checkpoint
for pre-Ampere cards, nvfp4 for Parakeet, the Whisper engine, and nf4 for
Whisper. None of them appeared anywhere, including the two that are the
strongest things we can say — ~100 languages, and a model that fits in
half a gigabyte.
Adds an engines section (Parakeet default / Whisper via --engine), makes
the pipeline diagram engine-agnostic, and turns the naming joke into the
reveal: we didn't need to name ourselves after Whisper, we just run it
when you want it.
Every number is re-measured on an RTX 5090 over the repo's 208-clip
corpus rather than carried over, and two were wrong:
* "~70ms to transcribe a 35-second clip" conflated two things. 75 ms is
the median dictation; a 35.6 s clip takes ~277 ms.
* "~2.3 GB VRAM" is now 1.4 GB peak in bf16, 0.5 GB in 4-bit.
Measuring those turned up a bug in scripts/compare_precision.py worth
recording: run in one process, its VRAM figures are contaminated —
Parakeet nvfp4 reports 1.82 GB of weights, more than bf16's 1.28, because
NeMo keeps model instances alive across the harness's reloads and the
previous precision is still resident when the next is measured. Measured
one precision per process the numbers match the committed table exactly
(bf16 peak 1.43 GB, nvfp4 weights 0.51 GB). The WER figures from that
script are unaffected and reproduce.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01376qJiterkkQwTWmN1AC4Q
@z33b0t
z33b0t merged commit 1831c19 into mainAug 19, 2026
7 checks passed
@z33b0t
z33b0t deleted the www-model-lineup branch August 19, 2026 20:53
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@z33b0t