[whisper] short-form: initial_prompt can cause silent tail truncation (early EOS at a pause) — same weights+prompt complete in whisper.cpp

[tosoyn]

Summary

Whisper short-form runs (audio ≤ 30 s, TRANSCRIBE_TIMESTAMPS_NONE) can silently drop the tail of an utterance after a mid-utterance pause when an initial_prompt is set (via RunExtension::Whisper { initial_prompt }). The decoder emits an early EOS at a prosodic pause; everything after it is lost. No error is surfaced — the run returns TRANSCRIBE_OK and transcribe_was_truncated() is false (the EOS is "legitimate"), so a consumer can't even warn the user. Reference whisper.cpp with the same weights and the same prompt transcribes the same audio completely.

Found in production via Handy (which passes its "custom words" setting as initial_prompt): dictated notes were regularly stored with the last clause missing.

Environment

• transcribe-cpp 0.1.3 as shipped in Handy v0.9.3 AppImage (Linux x86_64, downloaded 2026-07-17)
• Model: handy-computer/whisper-large-v3-turbo-gguf/whisper-large-v3-turbo-Q8_0.gguf
• Reproduces identically on Vulkan0 (RTX 5090) and CPU (Core Ultra 9 285K) — not a backend/numerics issue
• Run options: task=Transcribe, language=ru, timestamps=NONE, initial_prompt="Claude"

Repro

Test clip: 7.62 s of Russian speech, 16 kHz mono, natural short pause before the final clause; speech activity until ~7.1 s (verified via RMS envelope). Full audio reaches the engine (121 920 samples). I can share the WAV on request.

Driven through Handy's headless CLI (handy -f note.wav --json), which calls Session::run on this engine. Deterministic across repeated runs.

Expected (spoken): Притом я пошёл в Handy и посмотрел историю, я могу послушать аудиофайлы, и они действительно длиннее, чем транскрипция.
Actual (initial_prompt = "Claude"): Притом я пошёл в Handy и посмотрел историю Я могу послушать аудио файлы — the last ~2 s are gone.

Isolation matrix (same WAV, same weights)

Decode path	initial_prompt	Result
transcribe-cpp short-form, Vulkan	"Claude"	❌ tail dropped
transcribe-cpp short-form, CPU	"Claude"	❌ tail dropped (identical text)
transcribe-cpp short-form	"Foobar"	✅ full
transcribe-cpp short-form	none	✅ full
whisper.cpp (ggml-large-v3-turbo-q8_0, -l ru --prompt "Claude")	"Claude"	✅ full
whisper.cpp, no prompt	none	✅ full

Additional data points:

• Same content concatenated to 58 s → long-form seek path → ✅ full text even with the prompt. Only the short-form path is affected.
• Other archs on the same clip (GigaAM v3 e2e-RNNT, Canary 1b v2, Parakeet TDT v3) → ✅ full.
• The truncation is prompt-content-dependent ("Claude" trips it, "Foobar" doesn't), so any given user sees it "often but not always" across their notes — which is exactly how it was reported to me.

Analysis (hypothesis from reading src/arch/whisper/model.cpp)

With TIMESTAMPS_NONE the prefix is <|startofprev|> <prompt> <|sot|><|lang|><|task|><|notimestamps|> and the timestamp logits rules are skipped. In this mode EOS at a pause is a borderline decision for large-v3-turbo, and prev-context content can tip it over. Implementations that decode with timestamp rules (OpenAI reference, whisper.cpp, HF with return_timestamps=True) are protected by the pairing/coverage rules, which is consistent with the matrix above. Temperature fallback doesn't rescue the run because the truncated output is confident: avg_logprob/compression thresholds pass.

Suggestions

• Decode short-form with timestamp rules internally even when the caller requests text-only output (matching OpenAI/whisper.cpp behavior), or
• add a coverage guard: if decode ends while trailing speech remains (audio length vs last-attended position), retry with timestamps / without prev-context, or at minimum
• set transcribe_was_truncated() (or a new flag) in this case so consumers like Handy can warn or retry instead of silently storing partial text.

Related in spirit: #71 (Parakeet offline: multi-second span silently dropped) — different arch, same "silent partial output" failure class.

[tosoyn]

Audio sample attached (author's voice, shared with consent): https://gist.github.com/tosoyn/0390bdadf7da43a19fb8a5b079db02ce

Direct download: https://gist.githubusercontent.com/tosoyn/0390bdadf7da43a19fb8a5b079db02ce/raw/sample-7.6s-ru.wav (16 kHz mono s16, 7.62 s)

Quick check with this file:

handy -f sample-7.6s-ru.wav --json # custom_words ["Claude"] in settings → tail dropped

Expected tail after the pause: «…и они действительно длиннее, чем транскрипция.»

[cjpais]

@tosoyn mind trying #96 and seeing if this has been fixed for you? I've tried it on my side with your sample (ty) and it seems to work

[tosoyn]

Confirmed fixed on my original sample. I first reproduced the old truncation, then built b6a6aca / #96 and ran the same 7.62 s WAV with the Whisper Large v3 Turbo Q8 model, language=ru, timestamps=none, initial_prompt="Claude", and the CPU backend.

Before #96 (three runs, identical truncated output):

Притом я пошёл в Handy и посмотрел историю Я могу послушать аудио файлы

After #96 (--repeat 3):

Притом я пошёл в Handy и посмотрел историю, я могу послушать аудио файлы и они длиннее чем транскрипция

For comparison, current whisper.cpp produced:

Притом я пошёл в Handy и посмотрел историю, я могу послушать аудиофайлы и они действительно длиннее, чем транскрипция.

So the wording is not fully identical to whisper.cpp (transcribe.cpp omits “действительно”), but the silent tail truncation is gone: the final clause is preserved. Thanks for the fix!