A reasoning model's extraction times out and its reply carries the reasoning

[WaylandYang]

What happens

Configured with deepseek-ai/DeepSeek-V4-Flash (a reasoning model, available on SiliconFlow), extraction on the NVIDIA recall corpus did not finish in an hour:

• 64 chunks took over 60 minutes, against about 25 with DeepSeek-V3, and the bench harness timed out at 63/64;
• 10 chunks failed with LLM endpoint gave no usable answer: error sending request for url (…/chat/completions), which looks like LlmClient's read timeout on long reasoning replies;
• there was no rate limiting at all (0 throttle warnings, against 45–96 per V3 round the same night), so the model is attractive precisely when V3 is throttled.

Separately, the model returns its reasoning inline: settings/test came back as "OK</think>OK". utopia_extract::json_block takes everything from the first { to the last }, so a brace inside the reasoning would make it cut the wrong span. It did not happen in this round, but only by luck.

Proposed

1. Keep only the reply after the reasoning. Strip everything up to the last </think> in the chat reply before any parsing, in LlmClient, so extraction, chat and adjudication all benefit. This is a protocol marker, not vocabulary. Also check whether the endpoint returns reasoning_content separately and ignore that field.
2. A longer read timeout for reasoning models, or a timeout that scales with the requested completion budget, so a long reasoning reply isn't cut off and retried.
3. Re-run the recall bench with V4 Flash after 1 and 2, so it can be offered as the stable option when V3 is throttled.

[WaylandYang]

Item 1 (keep only the reply after the reasoning) is in via #707 for everything that reads LlmClient::chat: extraction, adjudication, governance, the settings test. Still open:

• The chat loop streams through RigModel, not chat. A reasoning model configured for chat will still show its thinking inline in the answer. The stream needs to hold back text until </think> arrives, or drop the <think> span as it streams.
• Items 2 and 3 are still to do: a longer, or budget-scaled, read timeout for reasoning replies, and a recall-bench round with DeepSeek-V4-Flash once that timeout is in.

🤖 Generated with Claude Code

[WaylandYang]

All three are answered, so closing.

Strip the reasoning before parsing. strip_reasoning cuts everything up to the last </think> in LlmClient before any caller sees the string, and the streamed path added in #757 reads only delta.content, so a reasoning_content field never reaches the parser.

A read timeout that does not kill a thinking model. The timeout is not longer, it measures the right thing. A non-streamed call is silent until the whole answer exists, so the model's thinking counted as silence: one dense passage crossed 300 seconds three times and was declared dead. Streamed, the same passage has bytes at 2.6 seconds and finishes at 231 (#757). A stream that stops mid-answer is now its own failure and is retried, and a chunk that still cannot be extracted leaves a drop row.

Re-run the bench with V4 Flash. It has been the extraction model for every run since #731. The four NVDA filings, 55 chunks: 885 open statements, misworded 3.7%, not stated 0.0%, table figures reaching no statement 2%. One thing this issue could not have known: the model needs its reasoning for this work. With thinking off it answers {"e":[],"s":[],"n":[]} for table chunks and the same four filings give 460 statements. Capping the reasoning was measured in #759 and left alone, because halving it costs a sixth of the table figures for a quarter of the wall clock.

What is left of the latency is #588, where a document's chunks are extracted one at a time.