A reasoning model's reply keeps only what follows its think marker

[aniruddhaadak80]

A reasoning model configured for extraction returns its thinking inline: settings/test came back as "OK</think>OK" (#690). utopia_extract::json_block takes everything from the first { to the last }, so a brace inside the reasoning cuts the wrong span — and only by luck has it not happened yet on the recall bench.

Why strip in LlmClient

Extraction, adjudication, governance batch looks, mappings, type resolution and the settings connectivity test all read LlmClient::chat. One cut there fixes every non-streaming consumer at once, instead of patching each parser. </think> is a protocol marker, not vocabulary; cutting before the last one guards against the answer itself quoting it; replies without the marker return untouched, so non-reasoning models see no change.

The endpoint's separate reasoning_content field needed no handling: this client only ever reads message.content, so that field is already ignored.

What changed

• utopia-llm LlmClient::chat returns the reply after the last </think> (strip_reasoning).
• utopia-extract json_block also skips past the last </think> before scanning for {. This is the backstop, not duplication: its contract is already to tolerate preamble (fences, prose), and the chat_tools path never passes through chat.
• Streaming paths (chat_stream, chat_tools_stream) deliberately untouched: deltas are yielded live to the UI, cutting the first half would desync already-emitted text.

Not in this PR (needs real infra)

• A reasoning model's extraction times out and its reply carries the reasoning #690 item 2 (longer/scaled read timeout for reasoning models) and item 3 (re-running the recall bench with V4 Flash): both need a live reasoning endpoint and the bench corpus, which I don't have. This PR is item 1 only, so it says Addresses, not Fixes.

Addresses #690

Verification

• New tests, each red without the fix:
  • a_reasoning_reply_keeps_only_what_follows_think (mock-socket through chat(), braces planted in the thinking to catch the span hazard),
  • a_reply_without_a_think_marker_is_untouched (non-reasoning replies byte-identical),
  • strip_reasoning_cuts_before_the_last_marker, strip_reasoning_fixes_the_settings_test_echo (the exact OK</think>OK shape),
  • parse_response_ignores_a_think_block_before_the_json (an unclosed { in the thinking makes the uncut span unparseable, so .unwrap() fails pre-fix and yields one entity post-fix).
• cargo fmt --all --check clean.
• Toolchain note: this machine has no C linker (no MSVC Build Tools, no gcc), so cargo test/clippy cannot link locally. Relying on CI (backend job) as proof; the string logic was additionally sanity-checked with an equivalent Node script (cut/uncut span parse behaviour confirmed). If CI flags anything I will follow up on the same branch.

[WaylandYang]

@aniruddhaadak80 thanks. You couldn't link locally, so I ran it on a merge with current dev: cargo clippy --workspace --all-targets -D warnings and fmt are clean; utopia-llm 21 tests and utopia-extract 57 tests pass, including your five new ones. Merging as is.

One thing the PR leaves open, which I'll note on #690 as well: the chat loop streams through RigModel, not LlmClient::chat, so a reasoning model configured for chat will still show its thinking inline in the answer. As you say, cutting a stream needs different handling, so it doesn't belong in this change.

🤖 Generated with Claude Code