Uh oh!
There was an error while loading. Please reload this page.
Take tinyinference's null-tolerant sequence decoder - #138
Conversation
Several OpenAI-compatible endpoints spell "no tool calls" as "tool_calls": null rather than by omitting the key, and Mistral-family models do it on every plain-text completion. #[serde(default)] covers only an absent key, so such a response failed the whole decode with "invalid type: null, expected a sequence" — a model that simply answered in prose surfaced to the harness as a transport fault. Any agent on such a rung could not complete a single turn. Found on a Lean-specialised model, where a sub-agent delegation failed 100% of the time. tinyhumansai/tinyinference#4. Co-authored-by: Medulla <medulla@tinyhumans.ai>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe PR updates the ChangesTinyinference update
Estimated code review effort: 1 (Trivial) | ~2 minutes Merge Risk:⚪ Minimal · up to This localized dependency update makes OpenAI-compatible responses with null tool-call sequences decode successfully without changing the public API; no actionable merge-blocking risk remains after normal checks and review. Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.) Warning Your free Security trial is over. An organization admin can activate Security or dismiss this notice. Comment |
What changed
The
vendor/tinyinferencegitlink, todbf7897— the merge oftinyhumansai/tinyinference#4. Nothing else.
Why
Several OpenAI-compatible endpoints spell "no tool calls" as
"tool_calls": nullrather than by omitting the key, and Mistral-family models do it on every
plain-text completion:
{"choices":[{"index":0,"finish_reason":"stop", "message":{"role":"assistant","tool_calls":null,"content":"Hi!"}}]}#[serde(default)]covers an absent key and nothing else — a key present withnullstill reaches theVecvisitor and fails the whole response withinvalid type: null, expected a sequence.The consequence for this crate is not a degraded response, it is a role that
cannot function. Found on a Lean-specialised rung (
labs-leanstral-1-5), whereevery sub-agent delegation to that model failed with
serialization error: invalid type: null, expected a sequencewhile the modelitself was answering perfectly. Any agent placed on such a rung could not
complete a single turn.
Public API or behavior changes
None here. One behavior change inherited from the bump, pinned by a test
upstream: a body carrying
"choices": nullnow fails withModel("openai response contained no choices")rather than a serde error — theprovider fact a caller can act on, rather than the transport being named for a
body it read perfectly.
Validation
Upstream:
cargo fmt --checkok,cargo clippy --all-targets -D warnings0warnings,
cargo test -p tinyinference --lib266 passed, three new tests for thenull shapes.
Summary by CodeRabbit