docs: focus coding-agents guide on Claude Code with translation proxy

[typhoonzero]

Summary

Extend the coding agents guide to cover reasoning models, Claude Code proxy setup, and model recommendations for on-premise LLMs. Also update the MLOps tuning guide to cover llama.cpp (GGUF) and GPU-level observability.

Files changed

• docs/en/agentic_mlops/coding-agents-with-inference-service.mdx
• docs/en/agentic_mlops/mlops-with-coding-agents.mdx

coding-agents-with-inference-service.mdx changes

Claude Code translation proxy

• Claude Code speaks the Anthropic Messages API and now requires a translation proxy
• Two proxy options documented:
  • Option 1: LiteLLM proxy — general-purpose proxy exposing /v1/messages endpoint
  • Option 2: claude-code-router — Claude Code-specific proxy, config-driven model selection via ~/.claude-code-router/config.json
• Both options include complete environment variable setup and on-prem guardrail flags

Step 2b: Configure reasoning models and reasoning effort

• Added server-side flags table mapping model families (DeepSeek R1, QwQ, Qwen3, Hunyuan-A13B, Cohere Command A) to their --tool-call-parser + --reasoning-parser values
• Documented reasoning effort controls: server-side --chat-template defaults, request-time chat_template_kwargs, thinking_token_budget, and reasoning_effort (where supported)
• Removed the incorrect --default-chat-template-kwargs flag (non-existent in vLLM)

Model recommendations

• Added Qwen3.6 and Gemma 4 as recommended model families for coding agents, with vLLM parser guidance
• Added Unsloth quantized models table (8 models: GGUF, QAT, MoE variants)
• Updated hardware fit guide to recommend specific Unsloth models per GPU tier
• Updated "vibe coding" guidance to start with Qwen3.6 or Gemma 4

Architecture update

• Updated the architecture diagram to show the Claude Code → proxy → InferenceService path separately from opencode/Codex CLI → InferenceService
• Kept opencode and Codex CLI sections — they call the InferenceService directly since they speak OpenAI Chat Completions API natively

mlops-with-coding-agents.mdx changes

Extended tuning guide for llama.cpp (GGUF)

• Removed vLLM-only framing; added llama.cpp-specific knobs (--cache-type-k, --cache-type-v, --ctx-size, --parallel, --reasoning/--reasoning-budget)
• Added llama.cpp examples in the one-change-per-iteration section

Reasoning overhead as a benchmarking metric

• Added reasoning-overhead ratio measurement: reasoning_tokens / content_tokens
• New best practices guardrail: account for reasoning overhead in benchmark reports

GPU-level metrics in performance tuning

• Added GPU memory headroom target (≥10% free for KV cache growth)
• Step 4 requires checking GPU utilization, memory usage, and power draw
• Added GPU memory headroom <5% as a hardware ceiling

Observability guidance

• Added guidance on enabling metrics: vLLM OpenAI-compatible /metrics endpoint and llama.cpp --metrics flag
• Fallback to per-request latency measurement if metrics endpoint is unavailable

PR history

This PR builds on the initial coding agents documentation (#245) by extending coverage to reasoning models, Claude Code proxy setup, and model recommendations.

[coderabbitai]

Walkthrough

Clarifies that tuning should be agent-driven, adds a GPU memory headroom SLO, expands benchmark metrics to include TTFT/ITL/TPS and reasoning overhead, provides richer one-change proposals and GPU-level monitoring, and adds guardrails for measuring and bounding reasoning costs.

Changes

Agent-driven tuning workflow revisions

Layer / File(s)	Summary
Agent-driven tuning intro and knobs docs/en/agentic_mlops/mlops-with-coding-agents.mdx	Adds an introductory sentence specifying agent-led tuning and lists runtime-specific knobs/flags to consider (vLLM, llama.cpp/GGUF parameters).
Expanded tuning loop and benchmarks docs/en/agentic_mlops/mlops-with-coding-agents.mdx	Enriches the tuning loop: adds GPU memory headroom objective; requires TTFT/ITL/TPS collection for streaming vs non-streaming; requires measuring reasoning overhead when enabled; updates example change proposals (KV-cache, reasoning budgets, concurrency); and expands apply/measure/record with GPU-level checks and stopping/ceiling conditions.
Reasoning overhead guardrail docs/en/agentic_mlops/mlops-with-coding-agents.mdx	Adds a best-practices guardrail to explicitly account for reasoning overhead during benchmarking, gives an example overhead ratio, and provides runtime-specific advice for bounding reasoning work.

🎯 3 (Moderate) | ⏱️ ~20 minutes

A rabbit tunes with careful hops and cheer,
measuring tokens, headroom, GPU gear.
Benchmarks hum, reasoning costs in sight,
agents tweak, record, then sleep at night. 🐇🔧✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately reflects the main focus: narrowing the guide to Claude Code with translation proxy configuration, which is the most substantial change across both documentation files.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch docs/coding-agents-inference-service

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/en/agentic_mlops/coding-agents-with-inference-service.mdx`:
- Line 106: Clarify that the chat template path
`examples/tool_chat_template_deepseekr1.jinja` referenced in the DeepSeek R1
table is relative to the vLLM repository root, or replace it with a full
external URL (e.g., the GitHub or HuggingFace raw file URL) so users can locate
the template; update the table cell for DeepSeek R1 / `deepseek_v3` to either
prepend "vLLM repo root:" before the relative path or supply the canonical
remote link to the template.
- Around line 135-143: The example JSON incorrectly nests the reasoning_effort
setting inside an SDK-specific extra_body object; update the JSON example so
reasoning_effort is a top-level field (remove the extra_body wrapper) — locate
the snippet that defines "extra_body" and "reasoning_effort" and move
reasoning_effort out to be a sibling of "model" and "messages" so the request
body matches the raw HTTP API.
- Around line 104-111: Update the model table entries so the vLLM parser flags
match documented support: remove `deepseek_v3` from the DeepSeek R1 row and set
its `--reasoning-parser` to `deepseek_r1` (no `--tool-call-parser`), change the
QwQ / Qwen row to use `--tool-call-parser hermes` and `--reasoning-parser
deepseek_r1`, change Hunyuan-A13B-Instruct to use `--tool-call-parser
hunyuan_a13b` and `--reasoning-parser hunyuan_a13b` (replace `deepseek_v3`), and
change Cohere Command A Reasoning to `--tool-call-parser cohere_command3` and
`--reasoning-parser cohere_command3` (note the optional `cohere_melody`
dependency); keep the guidance to consult model cards and the vLLM tool calling
docs for other families.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 09ee6c7b-8d23-4d8c-8c3d-4c48b4e1439f

📥 Commits

Reviewing files that changed from the base of the PR and between 0ab8e31 and c0730ef.

📒 Files selected for processing (1)

• docs/en/agentic_mlops/coding-agents-with-inference-service.mdx

[coderabbitai]

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Clarify the chat template path.

The path examples/tool_chat_template_deepseekr1.jinja is relative without context. Users won't know where to find this file. Clarify that this path is relative to the vLLM repository root, or provide the full HuggingFace/GitHub URL where users can locate the template.

📝 Suggested clarification

-| DeepSeek R1 (`deepseek-ai/DeepSeek-R1-*`) | `deepseek_v3` | *(none required)* | Also needs `--chat-template examples/tool_chat_template_deepseekr1.jinja` |
+| DeepSeek R1 (`deepseek-ai/DeepSeek-R1-*`) | `deepseek_v3` | *(none required)* | Also needs `--chat-template` pointing to `examples/tool_chat_template_deepseekr1.jinja` from the vLLM repository |

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	| DeepSeek R1 (`deepseek-ai/DeepSeek-R1-*`) | `deepseek_v3` | *(none required)* | Also needs `--chat-template examples/tool_chat_template_deepseekr1.jinja` |
	| DeepSeek R1 (`deepseek-ai/DeepSeek-R1-*`) | `deepseek_v3` | *(none required)* | Also needs `--chat-template` pointing to `examples/tool_chat_template_deepseekr1.jinja` from the vLLM repository |

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/en/agentic_mlops/coding-agents-with-inference-service.mdx` at line 106,
Clarify that the chat template path
`examples/tool_chat_template_deepseekr1.jinja` referenced in the DeepSeek R1
table is relative to the vLLM repository root, or replace it with a full
external URL (e.g., the GitHub or HuggingFace raw file URL) so users can locate
the template; update the table cell for DeepSeek R1 / `deepseek_v3` to either
prepend "vLLM repo root:" before the relative path or supply the canonical
remote link to the template.

[cloudflare-workers-and-pages]

Deploying alauda-ai with    Cloudflare Pages

Latest commit:	b18b5cd
Status:	✅  Deploy successful!
Preview URL:	https://2b48441d.alauda-ai.pages.dev
Branch Preview URL:	https://docs-coding-agents-inference.alauda-ai.pages.dev

View logs