Summary
The Anthropic Messages API's output_config parameter (GA structured outputs) is not captured in span metadata. When users make structured output calls with output_config={"format": {"type": "json_schema", "schema": {...}}}, the JSON schema constraint is silently dropped from the Braintrust span, making it impossible to see what schema the model was constrained to.
The legacy output_format parameter (beta-era name, still supported during transition) is also not tracked.
What is missing
The METADATA_PARAMS tuple in the Anthropic tracing module explicitly allowlists which request parameters are logged as span metadata:
METADATA_PARAMS= (
"model",
"max_tokens",
"temperature",
"top_k",
"top_p",
"stop_sequences",
"tool_choice",
"tools",
"stream",
"thinking",
)
Neither output_config nor output_format is included. By contrast:
| Provider | Structured output parameter | Captured? |
|---|
| OpenAI | response_format | Yes — all request params flow into metadata via pass-through approach |
| Mistral | response_format | Yes — explicitly listed in _CHAT_METADATA_KEYS and _AGENTS_METADATA_KEYS |
| Google GenAI | response_schema / response_json_schema | Yes — included in input via config deep-copy |
| Anthropic | output_config / output_format | No |
The Anthropic integration is the only major provider wrapper that uses a restrictive allowlist approach for metadata, causing it to fall behind when new parameters are added upstream.
Why this matters
- Structured outputs is a GA feature on the Anthropic API, supported on Claude Opus 4.6, Sonnet 4.6, Sonnet 4.5, Opus 4.5, and Haiku 4.5
- The
output_config parameter carries the JSON schema that constrains the model's output — critical context for understanding model behavior and debugging schema mismatches - Users cannot see what schema their model was constrained to when reviewing Braintrust spans
- The
strict: true parameter on individual tool definitions IS captured (since tools is in METADATA_PARAMS), but the top-level output_config for JSON output mode is not
Braintrust docs status
unclear — The Anthropic integration page mentions "Structured outputs through the gateway" but does not document whether the SDK wrapper captures output_config in span metadata.
Upstream sources
Local files inspected
py/src/braintrust/integrations/anthropic/tracing.py — METADATA_PARAMS (line 30–41) does not include output_config or output_format; _get_metadata_from_kwargs (line 453–459) only extracts keys listed in METADATA_PARAMSpy/src/braintrust/integrations/anthropic/_utils.py — no reference to output_configpy/src/braintrust/integrations/anthropic/test_anthropic.py — no test cases for structured outputspy/src/braintrust/integrations/openai/tracing.py — OpenAI uses a pass-through approach ({**params, "provider": "openai"}) that captures ALL request params including response_formatpy/src/braintrust/integrations/mistral/tracing.py — Mistral explicitly includes response_format in _CHAT_METADATA_KEYS (line 37)
Summary
The Anthropic Messages API's
output_configparameter (GA structured outputs) is not captured in span metadata. When users make structured output calls withoutput_config={"format": {"type": "json_schema", "schema": {...}}}, the JSON schema constraint is silently dropped from the Braintrust span, making it impossible to see what schema the model was constrained to.The legacy
output_formatparameter (beta-era name, still supported during transition) is also not tracked.What is missing
The
METADATA_PARAMStuple in the Anthropic tracing module explicitly allowlists which request parameters are logged as span metadata:Neither
output_confignoroutput_formatis included. By contrast:response_formatresponse_format_CHAT_METADATA_KEYSand_AGENTS_METADATA_KEYSresponse_schema/response_json_schemaoutput_config/output_formatThe Anthropic integration is the only major provider wrapper that uses a restrictive allowlist approach for metadata, causing it to fall behind when new parameters are added upstream.
Why this matters
output_configparameter carries the JSON schema that constrains the model's output — critical context for understanding model behavior and debugging schema mismatchesstrict: trueparameter on individual tool definitions IS captured (sincetoolsis inMETADATA_PARAMS), but the top-leveloutput_configfor JSON output mode is notBraintrust docs status
unclear — The Anthropic integration page mentions "Structured outputs through the gateway" but does not document whether the SDK wrapper captures
output_configin span metadata.Upstream sources
output_formatparameter has moved tooutput_config.formatand beta headers are no longer requiredmessages.create()acceptsoutput_configas a top-level parameterLocal files inspected
py/src/braintrust/integrations/anthropic/tracing.py—METADATA_PARAMS(line 30–41) does not includeoutput_configoroutput_format;_get_metadata_from_kwargs(line 453–459) only extracts keys listed inMETADATA_PARAMSpy/src/braintrust/integrations/anthropic/_utils.py— no reference tooutput_configpy/src/braintrust/integrations/anthropic/test_anthropic.py— no test cases for structured outputspy/src/braintrust/integrations/openai/tracing.py— OpenAI uses a pass-through approach ({**params, "provider": "openai"}) that captures ALL request params includingresponse_formatpy/src/braintrust/integrations/mistral/tracing.py— Mistral explicitly includesresponse_formatin_CHAT_METADATA_KEYS(line 37)