Skip to content

feat(bedrock): trace bedrock runtime calls#519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit into from
Jun 16, 2026
Merged

feat(bedrock): trace bedrock runtime calls #519

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions examples/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,7 @@ Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `bedrock_runtime/` | boto3 Bedrock Runtime `converse()` call against Amazon Nova Lite |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
Expand Down
25 changes: 25 additions & 0 deletions examples/bedrock_runtime/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
# AWS Bedrock + Braintrust

Calls `braintrust.auto_instrument()` to patch boto3's Bedrock Runtime client factory, then runs a single `client.converse(...)` call against Amazon Nova Lite. The trace captures a parent `answer_question` span plus a child `bedrock.converse` LLM span with input, output, metadata, and token metrics.

## Run

```bash
export BRAINTRUST_API_KEY=...
export AWS_BEARER_TOKEN_BEDROCK=...
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

uv sync
uv run python example.py
```

If you use IAM credentials instead of a Bedrock API key, set the usual AWS variables instead:

```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if using temporary credentials
```

The default model is `us.amazon.nova-lite-v1:0`. Override it with `BRAINTRUST_BEDROCK_MODEL` if needed.
38 changes: 38 additions & 0 deletions examples/bedrock_runtime/example.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env python
"""boto3 Bedrock Runtime client traced via braintrust.auto_instrument()."""

import os

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-bedrock")

import boto3 # noqa: E402


MODEL = os.getenv("BRAINTRUST_BEDROCK_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"


@braintrust.traced
def answer_question(question: str) -> str:
client = boto3.client("bedrock-runtime", region_name=REGION)
response = client.converse(
modelId=MODEL,
messages=[{"role": "user", "content": [{"text": question}]}],
inferenceConfig={"maxTokens": 64, "temperature": 0},
)
message = response["output"]["message"]
return "".join(block.get("text", "") for block in message.get("content", []))


def main() -> None:
answer = answer_question("What is the capital of Australia? Reply in one sentence.")
print(answer)
braintrust.flush()


if __name__ == "__main__":
main()
12 changes: 12 additions & 0 deletions examples/bedrock_runtime/pyproject.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
[project]
name = "braintrust-bedrock-example"
version = "0.1.0"
description = "boto3 Bedrock Runtime client traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
"braintrust",
"boto3",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }
13 changes: 13 additions & 0 deletions py/noxfile.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -230,6 +230,19 @@ def test_cohere(session, version):
_run_tests(session, f"{INTEGRATION_DIR}/cohere/test_cohere.py", version=version)


BOTO3_VERSIONS = _get_matrix_versions("boto3")


@nox.session()
@nox.parametrize("version", BOTO3_VERSIONS, ids=BOTO3_VERSIONS)
def test_bedrock_runtime(session, version):
"""Test the boto3 Bedrock Runtime integration."""
_install_test_deps(session)
_install_matrix_dep(session, "boto3", version)
_install_matrix_dep(session, "botocore", version)
_run_tests(session, f"{INTEGRATION_DIR}/bedrock_runtime/test_bedrock_runtime.py", version=version)


INSTRUCTOR_VERSIONS = _get_matrix_versions("instructor")


Expand Down
11 changes: 11 additions & 0 deletions py/pyproject.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,14 @@ latest = "pytest==9.0.3"
[tool.braintrust.matrix.braintrust-core]
latest = "braintrust-core==0.0.59"

[tool.braintrust.matrix.boto3]
latest = "boto3==1.43.29"
"1.34.116" = "boto3==1.34.116"

[tool.braintrust.matrix.botocore]
latest = "botocore==1.43.29"
"1.34.116" = "botocore==1.34.116"

# ---------------------------------------------------------------------------
# Vendor packages — optional third-party packages the SDK can work without.
# Keys are matrix keys; values are Python import names. The noxfile uses this
Expand All@@ -461,6 +469,7 @@ agentscope = ["agentscope"]
agno = ["agno"]
autogen = ["autogen-agentchat"]
anthropic = ["anthropic"]
bedrock_runtime = ["boto3", "botocore"]
cohere = ["cohere"]
claude_agent_sdk = ["claude-agent-sdk"]
crewai = ["crewai"]
Expand DownExpand Up@@ -488,6 +497,8 @@ anthropic = "anthropic"
cohere = "cohere"
autoevals = "autoevals"
braintrust-core = "braintrust_core"
boto3 = "boto3"
botocore = "botocore"
crewai = "crewai"
dspy = "dspy"
google-adk = "google.adk"
Expand Down
5 changes: 5 additions & 0 deletions py/src/braintrust/auto.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,7 @@
AgnoIntegration,
AnthropicIntegration,
AutoGenIntegration,
BedrockRuntimeIntegration,
ClaudeAgentSDKIntegration,
CohereIntegration,
CrewAIIntegration,
Expand DownExpand Up@@ -72,6 +73,7 @@ def auto_instrument(
openai_agents: bool = True,
cohere: bool = True,
autogen: bool = True,
bedrock: bool = True,
crewai: bool = True,
strands: bool = True,
temporal: bool = True,
Expand DownExpand Up@@ -106,6 +108,7 @@ def auto_instrument(
openai_agents: Enable OpenAI Agents SDK instrumentation (default: True)
cohere: Enable Cohere instrumentation (default: True)
autogen: Enable AutoGen instrumentation (default: True)
bedrock: Enable boto3 Bedrock Runtime instrumentation (default: True)
crewai: Enable CrewAI instrumentation (default: True)
strands: Enable Strands Agents instrumentation (default: True)
temporal: Enable Temporal instrumentation (default: True)
Expand DownExpand Up@@ -195,6 +198,8 @@ def auto_instrument(
results["cohere"] = _instrument_integration(CohereIntegration)
if autogen:
results["autogen"] = _instrument_integration(AutoGenIntegration)
if bedrock:
results["bedrock_runtime"] = _instrument_integration(BedrockRuntimeIntegration)
if crewai:
results["crewai"] = _instrument_integration(CrewAIIntegration)
if strands:
Expand Down
6 changes: 6 additions & 0 deletions py/src/braintrust/conftest.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -247,6 +247,12 @@ def get_vcr_config():
"openai-api-key",
"x-goog-api-key",
"x-bt-auth-token",
"x-amz-security-token",
"x-amz-date",
"x-amz-content-sha256",
"amz-sdk-invocation-id",
"amz-sdk-request",
"x-amzn-bedrock-api-key",
],
}

Expand Down
2 changes: 2 additions & 0 deletions py/src/braintrust/integrations/__init__.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@
from .agno import AgnoIntegration
from .anthropic import AnthropicIntegration
from .autogen import AutoGenIntegration
from .bedrock_runtime import BedrockRuntimeIntegration
from .claude_agent_sdk import ClaudeAgentSDKIntegration
from .cohere import CohereIntegration
from .crewai import CrewAIIntegration
Expand All@@ -29,6 +30,7 @@
"AgnoIntegration",
"AnthropicIntegration",
"AutoGenIntegration",
"BedrockRuntimeIntegration",
"ClaudeAgentSDKIntegration",
"CohereIntegration",
"CrewAIIntegration",
Expand Down
43 changes: 43 additions & 0 deletions py/src/braintrust/integrations/auto_test_scripts/test_auto_bedrock_runtime.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
"""Test auto_instrument for boto3 Bedrock Runtime."""

import os


os.environ.setdefault("AWS_EC2_METADATA_DISABLED", "true")
if not os.environ.get("AWS_PROFILE") and not os.environ.get("AWS_BEARER_TOKEN_BEDROCK"):
os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
os.environ.setdefault("AWS_SESSION_TOKEN", "testing")

import boto3
from braintrust.auto import auto_instrument
from braintrust.integrations.test_utils import autoinstrument_test_context


MODEL = os.getenv("BRAINTRUST_BEDROCK_CONVERSE_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"

results = auto_instrument()
assert results.get("bedrock_runtime") is True

results2 = auto_instrument()
assert results2.get("bedrock_runtime") is True

with autoinstrument_test_context("test_auto_bedrock_runtime", integration="bedrock_runtime") as memory_logger:
client = boto3.client("bedrock-runtime", region_name=REGION)
response = client.converse(
modelId=MODEL,
messages=[{"role": "user", "content": [{"text": "Say hello in one word."}]}],
inferenceConfig={"maxTokens": 20, "temperature": 0},
)
assert response["output"]["message"]["role"] == "assistant"

spans = memory_logger.pop()
assert len(spans) == 1, f"Expected 1 span, got {len(spans)}"
span = spans[0]
assert span["metadata"]["provider"] == "bedrock"
assert span["metadata"]["model"] == MODEL
assert span["metadata"]["endpoint"] == "converse"
assert span["span_attributes"]["name"] == "bedrock.converse"

print("SUCCESS")
17 changes: 17 additions & 0 deletions py/src/braintrust/integrations/bedrock_runtime/__init__.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
"""Public entry points for the boto3 Bedrock Runtime integration."""

from .integration import BedrockRuntimeIntegration
from .patchers import wrap_bedrock_client


__all__ = ["BedrockRuntimeIntegration", "setup_bedrock", "wrap_bedrock"]


def setup_bedrock() -> bool:
"""Patch botocore client creation to auto-wrap Bedrock Runtime clients."""
return BedrockRuntimeIntegration.setup()


def wrap_bedrock(client):
"""Instrument a boto3 Bedrock Runtime client instance in place."""
return wrap_bedrock_client(client)
35 changes: 35 additions & 0 deletions ...braintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_auto_bedrock_runtime.yaml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
interactions:
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "Say hello in one word."}]}],
"inferenceConfig": {"maxTokens": 20, "temperature": 0}}'
headers:
Content-Length:
- '137'
Content-Type:
- !!binary |
YXBwbGljYXRpb24vanNvbg==
User-Agent:
- !!binary |
Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9aLGIs
RCwzIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":511},"output":{"message":{"content":[{"text":"Hi."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":6,"outputTokens":3,"serverToolUsage":{},"totalTokens":9}}'
headers:
Connection:
- keep-alive
Content-Length:
- '202'
Content-Type:
- application/json
Date:
- Mon, 15 Jun 2026 20:24:03 GMT
x-amzn-RequestId:
- caa9fd4f-69d2-459b-baf5-0ad888a79845
status:
code: 200
message: OK
version: 1
36 changes: 36 additions & 0 deletions ...edrock_runtime/cassettes/1.34.116/test_setup_bedrock_converse_auto_wraps_new_clients.yaml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
[{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
{"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
headers:
Content-Length:
- '242'
Content-Type:
- !!binary |
YXBwbGljYXRpb24vanNvbg==
User-Agent:
- !!binary |
Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":613},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
headers:
Connection:
- keep-alive
Content-Length:
- '203'
Content-Type:
- application/json
Date:
- Mon, 15 Jun 2026 20:24:01 GMT
x-amzn-RequestId:
- 7c3bdcc8-1124-46b5-982e-3ced2d1a5371
status:
code: 200
message: OK
version: 1
36 changes: 36 additions & 0 deletions ...raintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_wrap_bedrock_converse.yaml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
[{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
{"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
headers:
Content-Length:
- '242'
Content-Type:
- !!binary |
YXBwbGljYXRpb24vanNvbg==
User-Agent:
- !!binary |
Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":486},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
headers:
Connection:
- keep-alive
Content-Length:
- '203'
Content-Type:
- application/json
Date:
- Mon, 15 Jun 2026 20:24:00 GMT
x-amzn-RequestId:
- 1935727d-5309-4a70-8c15-1ce0ffbe877e
status:
code: 200
message: OK
version: 1
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
feat(bedrock): trace bedrock runtime calls by AbhiPrasad · Pull Request #519 · braintrustdata/braintrust-sdk-python · GitHub
Skip to content
feat(bedrock): trace bedrock runtime calls#519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Abhijeet Prasad (AbhiPrasad)
merged 1 commit into

from

Jun 16, 2026
 Merged


feat(bedrock): trace bedrock runtime calls
#519
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
examples/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,7 @@ Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `bedrock_runtime/` | boto3 Bedrock Runtime `converse()` call against Amazon Nova Lite |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
Expand Down

25 changes: 25 additions & 0 deletions
examples/bedrock_runtime/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
# AWS Bedrock + Braintrust

Calls `braintrust.auto_instrument()` to patch boto3's Bedrock Runtime client factory, then runs a single `client.converse(...)` call against Amazon Nova Lite. The trace captures a parent `answer_question` span plus a child `bedrock.converse` LLM span with input, output, metadata, and token metrics.

## Run

```bash
export BRAINTRUST_API_KEY=...
export AWS_BEARER_TOKEN_BEDROCK=...
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

uv sync
uv run python example.py
```

If you use IAM credentials instead of a Bedrock API key, set the usual AWS variables instead:

```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if using temporary credentials
```

The default model is `us.amazon.nova-lite-v1:0`. Override it with `BRAINTRUST_BEDROCK_MODEL` if needed.

38 changes: 38 additions & 0 deletions
examples/bedrock_runtime/example.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env python
"""boto3 Bedrock Runtime client traced via braintrust.auto_instrument()."""

import os

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-bedrock")

import boto3 # noqa: E402


MODEL = os.getenv("BRAINTRUST_BEDROCK_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"


@braintrust.traced
def answer_question(question: str) -> str:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": question}]}],
 inferenceConfig={"maxTokens": 64, "temperature": 0},
 )
 message = response["output"]["message"]
 return "".join(block.get("text", "") for block in message.get("content", []))


def main() -> None:
 answer = answer_question("What is the capital of Australia? Reply in one sentence.")
 print(answer)
 braintrust.flush()


if __name__ == "__main__":
 main()

12 changes: 12 additions & 0 deletions
examples/bedrock_runtime/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
[project]
name = "braintrust-bedrock-example"
version = "0.1.0"
description = "boto3 Bedrock Runtime client traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
 "braintrust",
 "boto3",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }

13 changes: 13 additions & 0 deletions
py/noxfile.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -230,6 +230,19 @@ def test_cohere(session, version):
 _run_tests(session, f"{INTEGRATION_DIR}/cohere/test_cohere.py", version=version)


BOTO3_VERSIONS = _get_matrix_versions("boto3")


@nox.session()
@nox.parametrize("version", BOTO3_VERSIONS, ids=BOTO3_VERSIONS)
def test_bedrock_runtime(session, version):
 """Test the boto3 Bedrock Runtime integration."""
 _install_test_deps(session)
 _install_matrix_dep(session, "boto3", version)
 _install_matrix_dep(session, "botocore", version)
 _run_tests(session, f"{INTEGRATION_DIR}/bedrock_runtime/test_bedrock_runtime.py", version=version)


INSTRUCTOR_VERSIONS = _get_matrix_versions("instructor")


Expand Down

11 changes: 11 additions & 0 deletions
py/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,14 @@ latest = "pytest==9.0.3"
[tool.braintrust.matrix.braintrust-core]
latest = "braintrust-core==0.0.59"

[tool.braintrust.matrix.boto3]
latest = "boto3==1.43.29"
"1.34.116" = "boto3==1.34.116"

[tool.braintrust.matrix.botocore]
latest = "botocore==1.43.29"
"1.34.116" = "botocore==1.34.116"

# ---------------------------------------------------------------------------
# Vendor packages — optional third-party packages the SDK can work without.
# Keys are matrix keys; values are Python import names. The noxfile uses this
Expand All@@ -461,6 +469,7 @@ agentscope = ["agentscope"]
agno = ["agno"]
autogen = ["autogen-agentchat"]
anthropic = ["anthropic"]
bedrock_runtime = ["boto3", "botocore"]
cohere = ["cohere"]
claude_agent_sdk = ["claude-agent-sdk"]
crewai = ["crewai"]
Expand DownExpand Up@@ -488,6 +497,8 @@ anthropic = "anthropic"
cohere = "cohere"
autoevals = "autoevals"
braintrust-core = "braintrust_core"
boto3 = "boto3"
botocore = "botocore"
crewai = "crewai"
dspy = "dspy"
google-adk = "google.adk"
Expand Down

5 changes: 5 additions & 0 deletions
py/src/braintrust/auto.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,7 @@
 AgnoIntegration,
 AnthropicIntegration,
 AutoGenIntegration,
 BedrockRuntimeIntegration,
 ClaudeAgentSDKIntegration,
 CohereIntegration,
 CrewAIIntegration,
Expand DownExpand Up@@ -72,6 +73,7 @@ def auto_instrument(
 openai_agents: bool = True,
 cohere: bool = True,
 autogen: bool = True,
 bedrock: bool = True,
 crewai: bool = True,
 strands: bool = True,
 temporal: bool = True,
Expand DownExpand Up@@ -106,6 +108,7 @@ def auto_instrument(
 openai_agents: Enable OpenAI Agents SDK instrumentation (default: True)
 cohere: Enable Cohere instrumentation (default: True)
 autogen: Enable AutoGen instrumentation (default: True)
 bedrock: Enable boto3 Bedrock Runtime instrumentation (default: True)
 crewai: Enable CrewAI instrumentation (default: True)
 strands: Enable Strands Agents instrumentation (default: True)
 temporal: Enable Temporal instrumentation (default: True)
Expand DownExpand Up@@ -195,6 +198,8 @@ def auto_instrument(
 results["cohere"] = _instrument_integration(CohereIntegration)
 if autogen:
 results["autogen"] = _instrument_integration(AutoGenIntegration)
 if bedrock:
 results["bedrock_runtime"] = _instrument_integration(BedrockRuntimeIntegration)
 if crewai:
 results["crewai"] = _instrument_integration(CrewAIIntegration)
 if strands:
Expand Down

6 changes: 6 additions & 0 deletions
py/src/braintrust/conftest.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -247,6 +247,12 @@ def get_vcr_config():
 "openai-api-key",
 "x-goog-api-key",
 "x-bt-auth-token",
 "x-amz-security-token",
 "x-amz-date",
 "x-amz-content-sha256",
 "amz-sdk-invocation-id",
 "amz-sdk-request",
 "x-amzn-bedrock-api-key",
 ],
 }

Expand Down

2 changes: 2 additions & 0 deletions
py/src/braintrust/integrations/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@
from .agno import AgnoIntegration
from .anthropic import AnthropicIntegration
from .autogen import AutoGenIntegration
from .bedrock_runtime import BedrockRuntimeIntegration
from .claude_agent_sdk import ClaudeAgentSDKIntegration
from .cohere import CohereIntegration
from .crewai import CrewAIIntegration
Expand All@@ -29,6 +30,7 @@
 "AgnoIntegration",
 "AnthropicIntegration",
 "AutoGenIntegration",
 "BedrockRuntimeIntegration",
 "ClaudeAgentSDKIntegration",
 "CohereIntegration",
 "CrewAIIntegration",
Expand Down

43 changes: 43 additions & 0 deletions
py/src/braintrust/integrations/auto_test_scripts/test_auto_bedrock_runtime.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
"""Test auto_instrument for boto3 Bedrock Runtime."""

import os


os.environ.setdefault("AWS_EC2_METADATA_DISABLED", "true")
if not os.environ.get("AWS_PROFILE") and not os.environ.get("AWS_BEARER_TOKEN_BEDROCK"):
 os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
 os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
 os.environ.setdefault("AWS_SESSION_TOKEN", "testing")

import boto3
from braintrust.auto import auto_instrument
from braintrust.integrations.test_utils import autoinstrument_test_context


MODEL = os.getenv("BRAINTRUST_BEDROCK_CONVERSE_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"

results = auto_instrument()
assert results.get("bedrock_runtime") is True

results2 = auto_instrument()
assert results2.get("bedrock_runtime") is True

with autoinstrument_test_context("test_auto_bedrock_runtime", integration="bedrock_runtime") as memory_logger:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 inferenceConfig={"maxTokens": 20, "temperature": 0},
 )
 assert response["output"]["message"]["role"] == "assistant"

 spans = memory_logger.pop()
 assert len(spans) == 1, f"Expected 1 span, got {len(spans)}"
 span = spans[0]
 assert span["metadata"]["provider"] == "bedrock"
 assert span["metadata"]["model"] == MODEL
 assert span["metadata"]["endpoint"] == "converse"
 assert span["span_attributes"]["name"] == "bedrock.converse"

print("SUCCESS")

17 changes: 17 additions & 0 deletions
py/src/braintrust/integrations/bedrock_runtime/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
"""Public entry points for the boto3 Bedrock Runtime integration."""

from .integration import BedrockRuntimeIntegration
from .patchers import wrap_bedrock_client


__all__ = ["BedrockRuntimeIntegration", "setup_bedrock", "wrap_bedrock"]


def setup_bedrock() -> bool:
 """Patch botocore client creation to auto-wrap Bedrock Runtime clients."""
 return BedrockRuntimeIntegration.setup()


def wrap_bedrock(client):
 """Instrument a boto3 Bedrock Runtime client instance in place."""
 return wrap_bedrock_client(client)

35 changes: 35 additions & 0 deletions
...braintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_auto_bedrock_runtime.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
interactions:
- request:
 body: '{"messages": [{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 "inferenceConfig": {"maxTokens": 20, "temperature": 0}}'
 headers:
 Content-Length:
 - '137'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9aLGIs
 RCwzIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":511},"output":{"message":{"content":[{"text":"Hi."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":6,"outputTokens":3,"serverToolUsage":{},"totalTokens":9}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '202'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:03 GMT
 x-amzn-RequestId:
 - caa9fd4f-69d2-459b-baf5-0ad888a79845
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...edrock_runtime/cassettes/1.34.116/test_setup_bedrock_converse_auto_wraps_new_clients.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":613},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:01 GMT
 x-amzn-RequestId:
 - 7c3bdcc8-1124-46b5-982e-3ced2d1a5371
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...raintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_wrap_bedrock_converse.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":486},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:00 GMT
 x-amzn-RequestId:
 - 1935727d-5309-4a70-8c15-1ce0ffbe877e
 status:
 code: 200
 message: OK
version: 1
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(bedrock): trace bedrock runtime calls by AbhiPrasad · Pull Request #519 · braintrustdata/braintrust-sdk-python · GitHub
Skip to content
feat(bedrock): trace bedrock runtime calls#519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Abhijeet Prasad (AbhiPrasad)
merged 1 commit into

from

Jun 16, 2026
 Merged


feat(bedrock): trace bedrock runtime calls
#519
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
examples/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,7 @@ Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `bedrock_runtime/` | boto3 Bedrock Runtime `converse()` call against Amazon Nova Lite |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
Expand Down

25 changes: 25 additions & 0 deletions
examples/bedrock_runtime/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
# AWS Bedrock + Braintrust

Calls `braintrust.auto_instrument()` to patch boto3's Bedrock Runtime client factory, then runs a single `client.converse(...)` call against Amazon Nova Lite. The trace captures a parent `answer_question` span plus a child `bedrock.converse` LLM span with input, output, metadata, and token metrics.

## Run

```bash
export BRAINTRUST_API_KEY=...
export AWS_BEARER_TOKEN_BEDROCK=...
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

uv sync
uv run python example.py
```

If you use IAM credentials instead of a Bedrock API key, set the usual AWS variables instead:

```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if using temporary credentials
```

The default model is `us.amazon.nova-lite-v1:0`. Override it with `BRAINTRUST_BEDROCK_MODEL` if needed.

38 changes: 38 additions & 0 deletions
examples/bedrock_runtime/example.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env python
"""boto3 Bedrock Runtime client traced via braintrust.auto_instrument()."""

import os

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-bedrock")

import boto3 # noqa: E402


MODEL = os.getenv("BRAINTRUST_BEDROCK_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"


@braintrust.traced
def answer_question(question: str) -> str:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": question}]}],
 inferenceConfig={"maxTokens": 64, "temperature": 0},
 )
 message = response["output"]["message"]
 return "".join(block.get("text", "") for block in message.get("content", []))


def main() -> None:
 answer = answer_question("What is the capital of Australia? Reply in one sentence.")
 print(answer)
 braintrust.flush()


if __name__ == "__main__":
 main()

12 changes: 12 additions & 0 deletions
examples/bedrock_runtime/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
[project]
name = "braintrust-bedrock-example"
version = "0.1.0"
description = "boto3 Bedrock Runtime client traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
 "braintrust",
 "boto3",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }

13 changes: 13 additions & 0 deletions
py/noxfile.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -230,6 +230,19 @@ def test_cohere(session, version):
 _run_tests(session, f"{INTEGRATION_DIR}/cohere/test_cohere.py", version=version)


BOTO3_VERSIONS = _get_matrix_versions("boto3")


@nox.session()
@nox.parametrize("version", BOTO3_VERSIONS, ids=BOTO3_VERSIONS)
def test_bedrock_runtime(session, version):
 """Test the boto3 Bedrock Runtime integration."""
 _install_test_deps(session)
 _install_matrix_dep(session, "boto3", version)
 _install_matrix_dep(session, "botocore", version)
 _run_tests(session, f"{INTEGRATION_DIR}/bedrock_runtime/test_bedrock_runtime.py", version=version)


INSTRUCTOR_VERSIONS = _get_matrix_versions("instructor")


Expand Down

11 changes: 11 additions & 0 deletions
py/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,14 @@ latest = "pytest==9.0.3"
[tool.braintrust.matrix.braintrust-core]
latest = "braintrust-core==0.0.59"

[tool.braintrust.matrix.boto3]
latest = "boto3==1.43.29"
"1.34.116" = "boto3==1.34.116"

[tool.braintrust.matrix.botocore]
latest = "botocore==1.43.29"
"1.34.116" = "botocore==1.34.116"

# ---------------------------------------------------------------------------
# Vendor packages — optional third-party packages the SDK can work without.
# Keys are matrix keys; values are Python import names. The noxfile uses this
Expand All@@ -461,6 +469,7 @@ agentscope = ["agentscope"]
agno = ["agno"]
autogen = ["autogen-agentchat"]
anthropic = ["anthropic"]
bedrock_runtime = ["boto3", "botocore"]
cohere = ["cohere"]
claude_agent_sdk = ["claude-agent-sdk"]
crewai = ["crewai"]
Expand DownExpand Up@@ -488,6 +497,8 @@ anthropic = "anthropic"
cohere = "cohere"
autoevals = "autoevals"
braintrust-core = "braintrust_core"
boto3 = "boto3"
botocore = "botocore"
crewai = "crewai"
dspy = "dspy"
google-adk = "google.adk"
Expand Down

5 changes: 5 additions & 0 deletions
py/src/braintrust/auto.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,7 @@
 AgnoIntegration,
 AnthropicIntegration,
 AutoGenIntegration,
 BedrockRuntimeIntegration,
 ClaudeAgentSDKIntegration,
 CohereIntegration,
 CrewAIIntegration,
Expand DownExpand Up@@ -72,6 +73,7 @@ def auto_instrument(
 openai_agents: bool = True,
 cohere: bool = True,
 autogen: bool = True,
 bedrock: bool = True,
 crewai: bool = True,
 strands: bool = True,
 temporal: bool = True,
Expand DownExpand Up@@ -106,6 +108,7 @@ def auto_instrument(
 openai_agents: Enable OpenAI Agents SDK instrumentation (default: True)
 cohere: Enable Cohere instrumentation (default: True)
 autogen: Enable AutoGen instrumentation (default: True)
 bedrock: Enable boto3 Bedrock Runtime instrumentation (default: True)
 crewai: Enable CrewAI instrumentation (default: True)
 strands: Enable Strands Agents instrumentation (default: True)
 temporal: Enable Temporal instrumentation (default: True)
Expand DownExpand Up@@ -195,6 +198,8 @@ def auto_instrument(
 results["cohere"] = _instrument_integration(CohereIntegration)
 if autogen:
 results["autogen"] = _instrument_integration(AutoGenIntegration)
 if bedrock:
 results["bedrock_runtime"] = _instrument_integration(BedrockRuntimeIntegration)
 if crewai:
 results["crewai"] = _instrument_integration(CrewAIIntegration)
 if strands:
Expand Down

6 changes: 6 additions & 0 deletions
py/src/braintrust/conftest.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -247,6 +247,12 @@ def get_vcr_config():
 "openai-api-key",
 "x-goog-api-key",
 "x-bt-auth-token",
 "x-amz-security-token",
 "x-amz-date",
 "x-amz-content-sha256",
 "amz-sdk-invocation-id",
 "amz-sdk-request",
 "x-amzn-bedrock-api-key",
 ],
 }

Expand Down

2 changes: 2 additions & 0 deletions
py/src/braintrust/integrations/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@
from .agno import AgnoIntegration
from .anthropic import AnthropicIntegration
from .autogen import AutoGenIntegration
from .bedrock_runtime import BedrockRuntimeIntegration
from .claude_agent_sdk import ClaudeAgentSDKIntegration
from .cohere import CohereIntegration
from .crewai import CrewAIIntegration
Expand All@@ -29,6 +30,7 @@
 "AgnoIntegration",
 "AnthropicIntegration",
 "AutoGenIntegration",
 "BedrockRuntimeIntegration",
 "ClaudeAgentSDKIntegration",
 "CohereIntegration",
 "CrewAIIntegration",
Expand Down

43 changes: 43 additions & 0 deletions
py/src/braintrust/integrations/auto_test_scripts/test_auto_bedrock_runtime.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
"""Test auto_instrument for boto3 Bedrock Runtime."""

import os


os.environ.setdefault("AWS_EC2_METADATA_DISABLED", "true")
if not os.environ.get("AWS_PROFILE") and not os.environ.get("AWS_BEARER_TOKEN_BEDROCK"):
 os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
 os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
 os.environ.setdefault("AWS_SESSION_TOKEN", "testing")

import boto3
from braintrust.auto import auto_instrument
from braintrust.integrations.test_utils import autoinstrument_test_context


MODEL = os.getenv("BRAINTRUST_BEDROCK_CONVERSE_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"

results = auto_instrument()
assert results.get("bedrock_runtime") is True

results2 = auto_instrument()
assert results2.get("bedrock_runtime") is True

with autoinstrument_test_context("test_auto_bedrock_runtime", integration="bedrock_runtime") as memory_logger:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 inferenceConfig={"maxTokens": 20, "temperature": 0},
 )
 assert response["output"]["message"]["role"] == "assistant"

 spans = memory_logger.pop()
 assert len(spans) == 1, f"Expected 1 span, got {len(spans)}"
 span = spans[0]
 assert span["metadata"]["provider"] == "bedrock"
 assert span["metadata"]["model"] == MODEL
 assert span["metadata"]["endpoint"] == "converse"
 assert span["span_attributes"]["name"] == "bedrock.converse"

print("SUCCESS")

17 changes: 17 additions & 0 deletions
py/src/braintrust/integrations/bedrock_runtime/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
"""Public entry points for the boto3 Bedrock Runtime integration."""

from .integration import BedrockRuntimeIntegration
from .patchers import wrap_bedrock_client


__all__ = ["BedrockRuntimeIntegration", "setup_bedrock", "wrap_bedrock"]


def setup_bedrock() -> bool:
 """Patch botocore client creation to auto-wrap Bedrock Runtime clients."""
 return BedrockRuntimeIntegration.setup()


def wrap_bedrock(client):
 """Instrument a boto3 Bedrock Runtime client instance in place."""
 return wrap_bedrock_client(client)

35 changes: 35 additions & 0 deletions
...braintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_auto_bedrock_runtime.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
interactions:
- request:
 body: '{"messages": [{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 "inferenceConfig": {"maxTokens": 20, "temperature": 0}}'
 headers:
 Content-Length:
 - '137'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9aLGIs
 RCwzIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":511},"output":{"message":{"content":[{"text":"Hi."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":6,"outputTokens":3,"serverToolUsage":{},"totalTokens":9}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '202'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:03 GMT
 x-amzn-RequestId:
 - caa9fd4f-69d2-459b-baf5-0ad888a79845
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...edrock_runtime/cassettes/1.34.116/test_setup_bedrock_converse_auto_wraps_new_clients.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":613},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:01 GMT
 x-amzn-RequestId:
 - 7c3bdcc8-1124-46b5-982e-3ced2d1a5371
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...raintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_wrap_bedrock_converse.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":486},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:00 GMT
 x-amzn-RequestId:
 - 1935727d-5309-4a70-8c15-1ce0ffbe877e
 status:
 code: 200
 message: OK
version: 1
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(bedrock): trace bedrock runtime calls by AbhiPrasad · Pull Request #519 · braintrustdata/braintrust-sdk-python · GitHub
Skip to content
feat(bedrock): trace bedrock runtime calls#519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Abhijeet Prasad (AbhiPrasad)
merged 1 commit into

from

Jun 16, 2026
 Merged


feat(bedrock): trace bedrock runtime calls
#519
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
examples/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,7 @@ Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `bedrock_runtime/` | boto3 Bedrock Runtime `converse()` call against Amazon Nova Lite |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
Expand Down

25 changes: 25 additions & 0 deletions
examples/bedrock_runtime/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
# AWS Bedrock + Braintrust

Calls `braintrust.auto_instrument()` to patch boto3's Bedrock Runtime client factory, then runs a single `client.converse(...)` call against Amazon Nova Lite. The trace captures a parent `answer_question` span plus a child `bedrock.converse` LLM span with input, output, metadata, and token metrics.

## Run

```bash
export BRAINTRUST_API_KEY=...
export AWS_BEARER_TOKEN_BEDROCK=...
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

uv sync
uv run python example.py
```

If you use IAM credentials instead of a Bedrock API key, set the usual AWS variables instead:

```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if using temporary credentials
```

The default model is `us.amazon.nova-lite-v1:0`. Override it with `BRAINTRUST_BEDROCK_MODEL` if needed.

38 changes: 38 additions & 0 deletions
examples/bedrock_runtime/example.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env python
"""boto3 Bedrock Runtime client traced via braintrust.auto_instrument()."""

import os

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-bedrock")

import boto3 # noqa: E402


MODEL = os.getenv("BRAINTRUST_BEDROCK_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"


@braintrust.traced
def answer_question(question: str) -> str:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": question}]}],
 inferenceConfig={"maxTokens": 64, "temperature": 0},
 )
 message = response["output"]["message"]
 return "".join(block.get("text", "") for block in message.get("content", []))


def main() -> None:
 answer = answer_question("What is the capital of Australia? Reply in one sentence.")
 print(answer)
 braintrust.flush()


if __name__ == "__main__":
 main()

12 changes: 12 additions & 0 deletions
examples/bedrock_runtime/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
[project]
name = "braintrust-bedrock-example"
version = "0.1.0"
description = "boto3 Bedrock Runtime client traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
 "braintrust",
 "boto3",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }

13 changes: 13 additions & 0 deletions
py/noxfile.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -230,6 +230,19 @@ def test_cohere(session, version):
 _run_tests(session, f"{INTEGRATION_DIR}/cohere/test_cohere.py", version=version)


BOTO3_VERSIONS = _get_matrix_versions("boto3")


@nox.session()
@nox.parametrize("version", BOTO3_VERSIONS, ids=BOTO3_VERSIONS)
def test_bedrock_runtime(session, version):
 """Test the boto3 Bedrock Runtime integration."""
 _install_test_deps(session)
 _install_matrix_dep(session, "boto3", version)
 _install_matrix_dep(session, "botocore", version)
 _run_tests(session, f"{INTEGRATION_DIR}/bedrock_runtime/test_bedrock_runtime.py", version=version)


INSTRUCTOR_VERSIONS = _get_matrix_versions("instructor")


Expand Down

11 changes: 11 additions & 0 deletions
py/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,14 @@ latest = "pytest==9.0.3"
[tool.braintrust.matrix.braintrust-core]
latest = "braintrust-core==0.0.59"

[tool.braintrust.matrix.boto3]
latest = "boto3==1.43.29"
"1.34.116" = "boto3==1.34.116"

[tool.braintrust.matrix.botocore]
latest = "botocore==1.43.29"
"1.34.116" = "botocore==1.34.116"

# ---------------------------------------------------------------------------
# Vendor packages — optional third-party packages the SDK can work without.
# Keys are matrix keys; values are Python import names. The noxfile uses this
Expand All@@ -461,6 +469,7 @@ agentscope = ["agentscope"]
agno = ["agno"]
autogen = ["autogen-agentchat"]
anthropic = ["anthropic"]
bedrock_runtime = ["boto3", "botocore"]
cohere = ["cohere"]
claude_agent_sdk = ["claude-agent-sdk"]
crewai = ["crewai"]
Expand DownExpand Up@@ -488,6 +497,8 @@ anthropic = "anthropic"
cohere = "cohere"
autoevals = "autoevals"
braintrust-core = "braintrust_core"
boto3 = "boto3"
botocore = "botocore"
crewai = "crewai"
dspy = "dspy"
google-adk = "google.adk"
Expand Down

5 changes: 5 additions & 0 deletions
py/src/braintrust/auto.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,7 @@
 AgnoIntegration,
 AnthropicIntegration,
 AutoGenIntegration,
 BedrockRuntimeIntegration,
 ClaudeAgentSDKIntegration,
 CohereIntegration,
 CrewAIIntegration,
Expand DownExpand Up@@ -72,6 +73,7 @@ def auto_instrument(
 openai_agents: bool = True,
 cohere: bool = True,
 autogen: bool = True,
 bedrock: bool = True,
 crewai: bool = True,
 strands: bool = True,
 temporal: bool = True,
Expand DownExpand Up@@ -106,6 +108,7 @@ def auto_instrument(
 openai_agents: Enable OpenAI Agents SDK instrumentation (default: True)
 cohere: Enable Cohere instrumentation (default: True)
 autogen: Enable AutoGen instrumentation (default: True)
 bedrock: Enable boto3 Bedrock Runtime instrumentation (default: True)
 crewai: Enable CrewAI instrumentation (default: True)
 strands: Enable Strands Agents instrumentation (default: True)
 temporal: Enable Temporal instrumentation (default: True)
Expand DownExpand Up@@ -195,6 +198,8 @@ def auto_instrument(
 results["cohere"] = _instrument_integration(CohereIntegration)
 if autogen:
 results["autogen"] = _instrument_integration(AutoGenIntegration)
 if bedrock:
 results["bedrock_runtime"] = _instrument_integration(BedrockRuntimeIntegration)
 if crewai:
 results["crewai"] = _instrument_integration(CrewAIIntegration)
 if strands:
Expand Down

6 changes: 6 additions & 0 deletions
py/src/braintrust/conftest.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -247,6 +247,12 @@ def get_vcr_config():
 "openai-api-key",
 "x-goog-api-key",
 "x-bt-auth-token",
 "x-amz-security-token",
 "x-amz-date",
 "x-amz-content-sha256",
 "amz-sdk-invocation-id",
 "amz-sdk-request",
 "x-amzn-bedrock-api-key",
 ],
 }

Expand Down

2 changes: 2 additions & 0 deletions
py/src/braintrust/integrations/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@
from .agno import AgnoIntegration
from .anthropic import AnthropicIntegration
from .autogen import AutoGenIntegration
from .bedrock_runtime import BedrockRuntimeIntegration
from .claude_agent_sdk import ClaudeAgentSDKIntegration
from .cohere import CohereIntegration
from .crewai import CrewAIIntegration
Expand All@@ -29,6 +30,7 @@
 "AgnoIntegration",
 "AnthropicIntegration",
 "AutoGenIntegration",
 "BedrockRuntimeIntegration",
 "ClaudeAgentSDKIntegration",
 "CohereIntegration",
 "CrewAIIntegration",
Expand Down

43 changes: 43 additions & 0 deletions
py/src/braintrust/integrations/auto_test_scripts/test_auto_bedrock_runtime.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
"""Test auto_instrument for boto3 Bedrock Runtime."""

import os


os.environ.setdefault("AWS_EC2_METADATA_DISABLED", "true")
if not os.environ.get("AWS_PROFILE") and not os.environ.get("AWS_BEARER_TOKEN_BEDROCK"):
 os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
 os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
 os.environ.setdefault("AWS_SESSION_TOKEN", "testing")

import boto3
from braintrust.auto import auto_instrument
from braintrust.integrations.test_utils import autoinstrument_test_context


MODEL = os.getenv("BRAINTRUST_BEDROCK_CONVERSE_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"

results = auto_instrument()
assert results.get("bedrock_runtime") is True

results2 = auto_instrument()
assert results2.get("bedrock_runtime") is True

with autoinstrument_test_context("test_auto_bedrock_runtime", integration="bedrock_runtime") as memory_logger:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 inferenceConfig={"maxTokens": 20, "temperature": 0},
 )
 assert response["output"]["message"]["role"] == "assistant"

 spans = memory_logger.pop()
 assert len(spans) == 1, f"Expected 1 span, got {len(spans)}"
 span = spans[0]
 assert span["metadata"]["provider"] == "bedrock"
 assert span["metadata"]["model"] == MODEL
 assert span["metadata"]["endpoint"] == "converse"
 assert span["span_attributes"]["name"] == "bedrock.converse"

print("SUCCESS")

17 changes: 17 additions & 0 deletions
py/src/braintrust/integrations/bedrock_runtime/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
"""Public entry points for the boto3 Bedrock Runtime integration."""

from .integration import BedrockRuntimeIntegration
from .patchers import wrap_bedrock_client


__all__ = ["BedrockRuntimeIntegration", "setup_bedrock", "wrap_bedrock"]


def setup_bedrock() -> bool:
 """Patch botocore client creation to auto-wrap Bedrock Runtime clients."""
 return BedrockRuntimeIntegration.setup()


def wrap_bedrock(client):
 """Instrument a boto3 Bedrock Runtime client instance in place."""
 return wrap_bedrock_client(client)

35 changes: 35 additions & 0 deletions
...braintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_auto_bedrock_runtime.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
interactions:
- request:
 body: '{"messages": [{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 "inferenceConfig": {"maxTokens": 20, "temperature": 0}}'
 headers:
 Content-Length:
 - '137'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9aLGIs
 RCwzIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":511},"output":{"message":{"content":[{"text":"Hi."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":6,"outputTokens":3,"serverToolUsage":{},"totalTokens":9}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '202'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:03 GMT
 x-amzn-RequestId:
 - caa9fd4f-69d2-459b-baf5-0ad888a79845
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...edrock_runtime/cassettes/1.34.116/test_setup_bedrock_converse_auto_wraps_new_clients.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":613},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:01 GMT
 x-amzn-RequestId:
 - 7c3bdcc8-1124-46b5-982e-3ced2d1a5371
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...raintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_wrap_bedrock_converse.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":486},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:00 GMT
 x-amzn-RequestId:
 - 1935727d-5309-4a70-8c15-1ce0ffbe877e
 status:
 code: 200
 message: OK
version: 1
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
feat(bedrock): trace bedrock runtime calls by AbhiPrasad · Pull Request #519 · braintrustdata/braintrust-sdk-python · GitHub
Skip to content
feat(bedrock): trace bedrock runtime calls#519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Abhijeet Prasad (AbhiPrasad)
merged 1 commit into

from

Jun 16, 2026
 Merged


feat(bedrock): trace bedrock runtime calls
#519
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
examples/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,7 @@ Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `bedrock_runtime/` | boto3 Bedrock Runtime `converse()` call against Amazon Nova Lite |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
Expand Down

25 changes: 25 additions & 0 deletions
examples/bedrock_runtime/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
# AWS Bedrock + Braintrust

Calls `braintrust.auto_instrument()` to patch boto3's Bedrock Runtime client factory, then runs a single `client.converse(...)` call against Amazon Nova Lite. The trace captures a parent `answer_question` span plus a child `bedrock.converse` LLM span with input, output, metadata, and token metrics.

## Run

```bash
export BRAINTRUST_API_KEY=...
export AWS_BEARER_TOKEN_BEDROCK=...
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

uv sync
uv run python example.py
```

If you use IAM credentials instead of a Bedrock API key, set the usual AWS variables instead:

```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if using temporary credentials
```

The default model is `us.amazon.nova-lite-v1:0`. Override it with `BRAINTRUST_BEDROCK_MODEL` if needed.

38 changes: 38 additions & 0 deletions
examples/bedrock_runtime/example.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env python
"""boto3 Bedrock Runtime client traced via braintrust.auto_instrument()."""

import os

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-bedrock")

import boto3 # noqa: E402


MODEL = os.getenv("BRAINTRUST_BEDROCK_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"


@braintrust.traced
def answer_question(question: str) -> str:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": question}]}],
 inferenceConfig={"maxTokens": 64, "temperature": 0},
 )
 message = response["output"]["message"]
 return "".join(block.get("text", "") for block in message.get("content", []))


def main() -> None:
 answer = answer_question("What is the capital of Australia? Reply in one sentence.")
 print(answer)
 braintrust.flush()


if __name__ == "__main__":
 main()

12 changes: 12 additions & 0 deletions
examples/bedrock_runtime/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
[project]
name = "braintrust-bedrock-example"
version = "0.1.0"
description = "boto3 Bedrock Runtime client traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
 "braintrust",
 "boto3",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }

13 changes: 13 additions & 0 deletions
py/noxfile.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -230,6 +230,19 @@ def test_cohere(session, version):
 _run_tests(session, f"{INTEGRATION_DIR}/cohere/test_cohere.py", version=version)


BOTO3_VERSIONS = _get_matrix_versions("boto3")


@nox.session()
@nox.parametrize("version", BOTO3_VERSIONS, ids=BOTO3_VERSIONS)
def test_bedrock_runtime(session, version):
 """Test the boto3 Bedrock Runtime integration."""
 _install_test_deps(session)
 _install_matrix_dep(session, "boto3", version)
 _install_matrix_dep(session, "botocore", version)
 _run_tests(session, f"{INTEGRATION_DIR}/bedrock_runtime/test_bedrock_runtime.py", version=version)


INSTRUCTOR_VERSIONS = _get_matrix_versions("instructor")


Expand Down

11 changes: 11 additions & 0 deletions
py/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,14 @@ latest = "pytest==9.0.3"
[tool.braintrust.matrix.braintrust-core]
latest = "braintrust-core==0.0.59"

[tool.braintrust.matrix.boto3]
latest = "boto3==1.43.29"
"1.34.116" = "boto3==1.34.116"

[tool.braintrust.matrix.botocore]
latest = "botocore==1.43.29"
"1.34.116" = "botocore==1.34.116"

# ---------------------------------------------------------------------------
# Vendor packages — optional third-party packages the SDK can work without.
# Keys are matrix keys; values are Python import names. The noxfile uses this
Expand All@@ -461,6 +469,7 @@ agentscope = ["agentscope"]
agno = ["agno"]
autogen = ["autogen-agentchat"]
anthropic = ["anthropic"]
bedrock_runtime = ["boto3", "botocore"]
cohere = ["cohere"]
claude_agent_sdk = ["claude-agent-sdk"]
crewai = ["crewai"]
Expand DownExpand Up@@ -488,6 +497,8 @@ anthropic = "anthropic"
cohere = "cohere"
autoevals = "autoevals"
braintrust-core = "braintrust_core"
boto3 = "boto3"
botocore = "botocore"
crewai = "crewai"
dspy = "dspy"
google-adk = "google.adk"
Expand Down

5 changes: 5 additions & 0 deletions
py/src/braintrust/auto.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,7 @@
 AgnoIntegration,
 AnthropicIntegration,
 AutoGenIntegration,
 BedrockRuntimeIntegration,
 ClaudeAgentSDKIntegration,
 CohereIntegration,
 CrewAIIntegration,
Expand DownExpand Up@@ -72,6 +73,7 @@ def auto_instrument(
 openai_agents: bool = True,
 cohere: bool = True,
 autogen: bool = True,
 bedrock: bool = True,
 crewai: bool = True,
 strands: bool = True,
 temporal: bool = True,
Expand DownExpand Up@@ -106,6 +108,7 @@ def auto_instrument(
 openai_agents: Enable OpenAI Agents SDK instrumentation (default: True)
 cohere: Enable Cohere instrumentation (default: True)
 autogen: Enable AutoGen instrumentation (default: True)
 bedrock: Enable boto3 Bedrock Runtime instrumentation (default: True)
 crewai: Enable CrewAI instrumentation (default: True)
 strands: Enable Strands Agents instrumentation (default: True)
 temporal: Enable Temporal instrumentation (default: True)
Expand DownExpand Up@@ -195,6 +198,8 @@ def auto_instrument(
 results["cohere"] = _instrument_integration(CohereIntegration)
 if autogen:
 results["autogen"] = _instrument_integration(AutoGenIntegration)
 if bedrock:
 results["bedrock_runtime"] = _instrument_integration(BedrockRuntimeIntegration)
 if crewai:
 results["crewai"] = _instrument_integration(CrewAIIntegration)
 if strands:
Expand Down

6 changes: 6 additions & 0 deletions
py/src/braintrust/conftest.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -247,6 +247,12 @@ def get_vcr_config():
 "openai-api-key",
 "x-goog-api-key",
 "x-bt-auth-token",
 "x-amz-security-token",
 "x-amz-date",
 "x-amz-content-sha256",
 "amz-sdk-invocation-id",
 "amz-sdk-request",
 "x-amzn-bedrock-api-key",
 ],
 }

Expand Down

2 changes: 2 additions & 0 deletions
py/src/braintrust/integrations/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@
from .agno import AgnoIntegration
from .anthropic import AnthropicIntegration
from .autogen import AutoGenIntegration
from .bedrock_runtime import BedrockRuntimeIntegration
from .claude_agent_sdk import ClaudeAgentSDKIntegration
from .cohere import CohereIntegration
from .crewai import CrewAIIntegration
Expand All@@ -29,6 +30,7 @@
 "AgnoIntegration",
 "AnthropicIntegration",
 "AutoGenIntegration",
 "BedrockRuntimeIntegration",
 "ClaudeAgentSDKIntegration",
 "CohereIntegration",
 "CrewAIIntegration",
Expand Down

43 changes: 43 additions & 0 deletions
py/src/braintrust/integrations/auto_test_scripts/test_auto_bedrock_runtime.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
"""Test auto_instrument for boto3 Bedrock Runtime."""

import os


os.environ.setdefault("AWS_EC2_METADATA_DISABLED", "true")
if not os.environ.get("AWS_PROFILE") and not os.environ.get("AWS_BEARER_TOKEN_BEDROCK"):
 os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
 os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
 os.environ.setdefault("AWS_SESSION_TOKEN", "testing")

import boto3
from braintrust.auto import auto_instrument
from braintrust.integrations.test_utils import autoinstrument_test_context


MODEL = os.getenv("BRAINTRUST_BEDROCK_CONVERSE_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"

results = auto_instrument()
assert results.get("bedrock_runtime") is True

results2 = auto_instrument()
assert results2.get("bedrock_runtime") is True

with autoinstrument_test_context("test_auto_bedrock_runtime", integration="bedrock_runtime") as memory_logger:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 inferenceConfig={"maxTokens": 20, "temperature": 0},
 )
 assert response["output"]["message"]["role"] == "assistant"

 spans = memory_logger.pop()
 assert len(spans) == 1, f"Expected 1 span, got {len(spans)}"
 span = spans[0]
 assert span["metadata"]["provider"] == "bedrock"
 assert span["metadata"]["model"] == MODEL
 assert span["metadata"]["endpoint"] == "converse"
 assert span["span_attributes"]["name"] == "bedrock.converse"

print("SUCCESS")

17 changes: 17 additions & 0 deletions
py/src/braintrust/integrations/bedrock_runtime/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
"""Public entry points for the boto3 Bedrock Runtime integration."""

from .integration import BedrockRuntimeIntegration
from .patchers import wrap_bedrock_client


__all__ = ["BedrockRuntimeIntegration", "setup_bedrock", "wrap_bedrock"]


def setup_bedrock() -> bool:
 """Patch botocore client creation to auto-wrap Bedrock Runtime clients."""
 return BedrockRuntimeIntegration.setup()


def wrap_bedrock(client):
 """Instrument a boto3 Bedrock Runtime client instance in place."""
 return wrap_bedrock_client(client)

35 changes: 35 additions & 0 deletions
...braintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_auto_bedrock_runtime.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
interactions:
- request:
 body: '{"messages": [{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 "inferenceConfig": {"maxTokens": 20, "temperature": 0}}'
 headers:
 Content-Length:
 - '137'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9aLGIs
 RCwzIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":511},"output":{"message":{"content":[{"text":"Hi."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":6,"outputTokens":3,"serverToolUsage":{},"totalTokens":9}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '202'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:03 GMT
 x-amzn-RequestId:
 - caa9fd4f-69d2-459b-baf5-0ad888a79845
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...edrock_runtime/cassettes/1.34.116/test_setup_bedrock_converse_auto_wraps_new_clients.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":613},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:01 GMT
 x-amzn-RequestId:
 - 7c3bdcc8-1124-46b5-982e-3ced2d1a5371
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...raintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_wrap_bedrock_converse.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":486},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:00 GMT
 x-amzn-RequestId:
 - 1935727d-5309-4a70-8c15-1ce0ffbe877e
 status:
 code: 200
 message: OK
version: 1
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(bedrock): trace bedrock runtime calls by AbhiPrasad · Pull Request #519 · braintrustdata/braintrust-sdk-python · GitHub
Skip to content
feat(bedrock): trace bedrock runtime calls#519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Abhijeet Prasad (AbhiPrasad)
merged 1 commit into

from

Jun 16, 2026
 Merged


feat(bedrock): trace bedrock runtime calls
#519
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
examples/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,7 @@ Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `bedrock_runtime/` | boto3 Bedrock Runtime `converse()` call against Amazon Nova Lite |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
Expand Down

25 changes: 25 additions & 0 deletions
examples/bedrock_runtime/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
# AWS Bedrock + Braintrust

Calls `braintrust.auto_instrument()` to patch boto3's Bedrock Runtime client factory, then runs a single `client.converse(...)` call against Amazon Nova Lite. The trace captures a parent `answer_question` span plus a child `bedrock.converse` LLM span with input, output, metadata, and token metrics.

## Run

```bash
export BRAINTRUST_API_KEY=...
export AWS_BEARER_TOKEN_BEDROCK=...
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

uv sync
uv run python example.py
```

If you use IAM credentials instead of a Bedrock API key, set the usual AWS variables instead:

```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if using temporary credentials
```

The default model is `us.amazon.nova-lite-v1:0`. Override it with `BRAINTRUST_BEDROCK_MODEL` if needed.

38 changes: 38 additions & 0 deletions
examples/bedrock_runtime/example.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env python
"""boto3 Bedrock Runtime client traced via braintrust.auto_instrument()."""

import os

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-bedrock")

import boto3 # noqa: E402


MODEL = os.getenv("BRAINTRUST_BEDROCK_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"


@braintrust.traced
def answer_question(question: str) -> str:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": question}]}],
 inferenceConfig={"maxTokens": 64, "temperature": 0},
 )
 message = response["output"]["message"]
 return "".join(block.get("text", "") for block in message.get("content", []))


def main() -> None:
 answer = answer_question("What is the capital of Australia? Reply in one sentence.")
 print(answer)
 braintrust.flush()


if __name__ == "__main__":
 main()

12 changes: 12 additions & 0 deletions
examples/bedrock_runtime/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
[project]
name = "braintrust-bedrock-example"
version = "0.1.0"
description = "boto3 Bedrock Runtime client traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
 "braintrust",
 "boto3",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }

13 changes: 13 additions & 0 deletions
py/noxfile.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -230,6 +230,19 @@ def test_cohere(session, version):
 _run_tests(session, f"{INTEGRATION_DIR}/cohere/test_cohere.py", version=version)


BOTO3_VERSIONS = _get_matrix_versions("boto3")


@nox.session()
@nox.parametrize("version", BOTO3_VERSIONS, ids=BOTO3_VERSIONS)
def test_bedrock_runtime(session, version):
 """Test the boto3 Bedrock Runtime integration."""
 _install_test_deps(session)
 _install_matrix_dep(session, "boto3", version)
 _install_matrix_dep(session, "botocore", version)
 _run_tests(session, f"{INTEGRATION_DIR}/bedrock_runtime/test_bedrock_runtime.py", version=version)


INSTRUCTOR_VERSIONS = _get_matrix_versions("instructor")


Expand Down

11 changes: 11 additions & 0 deletions
py/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,14 @@ latest = "pytest==9.0.3"
[tool.braintrust.matrix.braintrust-core]
latest = "braintrust-core==0.0.59"

[tool.braintrust.matrix.boto3]
latest = "boto3==1.43.29"
"1.34.116" = "boto3==1.34.116"

[tool.braintrust.matrix.botocore]
latest = "botocore==1.43.29"
"1.34.116" = "botocore==1.34.116"

# ---------------------------------------------------------------------------
# Vendor packages — optional third-party packages the SDK can work without.
# Keys are matrix keys; values are Python import names. The noxfile uses this
Expand All@@ -461,6 +469,7 @@ agentscope = ["agentscope"]
agno = ["agno"]
autogen = ["autogen-agentchat"]
anthropic = ["anthropic"]
bedrock_runtime = ["boto3", "botocore"]
cohere = ["cohere"]
claude_agent_sdk = ["claude-agent-sdk"]
crewai = ["crewai"]
Expand DownExpand Up@@ -488,6 +497,8 @@ anthropic = "anthropic"
cohere = "cohere"
autoevals = "autoevals"
braintrust-core = "braintrust_core"
boto3 = "boto3"
botocore = "botocore"
crewai = "crewai"
dspy = "dspy"
google-adk = "google.adk"
Expand Down

5 changes: 5 additions & 0 deletions
py/src/braintrust/auto.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,7 @@
 AgnoIntegration,
 AnthropicIntegration,
 AutoGenIntegration,
 BedrockRuntimeIntegration,
 ClaudeAgentSDKIntegration,
 CohereIntegration,
 CrewAIIntegration,
Expand DownExpand Up@@ -72,6 +73,7 @@ def auto_instrument(
 openai_agents: bool = True,
 cohere: bool = True,
 autogen: bool = True,
 bedrock: bool = True,
 crewai: bool = True,
 strands: bool = True,
 temporal: bool = True,
Expand DownExpand Up@@ -106,6 +108,7 @@ def auto_instrument(
 openai_agents: Enable OpenAI Agents SDK instrumentation (default: True)
 cohere: Enable Cohere instrumentation (default: True)
 autogen: Enable AutoGen instrumentation (default: True)
 bedrock: Enable boto3 Bedrock Runtime instrumentation (default: True)
 crewai: Enable CrewAI instrumentation (default: True)
 strands: Enable Strands Agents instrumentation (default: True)
 temporal: Enable Temporal instrumentation (default: True)
Expand DownExpand Up@@ -195,6 +198,8 @@ def auto_instrument(
 results["cohere"] = _instrument_integration(CohereIntegration)
 if autogen:
 results["autogen"] = _instrument_integration(AutoGenIntegration)
 if bedrock:
 results["bedrock_runtime"] = _instrument_integration(BedrockRuntimeIntegration)
 if crewai:
 results["crewai"] = _instrument_integration(CrewAIIntegration)
 if strands:
Expand Down

6 changes: 6 additions & 0 deletions
py/src/braintrust/conftest.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -247,6 +247,12 @@ def get_vcr_config():
 "openai-api-key",
 "x-goog-api-key",
 "x-bt-auth-token",
 "x-amz-security-token",
 "x-amz-date",
 "x-amz-content-sha256",
 "amz-sdk-invocation-id",
 "amz-sdk-request",
 "x-amzn-bedrock-api-key",
 ],
 }

Expand Down

2 changes: 2 additions & 0 deletions
py/src/braintrust/integrations/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@
from .agno import AgnoIntegration
from .anthropic import AnthropicIntegration
from .autogen import AutoGenIntegration
from .bedrock_runtime import BedrockRuntimeIntegration
from .claude_agent_sdk import ClaudeAgentSDKIntegration
from .cohere import CohereIntegration
from .crewai import CrewAIIntegration
Expand All@@ -29,6 +30,7 @@
 "AgnoIntegration",
 "AnthropicIntegration",
 "AutoGenIntegration",
 "BedrockRuntimeIntegration",
 "ClaudeAgentSDKIntegration",
 "CohereIntegration",
 "CrewAIIntegration",
Expand Down

43 changes: 43 additions & 0 deletions
py/src/braintrust/integrations/auto_test_scripts/test_auto_bedrock_runtime.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
"""Test auto_instrument for boto3 Bedrock Runtime."""

import os


os.environ.setdefault("AWS_EC2_METADATA_DISABLED", "true")
if not os.environ.get("AWS_PROFILE") and not os.environ.get("AWS_BEARER_TOKEN_BEDROCK"):
 os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
 os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
 os.environ.setdefault("AWS_SESSION_TOKEN", "testing")

import boto3
from braintrust.auto import auto_instrument
from braintrust.integrations.test_utils import autoinstrument_test_context


MODEL = os.getenv("BRAINTRUST_BEDROCK_CONVERSE_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"

results = auto_instrument()
assert results.get("bedrock_runtime") is True

results2 = auto_instrument()
assert results2.get("bedrock_runtime") is True

with autoinstrument_test_context("test_auto_bedrock_runtime", integration="bedrock_runtime") as memory_logger:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 inferenceConfig={"maxTokens": 20, "temperature": 0},
 )
 assert response["output"]["message"]["role"] == "assistant"

 spans = memory_logger.pop()
 assert len(spans) == 1, f"Expected 1 span, got {len(spans)}"
 span = spans[0]
 assert span["metadata"]["provider"] == "bedrock"
 assert span["metadata"]["model"] == MODEL
 assert span["metadata"]["endpoint"] == "converse"
 assert span["span_attributes"]["name"] == "bedrock.converse"

print("SUCCESS")

17 changes: 17 additions & 0 deletions
py/src/braintrust/integrations/bedrock_runtime/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
"""Public entry points for the boto3 Bedrock Runtime integration."""

from .integration import BedrockRuntimeIntegration
from .patchers import wrap_bedrock_client


__all__ = ["BedrockRuntimeIntegration", "setup_bedrock", "wrap_bedrock"]


def setup_bedrock() -> bool:
 """Patch botocore client creation to auto-wrap Bedrock Runtime clients."""
 return BedrockRuntimeIntegration.setup()


def wrap_bedrock(client):
 """Instrument a boto3 Bedrock Runtime client instance in place."""
 return wrap_bedrock_client(client)

35 changes: 35 additions & 0 deletions
...braintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_auto_bedrock_runtime.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
interactions:
- request:
 body: '{"messages": [{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 "inferenceConfig": {"maxTokens": 20, "temperature": 0}}'
 headers:
 Content-Length:
 - '137'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9aLGIs
 RCwzIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":511},"output":{"message":{"content":[{"text":"Hi."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":6,"outputTokens":3,"serverToolUsage":{},"totalTokens":9}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '202'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:03 GMT
 x-amzn-RequestId:
 - caa9fd4f-69d2-459b-baf5-0ad888a79845
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...edrock_runtime/cassettes/1.34.116/test_setup_bedrock_converse_auto_wraps_new_clients.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":613},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:01 GMT
 x-amzn-RequestId:
 - 7c3bdcc8-1124-46b5-982e-3ced2d1a5371
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...raintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_wrap_bedrock_converse.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":486},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:00 GMT
 x-amzn-RequestId:
 - 1935727d-5309-4a70-8c15-1ce0ffbe877e
 status:
 code: 200
 message: OK
version: 1
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(bedrock): trace bedrock runtime calls by AbhiPrasad · Pull Request #519 · braintrustdata/braintrust-sdk-python · GitHub
Skip to content
feat(bedrock): trace bedrock runtime calls#519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Abhijeet Prasad (AbhiPrasad)
merged 1 commit into

from

Jun 16, 2026
 Merged


feat(bedrock): trace bedrock runtime calls
#519
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
examples/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,7 @@ Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `bedrock_runtime/` | boto3 Bedrock Runtime `converse()` call against Amazon Nova Lite |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
Expand Down

25 changes: 25 additions & 0 deletions
examples/bedrock_runtime/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
# AWS Bedrock + Braintrust

Calls `braintrust.auto_instrument()` to patch boto3's Bedrock Runtime client factory, then runs a single `client.converse(...)` call against Amazon Nova Lite. The trace captures a parent `answer_question` span plus a child `bedrock.converse` LLM span with input, output, metadata, and token metrics.

## Run

```bash
export BRAINTRUST_API_KEY=...
export AWS_BEARER_TOKEN_BEDROCK=...
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

uv sync
uv run python example.py
```

If you use IAM credentials instead of a Bedrock API key, set the usual AWS variables instead:

```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if using temporary credentials
```

The default model is `us.amazon.nova-lite-v1:0`. Override it with `BRAINTRUST_BEDROCK_MODEL` if needed.

38 changes: 38 additions & 0 deletions
examples/bedrock_runtime/example.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env python
"""boto3 Bedrock Runtime client traced via braintrust.auto_instrument()."""

import os

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-bedrock")

import boto3 # noqa: E402


MODEL = os.getenv("BRAINTRUST_BEDROCK_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"


@braintrust.traced
def answer_question(question: str) -> str:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": question}]}],
 inferenceConfig={"maxTokens": 64, "temperature": 0},
 )
 message = response["output"]["message"]
 return "".join(block.get("text", "") for block in message.get("content", []))


def main() -> None:
 answer = answer_question("What is the capital of Australia? Reply in one sentence.")
 print(answer)
 braintrust.flush()


if __name__ == "__main__":
 main()

12 changes: 12 additions & 0 deletions
examples/bedrock_runtime/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
[project]
name = "braintrust-bedrock-example"
version = "0.1.0"
description = "boto3 Bedrock Runtime client traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
 "braintrust",
 "boto3",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }

13 changes: 13 additions & 0 deletions
py/noxfile.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -230,6 +230,19 @@ def test_cohere(session, version):
 _run_tests(session, f"{INTEGRATION_DIR}/cohere/test_cohere.py", version=version)


BOTO3_VERSIONS = _get_matrix_versions("boto3")


@nox.session()
@nox.parametrize("version", BOTO3_VERSIONS, ids=BOTO3_VERSIONS)
def test_bedrock_runtime(session, version):
 """Test the boto3 Bedrock Runtime integration."""
 _install_test_deps(session)
 _install_matrix_dep(session, "boto3", version)
 _install_matrix_dep(session, "botocore", version)
 _run_tests(session, f"{INTEGRATION_DIR}/bedrock_runtime/test_bedrock_runtime.py", version=version)


INSTRUCTOR_VERSIONS = _get_matrix_versions("instructor")


Expand Down

11 changes: 11 additions & 0 deletions
py/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,14 @@ latest = "pytest==9.0.3"
[tool.braintrust.matrix.braintrust-core]
latest = "braintrust-core==0.0.59"

[tool.braintrust.matrix.boto3]
latest = "boto3==1.43.29"
"1.34.116" = "boto3==1.34.116"

[tool.braintrust.matrix.botocore]
latest = "botocore==1.43.29"
"1.34.116" = "botocore==1.34.116"

# ---------------------------------------------------------------------------
# Vendor packages — optional third-party packages the SDK can work without.
# Keys are matrix keys; values are Python import names. The noxfile uses this
Expand All@@ -461,6 +469,7 @@ agentscope = ["agentscope"]
agno = ["agno"]
autogen = ["autogen-agentchat"]
anthropic = ["anthropic"]
bedrock_runtime = ["boto3", "botocore"]
cohere = ["cohere"]
claude_agent_sdk = ["claude-agent-sdk"]
crewai = ["crewai"]
Expand DownExpand Up@@ -488,6 +497,8 @@ anthropic = "anthropic"
cohere = "cohere"
autoevals = "autoevals"
braintrust-core = "braintrust_core"
boto3 = "boto3"
botocore = "botocore"
crewai = "crewai"
dspy = "dspy"
google-adk = "google.adk"
Expand Down

5 changes: 5 additions & 0 deletions
py/src/braintrust/auto.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,7 @@
 AgnoIntegration,
 AnthropicIntegration,
 AutoGenIntegration,
 BedrockRuntimeIntegration,
 ClaudeAgentSDKIntegration,
 CohereIntegration,
 CrewAIIntegration,
Expand DownExpand Up@@ -72,6 +73,7 @@ def auto_instrument(
 openai_agents: bool = True,
 cohere: bool = True,
 autogen: bool = True,
 bedrock: bool = True,
 crewai: bool = True,
 strands: bool = True,
 temporal: bool = True,
Expand DownExpand Up@@ -106,6 +108,7 @@ def auto_instrument(
 openai_agents: Enable OpenAI Agents SDK instrumentation (default: True)
 cohere: Enable Cohere instrumentation (default: True)
 autogen: Enable AutoGen instrumentation (default: True)
 bedrock: Enable boto3 Bedrock Runtime instrumentation (default: True)
 crewai: Enable CrewAI instrumentation (default: True)
 strands: Enable Strands Agents instrumentation (default: True)
 temporal: Enable Temporal instrumentation (default: True)
Expand DownExpand Up@@ -195,6 +198,8 @@ def auto_instrument(
 results["cohere"] = _instrument_integration(CohereIntegration)
 if autogen:
 results["autogen"] = _instrument_integration(AutoGenIntegration)
 if bedrock:
 results["bedrock_runtime"] = _instrument_integration(BedrockRuntimeIntegration)
 if crewai:
 results["crewai"] = _instrument_integration(CrewAIIntegration)
 if strands:
Expand Down

6 changes: 6 additions & 0 deletions
py/src/braintrust/conftest.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -247,6 +247,12 @@ def get_vcr_config():
 "openai-api-key",
 "x-goog-api-key",
 "x-bt-auth-token",
 "x-amz-security-token",
 "x-amz-date",
 "x-amz-content-sha256",
 "amz-sdk-invocation-id",
 "amz-sdk-request",
 "x-amzn-bedrock-api-key",
 ],
 }

Expand Down

2 changes: 2 additions & 0 deletions
py/src/braintrust/integrations/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@
from .agno import AgnoIntegration
from .anthropic import AnthropicIntegration
from .autogen import AutoGenIntegration
from .bedrock_runtime import BedrockRuntimeIntegration
from .claude_agent_sdk import ClaudeAgentSDKIntegration
from .cohere import CohereIntegration
from .crewai import CrewAIIntegration
Expand All@@ -29,6 +30,7 @@
 "AgnoIntegration",
 "AnthropicIntegration",
 "AutoGenIntegration",
 "BedrockRuntimeIntegration",
 "ClaudeAgentSDKIntegration",
 "CohereIntegration",
 "CrewAIIntegration",
Expand Down

43 changes: 43 additions & 0 deletions
py/src/braintrust/integrations/auto_test_scripts/test_auto_bedrock_runtime.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
"""Test auto_instrument for boto3 Bedrock Runtime."""

import os


os.environ.setdefault("AWS_EC2_METADATA_DISABLED", "true")
if not os.environ.get("AWS_PROFILE") and not os.environ.get("AWS_BEARER_TOKEN_BEDROCK"):
 os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
 os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
 os.environ.setdefault("AWS_SESSION_TOKEN", "testing")

import boto3
from braintrust.auto import auto_instrument
from braintrust.integrations.test_utils import autoinstrument_test_context


MODEL = os.getenv("BRAINTRUST_BEDROCK_CONVERSE_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"

results = auto_instrument()
assert results.get("bedrock_runtime") is True

results2 = auto_instrument()
assert results2.get("bedrock_runtime") is True

with autoinstrument_test_context("test_auto_bedrock_runtime", integration="bedrock_runtime") as memory_logger:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 inferenceConfig={"maxTokens": 20, "temperature": 0},
 )
 assert response["output"]["message"]["role"] == "assistant"

 spans = memory_logger.pop()
 assert len(spans) == 1, f"Expected 1 span, got {len(spans)}"
 span = spans[0]
 assert span["metadata"]["provider"] == "bedrock"
 assert span["metadata"]["model"] == MODEL
 assert span["metadata"]["endpoint"] == "converse"
 assert span["span_attributes"]["name"] == "bedrock.converse"

print("SUCCESS")

17 changes: 17 additions & 0 deletions
py/src/braintrust/integrations/bedrock_runtime/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
"""Public entry points for the boto3 Bedrock Runtime integration."""

from .integration import BedrockRuntimeIntegration
from .patchers import wrap_bedrock_client


__all__ = ["BedrockRuntimeIntegration", "setup_bedrock", "wrap_bedrock"]


def setup_bedrock() -> bool:
 """Patch botocore client creation to auto-wrap Bedrock Runtime clients."""
 return BedrockRuntimeIntegration.setup()


def wrap_bedrock(client):
 """Instrument a boto3 Bedrock Runtime client instance in place."""
 return wrap_bedrock_client(client)

35 changes: 35 additions & 0 deletions
...braintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_auto_bedrock_runtime.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
interactions:
- request:
 body: '{"messages": [{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 "inferenceConfig": {"maxTokens": 20, "temperature": 0}}'
 headers:
 Content-Length:
 - '137'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9aLGIs
 RCwzIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":511},"output":{"message":{"content":[{"text":"Hi."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":6,"outputTokens":3,"serverToolUsage":{},"totalTokens":9}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '202'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:03 GMT
 x-amzn-RequestId:
 - caa9fd4f-69d2-459b-baf5-0ad888a79845
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...edrock_runtime/cassettes/1.34.116/test_setup_bedrock_converse_auto_wraps_new_clients.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":613},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:01 GMT
 x-amzn-RequestId:
 - 7c3bdcc8-1124-46b5-982e-3ced2d1a5371
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...raintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_wrap_bedrock_converse.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":486},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:00 GMT
 x-amzn-RequestId:
 - 1935727d-5309-4a70-8c15-1ce0ffbe877e
 status:
 code: 200
 message: OK
version: 1
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
feat(bedrock): trace bedrock runtime calls by AbhiPrasad · Pull Request #519 · braintrustdata/braintrust-sdk-python · GitHub
Skip to content
feat(bedrock): trace bedrock runtime calls#519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Abhijeet Prasad (AbhiPrasad)
merged 1 commit into

from

Jun 16, 2026
 Merged


feat(bedrock): trace bedrock runtime calls
#519
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


1 commit


Select commit


File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


1 change: 1 addition & 0 deletions
examples/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,7 @@ Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `bedrock_runtime/` | boto3 Bedrock Runtime `converse()` call against Amazon Nova Lite |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
Expand Down

25 changes: 25 additions & 0 deletions
examples/bedrock_runtime/README.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
# AWS Bedrock + Braintrust

Calls `braintrust.auto_instrument()` to patch boto3's Bedrock Runtime client factory, then runs a single `client.converse(...)` call against Amazon Nova Lite. The trace captures a parent `answer_question` span plus a child `bedrock.converse` LLM span with input, output, metadata, and token metrics.

## Run

```bash
export BRAINTRUST_API_KEY=...
export AWS_BEARER_TOKEN_BEDROCK=...
export AWS_REGION=us-east-1
export AWS_DEFAULT_REGION=us-east-1

uv sync
uv run python example.py
```

If you use IAM credentials instead of a Bedrock API key, set the usual AWS variables instead:

```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if using temporary credentials
```

The default model is `us.amazon.nova-lite-v1:0`. Override it with `BRAINTRUST_BEDROCK_MODEL` if needed.

38 changes: 38 additions & 0 deletions
examples/bedrock_runtime/example.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env python
"""boto3 Bedrock Runtime client traced via braintrust.auto_instrument()."""

import os

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-bedrock")

import boto3 # noqa: E402


MODEL = os.getenv("BRAINTRUST_BEDROCK_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"


@braintrust.traced
def answer_question(question: str) -> str:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": question}]}],
 inferenceConfig={"maxTokens": 64, "temperature": 0},
 )
 message = response["output"]["message"]
 return "".join(block.get("text", "") for block in message.get("content", []))


def main() -> None:
 answer = answer_question("What is the capital of Australia? Reply in one sentence.")
 print(answer)
 braintrust.flush()


if __name__ == "__main__":
 main()

12 changes: 12 additions & 0 deletions
examples/bedrock_runtime/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
[project]
name = "braintrust-bedrock-example"
version = "0.1.0"
description = "boto3 Bedrock Runtime client traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
 "braintrust",
 "boto3",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }

13 changes: 13 additions & 0 deletions
py/noxfile.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -230,6 +230,19 @@ def test_cohere(session, version):
 _run_tests(session, f"{INTEGRATION_DIR}/cohere/test_cohere.py", version=version)


BOTO3_VERSIONS = _get_matrix_versions("boto3")


@nox.session()
@nox.parametrize("version", BOTO3_VERSIONS, ids=BOTO3_VERSIONS)
def test_bedrock_runtime(session, version):
 """Test the boto3 Bedrock Runtime integration."""
 _install_test_deps(session)
 _install_matrix_dep(session, "boto3", version)
 _install_matrix_dep(session, "botocore", version)
 _run_tests(session, f"{INTEGRATION_DIR}/bedrock_runtime/test_bedrock_runtime.py", version=version)


INSTRUCTOR_VERSIONS = _get_matrix_versions("instructor")


Expand Down

11 changes: 11 additions & 0 deletions
py/pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,14 @@ latest = "pytest==9.0.3"
[tool.braintrust.matrix.braintrust-core]
latest = "braintrust-core==0.0.59"

[tool.braintrust.matrix.boto3]
latest = "boto3==1.43.29"
"1.34.116" = "boto3==1.34.116"

[tool.braintrust.matrix.botocore]
latest = "botocore==1.43.29"
"1.34.116" = "botocore==1.34.116"

# ---------------------------------------------------------------------------
# Vendor packages — optional third-party packages the SDK can work without.
# Keys are matrix keys; values are Python import names. The noxfile uses this
Expand All@@ -461,6 +469,7 @@ agentscope = ["agentscope"]
agno = ["agno"]
autogen = ["autogen-agentchat"]
anthropic = ["anthropic"]
bedrock_runtime = ["boto3", "botocore"]
cohere = ["cohere"]
claude_agent_sdk = ["claude-agent-sdk"]
crewai = ["crewai"]
Expand DownExpand Up@@ -488,6 +497,8 @@ anthropic = "anthropic"
cohere = "cohere"
autoevals = "autoevals"
braintrust-core = "braintrust_core"
boto3 = "boto3"
botocore = "botocore"
crewai = "crewai"
dspy = "dspy"
google-adk = "google.adk"
Expand Down

5 changes: 5 additions & 0 deletions
py/src/braintrust/auto.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,7 @@
 AgnoIntegration,
 AnthropicIntegration,
 AutoGenIntegration,
 BedrockRuntimeIntegration,
 ClaudeAgentSDKIntegration,
 CohereIntegration,
 CrewAIIntegration,
Expand DownExpand Up@@ -72,6 +73,7 @@ def auto_instrument(
 openai_agents: bool = True,
 cohere: bool = True,
 autogen: bool = True,
 bedrock: bool = True,
 crewai: bool = True,
 strands: bool = True,
 temporal: bool = True,
Expand DownExpand Up@@ -106,6 +108,7 @@ def auto_instrument(
 openai_agents: Enable OpenAI Agents SDK instrumentation (default: True)
 cohere: Enable Cohere instrumentation (default: True)
 autogen: Enable AutoGen instrumentation (default: True)
 bedrock: Enable boto3 Bedrock Runtime instrumentation (default: True)
 crewai: Enable CrewAI instrumentation (default: True)
 strands: Enable Strands Agents instrumentation (default: True)
 temporal: Enable Temporal instrumentation (default: True)
Expand DownExpand Up@@ -195,6 +198,8 @@ def auto_instrument(
 results["cohere"] = _instrument_integration(CohereIntegration)
 if autogen:
 results["autogen"] = _instrument_integration(AutoGenIntegration)
 if bedrock:
 results["bedrock_runtime"] = _instrument_integration(BedrockRuntimeIntegration)
 if crewai:
 results["crewai"] = _instrument_integration(CrewAIIntegration)
 if strands:
Expand Down

6 changes: 6 additions & 0 deletions
py/src/braintrust/conftest.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -247,6 +247,12 @@ def get_vcr_config():
 "openai-api-key",
 "x-goog-api-key",
 "x-bt-auth-token",
 "x-amz-security-token",
 "x-amz-date",
 "x-amz-content-sha256",
 "amz-sdk-invocation-id",
 "amz-sdk-request",
 "x-amzn-bedrock-api-key",
 ],
 }

Expand Down

2 changes: 2 additions & 0 deletions
py/src/braintrust/integrations/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@
from .agno import AgnoIntegration
from .anthropic import AnthropicIntegration
from .autogen import AutoGenIntegration
from .bedrock_runtime import BedrockRuntimeIntegration
from .claude_agent_sdk import ClaudeAgentSDKIntegration
from .cohere import CohereIntegration
from .crewai import CrewAIIntegration
Expand All@@ -29,6 +30,7 @@
 "AgnoIntegration",
 "AnthropicIntegration",
 "AutoGenIntegration",
 "BedrockRuntimeIntegration",
 "ClaudeAgentSDKIntegration",
 "CohereIntegration",
 "CrewAIIntegration",
Expand Down

43 changes: 43 additions & 0 deletions
py/src/braintrust/integrations/auto_test_scripts/test_auto_bedrock_runtime.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
"""Test auto_instrument for boto3 Bedrock Runtime."""

import os


os.environ.setdefault("AWS_EC2_METADATA_DISABLED", "true")
if not os.environ.get("AWS_PROFILE") and not os.environ.get("AWS_BEARER_TOKEN_BEDROCK"):
 os.environ.setdefault("AWS_ACCESS_KEY_ID", "testing")
 os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "testing")
 os.environ.setdefault("AWS_SESSION_TOKEN", "testing")

import boto3
from braintrust.auto import auto_instrument
from braintrust.integrations.test_utils import autoinstrument_test_context


MODEL = os.getenv("BRAINTRUST_BEDROCK_CONVERSE_MODEL", "us.amazon.nova-lite-v1:0")
REGION = os.getenv("AWS_REGION") or os.getenv("AWS_DEFAULT_REGION") or "us-east-1"

results = auto_instrument()
assert results.get("bedrock_runtime") is True

results2 = auto_instrument()
assert results2.get("bedrock_runtime") is True

with autoinstrument_test_context("test_auto_bedrock_runtime", integration="bedrock_runtime") as memory_logger:
 client = boto3.client("bedrock-runtime", region_name=REGION)
 response = client.converse(
 modelId=MODEL,
 messages=[{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 inferenceConfig={"maxTokens": 20, "temperature": 0},
 )
 assert response["output"]["message"]["role"] == "assistant"

 spans = memory_logger.pop()
 assert len(spans) == 1, f"Expected 1 span, got {len(spans)}"
 span = spans[0]
 assert span["metadata"]["provider"] == "bedrock"
 assert span["metadata"]["model"] == MODEL
 assert span["metadata"]["endpoint"] == "converse"
 assert span["span_attributes"]["name"] == "bedrock.converse"

print("SUCCESS")

17 changes: 17 additions & 0 deletions
py/src/braintrust/integrations/bedrock_runtime/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
"""Public entry points for the boto3 Bedrock Runtime integration."""

from .integration import BedrockRuntimeIntegration
from .patchers import wrap_bedrock_client


__all__ = ["BedrockRuntimeIntegration", "setup_bedrock", "wrap_bedrock"]


def setup_bedrock() -> bool:
 """Patch botocore client creation to auto-wrap Bedrock Runtime clients."""
 return BedrockRuntimeIntegration.setup()


def wrap_bedrock(client):
 """Instrument a boto3 Bedrock Runtime client instance in place."""
 return wrap_bedrock_client(client)

35 changes: 35 additions & 0 deletions
...braintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_auto_bedrock_runtime.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
interactions:
- request:
 body: '{"messages": [{"role": "user", "content": [{"text": "Say hello in one word."}]}],
 "inferenceConfig": {"maxTokens": 20, "temperature": 0}}'
 headers:
 Content-Length:
 - '137'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9aLGIs
 RCwzIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":511},"output":{"message":{"content":[{"text":"Hi."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":6,"outputTokens":3,"serverToolUsage":{},"totalTokens":9}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '202'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:03 GMT
 x-amzn-RequestId:
 - caa9fd4f-69d2-459b-baf5-0ad888a79845
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...edrock_runtime/cassettes/1.34.116/test_setup_bedrock_converse_auto_wraps_new_clients.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":613},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:01 GMT
 x-amzn-RequestId:
 - 7c3bdcc8-1124-46b5-982e-3ced2d1a5371
 status:
 code: 200
 message: OK
version: 1

36 changes: 36 additions & 0 deletions
...raintrust/integrations/bedrock_runtime/cassettes/1.34.116/test_wrap_bedrock_converse.yaml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
interactions:
- request:
 body: '{"system": [{"text": "You answer with concise lowercase text."}], "messages":
 [{"role": "user", "content": [{"text": "Say hello in one word."}]}], "inferenceConfig":
 {"maxTokens": 20, "temperature": 0, "topP": 0.9, "stopSequences": ["STOP"]}}'
 headers:
 Content-Length:
 - '242'
 Content-Type:
 - !!binary |
 YXBwbGljYXRpb24vanNvbg==
 User-Agent:
 - !!binary |
 Qm90bzMvMS40My4yOSBtZC9Cb3RvY29yZSMxLjQzLjI5IHVhLzIuMSBvcy9tYWNvcyMyNS41LjAg
 bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjE0LjMgbWQvcHlpbXBsI0NQeXRob24gbS9iLDMs
 WixEIGNmZy9yZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjQzLjI5
 method: POST
 uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.amazon.nova-lite-v1%3A0/converse
 response:
 body:
 string: '{"metrics":{"latencyMs":486},"output":{"message":{"content":[{"text":"hi"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":13,"outputTokens":2,"serverToolUsage":{},"totalTokens":15}}'
 headers:
 Connection:
 - keep-alive
 Content-Length:
 - '203'
 Content-Type:
 - application/json
 Date:
 - Mon, 15 Jun 2026 20:24:00 GMT
 x-amzn-RequestId:
 - 1935727d-5309-4a70-8c15-1ce0ffbe877e
 status:
 code: 200
 message: OK
version: 1
Loading
Loading