feat(functions): update functions tools and scorers

[Cedric / ViaDézo1er (viadezo1er)]

scorers.sh:

Details

# Command examples for `bt scorers update`.## Run these one at a time from the repository root. Replace the synthetic# slugs with scorers from your test environment.
SCORER="test-update-scorer"
CLASSIFIER="test-update-classifier"# Inspect a scorer before changing it.
bt scorers view "$SCORER"# Update fields shared by prompt- and code-backed scorers.
bt scorers update "$SCORER" \
--name "Test update scorer renamed" \
--new-slug test-update-scorer-renamed \
--description "Synthetic scorer updated from bt" \
--yes
SCORER="test-update-scorer-renamed"# Change only the model and model parameters. Existing messages and parser# configuration are preserved.
bt scorers update "$SCORER" \
--model gpt-5.4-nano \
--reasoning-effort none \
--temperature 0.1 \
--yes
# Replace chat messages from a JSON file.
bt scorers update "$SCORER" \
--messages @messages.json \
--yes
# Replace score choices and set the passing threshold.
bt scorers update "$SCORER" \
--choice-scores @choice-scores.json \
--pass-threshold 0.7 \
--use-cot=false \
--yes
# Replace classifier labels and permit no matching label.
bt scorers update "$CLASSIFIER" \
--classifications @classifications.json \
--allow-no-match \
--yes
# Deep-merge metadata without removing the pass threshold or existing keys.
bt scorers update "$SCORER" \
--metadata @metadata.json \
--yes
# Configure a structured model response.
bt scorers update "$SCORER" \
--response-format @response-format.json \
--yes
# Apply fields without dedicated flags through an arbitrary JSON patch.
bt scorers update "$SCORER" \
--patch @scorer-patch.json \
--yes
# Inspect the resulting definition and verify unchanged prompt fields remain.
bt scorers view "$SCORER"

The JSON files:

Details

choice-scores.json:

{
"PASS": 1,
"FAIL": 0
}

classifications.json:

["safe", "unsafe", "uncertain"]

messages.json:

[
{
"role": "system",
"content": "You are a strict evaluator. Return exactly PASS or FAIL."
},
{
"role": "user",
"content": "Question: {{input.question}}\nExpected: {{expected}}\nCandidate: {{output}}"
}
]

metadata.json:

{
"owner": "test-team",
"purpose": "synthetic-update-example"
}

response-format.json:

{
"type": "json_schema",
"json_schema": {
"name": "grader_result",
"description": "A synthetic grading result",
"schema": {
"type": "object",
"properties": {
"label": {
"type": "string",
"enum": ["PASS", "FAIL"]
}
},
"required": ["label"],
"additionalProperties": false
},
"strict": true
}
}

scorer-patch.json:

{
"prompt_data": {
"options": {
"params": {
"use_cache": false
}
}
}
}

And its output:

Details

Viewing Test update scorer
Slug: test-update-scorer
Type: scorer
Description: Synthetic scorer for bt update testing
Model: gpt-4o-mini
✓ Updated 'Test update scorer'
Run `bt scorers view test-update-scorer-renamed` to inspect the updated definition.
✓ Updated 'Test update scorer renamed'
Run `bt scorers view test-update-scorer-renamed` to inspect the updated definition.
✓ Updated 'Test update scorer renamed'
Run `bt scorers view test-update-scorer-renamed` to inspect the updated definition.
Viewing Test update scorer renamed
Slug: test-update-scorer-renamed
Type: scorer
Description: Synthetic scorer updated from bt
Model: gpt-5.4-nano
reasoning_effort: none
temperature: 0.1
use_cache: false
Metadata: {
"__pass_threshold": 0.7,
"owner": "test-team",
"purpose": "synthetic-update-example"
}

functions.sh:

Details

# Command examples for `bt functions update`.## `bt functions update` intentionally exposes common function fields and the# arbitrary patch escape hatch, rather than scorer-only prompt/parser flags.
FUNCTION="test-update-function"
bt functions view "$FUNCTION"# Update common fields and select the existing function by slug.
bt functions update "$FUNCTION" \
--name "Test update function renamed" \
--new-slug test-update-function-renamed \
--description "Synthetic function description" \
--metadata @metadata.json \
--yes
FUNCTION="test-update-function-renamed"# Restrict lookup to a function type when needed.
bt functions update "$FUNCTION" \
--type task \
--patch @function-patch.json \
--yes
# Function IDs are also accepted.
bt functions update --id fn_test_function \
--description "Updated by id" \
--yes
bt functions view "$FUNCTION"

The JSON file:

Details

function-patch.json:

{
"tags": ["synthetic", "updated-from-cli"]
}

And its output:

Details

Viewing Test update function
Slug: test-update-function
Type: task
Description: Synthetic code-backed resource for bt update examples
Runtime: node 22.15.0
✓ Updated 'Test update function'
Run `bt functions view test-update-function-renamed` to inspect the updated definition.
✓ Updated 'Test update function renamed'
Run `bt functions view test-update-function-renamed` to inspect the updated definition.
Viewing Test update function renamed
Slug: test-update-function-renamed
Type: task
Description: Synthetic function description
Runtime: node 22.15.0
Tags: synthetic, updated-from-cli

tools.sh:

Details

# Command examples for `bt tools update`.## Code-backed tools support common fields and arbitrary patches. Prompt-backed# tools additionally support completion prompts and chat messages.
TOOL="test-update-tool"
PROMPT_TOOL="test-prompt-tool"
bt tools view "$TOOL"# Update common fields on a code-backed tool.
bt tools update "$TOOL" \
--name "Test update tool renamed" \
--new-slug test-update-tool-renamed \
--description "Synthetic tool description" \
--metadata @metadata.json \
--yes
TOOL="test-update-tool-renamed"# Apply fields without dedicated flags through a JSON patch.
bt tools update "$TOOL" \
--patch @tool-patch.json \
--yes
# Replace a completion prompt on a prompt-backed tool.
bt tools update "$PROMPT_TOOL" \
--prompt @tool-prompt.txt \
--yes
# Or switch the prompt-backed tool to chat messages from JSON or YAML.
bt tools update "$PROMPT_TOOL" \
--messages @tool-messages.yaml \
--yes
bt tools view "$TOOL"
bt tools view "$PROMPT_TOOL"

The supporting files:

Details

tool-patch.json:

{
"tags": ["synthetic", "updated-from-cli"]
}

tool-prompt.txt:

Look up the synthetic value: {{value}}

tool-messages.yaml:

- role: systemcontent: Use the supplied value.
- role: usercontent: "{{value}}"

And its output:

Details

Viewing Test update tool
Slug: test-update-tool
Type: tool
Description: Synthetic code-backed tool for bt update testing
Runtime: node 22.15.0
✓ Updated 'Test update tool'
Run `bt tools view test-update-tool-renamed` to inspect the updated definition.
✓ Updated 'Test update tool renamed'
Run `bt tools view test-update-tool-renamed` to inspect the updated definition.
✓ Updated 'Test prompt tool'
Run `bt tools view test-prompt-tool` to inspect the updated definition.
✓ Updated 'Test prompt tool'
Run `bt tools view test-prompt-tool` to inspect the updated definition.
Viewing Test update tool renamed
Slug: test-update-tool-renamed
Type: tool
Description: Synthetic tool description
Runtime: node 22.15.0
Tags: synthetic, updated-from-cli
Viewing Test prompt tool
Slug: test-prompt-tool
Type: tool
Model: gpt-4o-mini
┃ system
│ Use the supplied value.
┃ user
│ {{value}}

AI Summary

• add distinct update interfaces for functions, tools, and scorers instead of exposing scorer-only flags in every namespace
• add common --name, --new-slug, --description, --metadata, and --patch options across all three update commands
• keep model, prompt-parser, score, and classification configuration on bt scorers update
• add --prompt and JSON/YAML --messages to bt tools update for prompt-backed tools
• support selection by slug or function ID, including interactive selection when no selector is provided
• preserve unchanged prompt_data and deep-merge metadata before calling the PATCH API

Behavior

• bt functions update exposes common function fields, --type, and the arbitrary JSON patch escape hatch
• bt tools update supports common fields for code-backed tools and prompt replacement for prompt-backed tools
• bt scorers update supports common fields for code scorers and full prompt/parser configuration for LLM scorers and classifiers
• --slug identifies the current resource; --new-slug changes it and preflights collisions
• switching between completion and chat prompts replaces the discriminated prompt block without leaving fields from the previous prompt kind
• rejects prompt updates for code-backed resources with an actionable bt functions push hint
• rejects incompatible scorer/classifier output changes and reports expected validation failures as user errors
• supports confirmation prompts, --yes, and machine-readable --json output
• keeps progress and status output on stderr

[github-actions]

Latest downloadable build artifacts for this PR commit 3a5cccd6b290:

• Workflow run: https://github.com/braintrustdata/bt/actions/runs/32537898143
• Download all artifacts (GitHub CLI): gh run download 32537898143 --repo braintrustdata/bt
• Installers are published from main automatically. To publish one for a PR branch, run release-canary manually via workflow_dispatch.

Available artifact names

• artifacts-build-global
• artifacts-build-local-aarch64-pc-windows-msvc
• artifacts-build-local-x86_64-pc-windows-msvc
• artifacts-build-local-x86_64-apple-darwin
• artifacts-build-local-x86_64-unknown-linux-musl
• artifacts-build-local-x86_64-unknown-linux-gnu
• artifacts-build-local-aarch64-apple-darwin
• artifacts-build-local-aarch64-unknown-linux-gnu
• artifacts-plan-dist-manifest
• cargo-dist-cache