Skip to content

fix(threads): title threads that open with a skill invocation - #3887

Open
Willhong wants to merge 1 commit into
get-bb:mainfrom
Willhong:fix/title-generation-skill-invocations
Open

Willhong wants to merge 1 commit into
get-bb:mainfrom
Willhong:fix/title-generation-skill-invocations

Conversation

@Willhong

Copy link
Copy Markdown
Contributor

Human comments

What was wrong

shouldGenerateThreadTitle decided eligibility from cleanPromptText, which concatenates part.text and ignores part.mentions, so a /skill-name command mention was counted as an ordinary word against the five-word minimum. A first message that invokes a skill is short by design, so it fell under the threshold and title inference was skipped entirely — the provisioning transcript records reason: "too-short" with durationMs: 0, the thread keeps title = NULL, and getThreadDisplayTitle falls back to the raw prompt, leaving sidebar rows that read /sync-repo. The gate landed in 6a4b7489e (2026-04-10), two months before command mentions existed in prompts (27361799e, #92), so it never made a decision about skills; this is an unhandled interaction, not a policy. Across a personal database of 1265 threads, 19 of 19 command-first prompts under five words were untitled.

What changed

apps/server/src/services/threads/title-generation.ts

  • collectInvokedPromptCommands reads kind: "command" mentions off the input, deduplicated by trigger + name and kept in prompt order.
  • shouldGenerateThreadTitle exempts a prompt carrying any command mention from MIN_TITLE_GENERATION_WORDS. Invoking a skill is itself a statement of intent. Prompts without a command mention keep the existing five-word rule exactly.
  • generateThreadMetadataWithOutcome now hands the model the prompt body with the command ranges removed (via removeCommandMentionsFromPromptInput) plus the invoked command names as a separate variable, instead of the raw text. When stripping leaves nothing — a bare /weekly-report — it falls back to the clamped raw text so the model still has the command to title from.
  • The existing 80-character clamp moved into clampPromptText so it applies to the stripped body as well as the fallback.
  • deriveTitleFallback is unchanged: a failed or skipped inference still degrades to today's behavior.

packages/templates/src/templates/generate-thread-metadata.md

  • New optional variable invokedCommands?, rendered only when the prompt invokes something: it tells the model those names describe how the work is carried out, so the title should name the work they are applied to — and that when the prompt says nothing else, the title should describe what the command itself does. The template already asked for this ("it is the problem that should be the title, not the tools"); it simply never received the information.

No wire changes, so HOST_DAEMON_PROTOCOL_VERSION is untouched. No CLI, config or plugin API surface changes. shouldGenerateThreadTitle is the only implementation of this rule in the repo, so no client-side duplicate needed updating.

How you verified

Tests added (all fail on the parent commit, pass on this one — verified by reverting only the two source files and rerunning):

  • apps/server/test/threads/title-generation.test.ts — a bare skill invocation and a short one with arguments are both eligible; the raw command text is still the fallback; commands are collected once each in prompt order.
  • apps/server/test/threads/generated-thread-titles.test.ts — the prompt sent to inference no longer contains the joined /sync-repo and drop text, does carry the invoked-commands sentence, and a plain prompt does not.
# before (source reverted, tests kept)
Tests  4 failed | 22 passed (26)

# after
pnpm exec turbo run test --filter=@bb/server -- title-generation generated-thread-titles
Tests  26 passed (26)

pnpm exec turbo run test --filter=@bb/server
Test Files  268 passed | 3 skipped (271)
     Tests  2730 passed | 1 skipped (2731)

pnpm exec turbo run typecheck --filter=@bb/server        # clean
pnpm exec turbo run typecheck test --filter=@bb/templates # clean, 55 passed

Fixes #3886

AGENT GENERATED

The five-word gate in shouldGenerateThreadTitle counted the `/skill-name`
token as an ordinary word, so a first message that invokes a skill almost
always fell under the threshold and inference never ran. The thread kept a
null title and the sidebar fell back to the raw prompt text.

Exempt prompts carrying a command mention from the length gate, strip the
command ranges out of the text handed to the model, and name the invoked
commands in the metadata template so the title describes the work rather
than the tool.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A thread whose first message invokes a skill is never titled

1 participant