Skip to content

Adding a ctx_* tool requires updating four unlinked lists, and missing one degrades the light preset silently #294

Description

@iceteaSA

The prompt-surface machinery keeps four separate enumerations of the active tool set, with no test asserting they agree. Adding a tool means finding all four; miss one and the failure mode ranges from a loud throw to a silent behaviour change.

#listfile
1ACTIVE_TOOL_IDSpackages/plugin/scripts/prompt-surface-measurement.ts:24
2buildToolDefinitions()'s definitions literalsame file, :124
3PROMPT_SURFACE_TOOL_IDSpackages/plugin/src/shared/prompt-surface-runtime.ts:26
4LIGHT_TOOL_DESCRIPTIONSsame file, :36

git grep -l ACTIVE_TOOL_IDS -- '*.test.ts' returns nothing, so nothing links them.

The one that matters

Missing #1 or #2 throws (Tool definition X is missing from the measurement catalog) — annoying but self-announcing. Missing #3 changes model-facing behaviour with no error at all: descriptionFor early-returns for any id outside the set, so under prompt_surface: light the tool keeps its full-length description.

// prompt-surface-runtime.ts:198descriptionFor(toolId,fullDescription){if(!PROMPT_SURFACE_TOOL_ID_SET.has(toolId))returnfullDescription;

Reproduced on clean master @ 8fd27463 (fresh worktree + bun install), asking the real runtime for a listed vs unlisted id under the light preset:

preset: light
listed ctx_search -> For ctx_search users, retrieve only hidden rec… ← light prose
UNLISTED new tool -> FULL-LENGTH-new-tool ← full prose
unlisted silently keeps FULL description: true

So a tool registered but absent from #3 quietly bypasses the light preset. Given light exists to hold a token ceiling, that is a budget leak the budget check cannot see — the light measurement only walks ACTIVE_TOOL_IDS, so a tool missing from #3 but present in #1 is measured as if it had a light description it never actually serves.

Why I am reporting it

Downstream I added two tools and hit all four separately over one release, in the worst order — #1 and #2 threw so I fixed them, tests went green, and #3 was still wrong. I only found it by reading descriptionFor. The next tool added upstream walks into the same trap, and if light is the surface being protected, this is the list most worth guarding.

Suggested fix

A guard test asserting the four agree, derived rather than hardcoded so it cannot rot:

  • ACTIVE_TOOL_IDS set === Object.keys(buildToolDefinitions())
  • PROMPT_SURFACE_TOOL_IDS set === Object.keys(LIGHT_TOOL_DESCRIPTIONS)
  • and the two sets equal each other

Three assertions catch every ordering of the mistake, including the silent one. Happy to open a PR if you want it — it is small, and I have the reproduction already.

Alternative worth considering

If the sets are meant to be identical, they could collapse to one exported constant with the others derived from it, making drift unrepresentable rather than merely detected. That is a larger change and touches a governed surface, so I would rather ask than assume: is there a reason PROMPT_SURFACE_TOOL_IDS is deliberately allowed to differ from ACTIVE_TOOL_IDS (e.g. a tool that should never get a light variant)? If so the guard test is the right shape and the collapse is wrong.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions