The prompt-surface machinery keeps four separate enumerations of the active tool set, with no test asserting they agree. Adding a tool means finding all four; miss one and the failure mode ranges from a loud throw to a silent behaviour change.
| # | list | file |
|---|
| 1 | ACTIVE_TOOL_IDS | packages/plugin/scripts/prompt-surface-measurement.ts:24 |
| 2 | buildToolDefinitions()'s definitions literal | same file, :124 |
| 3 | PROMPT_SURFACE_TOOL_IDS | packages/plugin/src/shared/prompt-surface-runtime.ts:26 |
| 4 | LIGHT_TOOL_DESCRIPTIONS | same file, :36 |
git grep -l ACTIVE_TOOL_IDS -- '*.test.ts' returns nothing, so nothing links them.
The one that matters
Missing #1 or #2 throws (Tool definition X is missing from the measurement catalog) — annoying but self-announcing. Missing #3 changes model-facing behaviour with no error at all: descriptionFor early-returns for any id outside the set, so under prompt_surface: light the tool keeps its full-length description.
// prompt-surface-runtime.ts:198descriptionFor(toolId,fullDescription){if(!PROMPT_SURFACE_TOOL_ID_SET.has(toolId))returnfullDescription;Reproduced on clean master @ 8fd27463 (fresh worktree + bun install), asking the real runtime for a listed vs unlisted id under the light preset:
preset: light
listed ctx_search -> For ctx_search users, retrieve only hidden rec… ← light prose
UNLISTED new tool -> FULL-LENGTH-new-tool ← full prose
unlisted silently keeps FULL description: true
So a tool registered but absent from #3 quietly bypasses the light preset. Given light exists to hold a token ceiling, that is a budget leak the budget check cannot see — the light measurement only walks ACTIVE_TOOL_IDS, so a tool missing from #3 but present in #1 is measured as if it had a light description it never actually serves.
Why I am reporting it
Downstream I added two tools and hit all four separately over one release, in the worst order — #1 and #2 threw so I fixed them, tests went green, and #3 was still wrong. I only found it by reading descriptionFor. The next tool added upstream walks into the same trap, and if light is the surface being protected, this is the list most worth guarding.
Suggested fix
A guard test asserting the four agree, derived rather than hardcoded so it cannot rot:
ACTIVE_TOOL_IDS set === Object.keys(buildToolDefinitions())PROMPT_SURFACE_TOOL_IDS set === Object.keys(LIGHT_TOOL_DESCRIPTIONS)- and the two sets equal each other
Three assertions catch every ordering of the mistake, including the silent one. Happy to open a PR if you want it — it is small, and I have the reproduction already.
Alternative worth considering
If the sets are meant to be identical, they could collapse to one exported constant with the others derived from it, making drift unrepresentable rather than merely detected. That is a larger change and touches a governed surface, so I would rather ask than assume: is there a reason PROMPT_SURFACE_TOOL_IDS is deliberately allowed to differ from ACTIVE_TOOL_IDS (e.g. a tool that should never get a light variant)? If so the guard test is the right shape and the collapse is wrong.
The prompt-surface machinery keeps four separate enumerations of the active tool set, with no test asserting they agree. Adding a tool means finding all four; miss one and the failure mode ranges from a loud throw to a silent behaviour change.
ACTIVE_TOOL_IDSpackages/plugin/scripts/prompt-surface-measurement.ts:24buildToolDefinitions()'sdefinitionsliteral:124PROMPT_SURFACE_TOOL_IDSpackages/plugin/src/shared/prompt-surface-runtime.ts:26LIGHT_TOOL_DESCRIPTIONS:36git grep -l ACTIVE_TOOL_IDS -- '*.test.ts'returns nothing, so nothing links them.The one that matters
Missing #1 or #2 throws (
Tool definition X is missing from the measurement catalog) — annoying but self-announcing. Missing #3 changes model-facing behaviour with no error at all:descriptionForearly-returns for any id outside the set, so underprompt_surface: lightthe tool keeps its full-length description.Reproduced on clean
master@8fd27463(fresh worktree +bun install), asking the real runtime for a listed vs unlisted id under the light preset:So a tool registered but absent from #3 quietly bypasses the light preset. Given
lightexists to hold a token ceiling, that is a budget leak the budget check cannot see — the light measurement only walksACTIVE_TOOL_IDS, so a tool missing from #3 but present in #1 is measured as if it had a light description it never actually serves.Why I am reporting it
Downstream I added two tools and hit all four separately over one release, in the worst order — #1 and #2 threw so I fixed them, tests went green, and #3 was still wrong. I only found it by reading
descriptionFor. The next tool added upstream walks into the same trap, and iflightis the surface being protected, this is the list most worth guarding.Suggested fix
A guard test asserting the four agree, derived rather than hardcoded so it cannot rot:
ACTIVE_TOOL_IDSset ===Object.keys(buildToolDefinitions())PROMPT_SURFACE_TOOL_IDSset ===Object.keys(LIGHT_TOOL_DESCRIPTIONS)Three assertions catch every ordering of the mistake, including the silent one. Happy to open a PR if you want it — it is small, and I have the reproduction already.
Alternative worth considering
If the sets are meant to be identical, they could collapse to one exported constant with the others derived from it, making drift unrepresentable rather than merely detected. That is a larger change and touches a governed surface, so I would rather ask than assume: is there a reason
PROMPT_SURFACE_TOOL_IDSis deliberately allowed to differ fromACTIVE_TOOL_IDS(e.g. a tool that should never get a light variant)? If so the guard test is the right shape and the collapse is wrong.