feat(cli-registry): drive install.sh and the Docker agent image from the CLI catalogue - #380
Open
opticon454 wants to merge 5 commits into
Open
feat(cli-registry): drive install.sh and the Docker agent image from the CLI catalogue#380opticon454 wants to merge 5 commits into
opticon454 wants to merge 5 commits into
Conversation
… them PR B replaces nine hand-written `*_SEARCH_PATHS` arrays in install.sh with one block generated from `STOCK_CLIS`. This lands FIRST, against the hand-written arrays, so the replacement has something to be measured against. The arrays are not uniform, which is why "generate them from the registry" is a claim rather than an obvious truth: claude alone has `~/.claude/local`, opencode alone has `~/go/bin`, opencode/codex/gemini/pi/omp carry `~/.bun/bin` while dsh/grok/agy do not, and omp's `~/.omp/bin` sits second rather than first. A generated list that silently narrows leaves a user with that CLI installed being told no AI CLI was found — upstream `b6d0f1fa` is that bug, fixed for omp by hand after it shipped. The test asserts a three-way identity: the pinned literals equal what install.sh contains today, AND equal `searchDirs x binaries` from the registry, dir-major so the probe ORDER is pinned too and not just the set. Both halves were verified to fail independently — dropping one path from install.sh fails the first, changing one `searchDirs` entry fails the second — because a pin that cannot fail is worse than no pin. A fourth case asserts every stock CLI with a binary is covered, which is the omp bug restated so it cannot recur silently. No production code changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
…ocker build Two consumers of the registry cannot import TypeScript: `install.sh`, which runs via `curl | bash` before any checkout exists, and `scripts/build-agent-image.mjs`. Both currently hand-maintain their own CLI lists, and both have already drifted. `scripts/generate-cli-catalog.mts` (`npm run generate:cli-catalog`, plus a `--check` mode) emits from `STOCK_CLIS`: - `config/clis.stock.json` for the `.mjs` and the tests. It carries `enabled` — the field the earlier attempt omitted, which is how a disabled CLI's npm package still got baked into every agent image. - a marker-delimited block inside `install.sh`, embedded rather than fetched. The embedded copy is the FULL catalogue on purpose: the earlier design fetched it and fell back to a hardcoded two-CLI list, degrading silently on an empty response. There is no degraded mode to fall into now. The block is bash 3.2 safe: parallel indexed arrays, no associative arrays, no namerefs, no mapfile. Variable-length lists use OFFSET/LENGTH windows into one flat array rather than a delimiter, so a $HOME containing a space needs no IFS handling and `shell` (no binaries) gets length 0 and is never iterated. Search paths are emitted dir-major, matching the probe order the hand-written arrays use and `test/install-sh-detection-parity.test.ts` pins. Only fields the two consumers need are exported. `launch`/`env`/`capabilities`/ `overlays` are spawn-time concerns the server alone interprets, and a test asserts they never leak into the artifact. `main()` sits behind an `isMainModule()` guard so the sync test can import the renderers. Without it, importing the module would rewrite the artifacts as a side effect of checking them — passing always, guarding never. This commit adds the block; it does not yet delete the hand-written arrays, so the detection pin keeps measuring both against each other. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
…om the catalogue install.sh carried nine search-path arrays, eighteen near-identical check_<cli>/get_<cli>_path functions, and three separately hand-maintained enumerations of all nine CLIs. They had to agree and did not: upstream b6d0f1f is "wire OMP into install.sh's CLI detection (it had none)", and the section comment above the roll-call named six of the nine. All of it now reads the generated catalogue. `detect_all_clis` resolves every CLI in one memoized pass into CLI_FOUND_PATH/CLI_FOUND_COUNT; `check_cli` and `get_cli_path` replace the eighteen pairs; the roll-call, the "no AI CLI found" gate and the closing reminder become loops. Probe order per CLI is unchanged and `test/install-sh-detection-parity.test.ts` proves it against the literals transcribed from the arrays this deletes. Behaviour changes worth naming: - The install menu is built from the catalogue, so it offers every enabled CLI that is not installed and ships a command — five instead of two. Gemini had a command in the registry and appeared in NO list in this script. - Its labels are now the registry's ("Claude" rather than "Claude Code"), the same trade PR A made for `codeman doctor` rows. A suffix map would just be the hand-maintained list again. - On a wget-only host the menu prints commands instead of running them. The registry's commands call curl, whereas the two literals this replaces went through download_to_stdout; rewriting curl to wget inside a string we are about to execute is the wrong instinct. The trust boundary is mechanical, not a promise: CLI_INSTALL_CMD_TRUSTED is written only from the generated per-platform arrays and is the only thing ever executed; CLI_INSTALL_CMD_DISPLAY is what the optional, opt-in refresh may rewrite. The refresh warns on all three failure shapes — empty body, unparseable content, failed fetch — which is the silent-degradation bug from the review, and it parses with node into tab-separated records read by `read`, never eval. Bash 3.2 throughout (macOS ships it): parallel indexed arrays, offset/length windows instead of delimiters, no associative arrays, namerefs, mapfile or here-strings. Verified by executing the script under a real bash 3.2 container, which is also now a CI step alongside `bash -n` and a catalogue `--check` — the empty-window case (`shell` has no binaries) is a runtime `set -u` abort that `bash -n` cannot see. Running it that way caught `detect_os` being called inside the platform loop: ten forks, and ten copies of one error, since a `die` inside `$( )` can only exit the subshell. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
docker/agent.Dockerfile hardcoded the four npm-published CLIs it installs, one of the several lists that had to be kept in step with the registry by hand. It now takes them as `ARG CLI_NPM_PACKAGES`, supplied by scripts/build-agent-image.mjs from config/clis.stock.json, with the default set to today's list so a bare `docker build` still produces the same image. The arg is expanded unquoted because word splitting is what turns the list into several arguments, which is exactly why every token is validated against ^[@A-Za-z0-9][@A-Za-z0-9/._-]*$ on the producing side; a package name carrying a space or a metacharacter is refused rather than reaching the RUN line. Verified by building the layer: four packages in, four arguments out, and the default still applies with no arg. The list is filtered on each entry's `enabled` flag — the field whose absence was the maintainer's §3 finding, where a CLI shipping disabled still got baked into every image. No stock entry is disabled today, so that assertion would pass vacuously; a unit test feeds the pure helper a fabricated disabled entry so the fix is covered now rather than the first time someone ships one.⚠️ It reads the STOCK catalogue, never the merged registry. A user's ~/.codeman/clis.json must not change what is inside an image tagged codeman/agent:base, or two machines holding that tag hold different images. Four CLIs keep hand-written layers because the registry cannot describe what makes them special: pi's --ignore-scripts, deepseek's pnpm companion and dsh-tui profile, and the three standalone installers. Rather than extend the schema for a Docker-only benefit, the coverage test requires each to carry a written reason AND still be present, so an exclusion cannot quietly become an omission. There are two producers of this command line and there have to be — the .mjs cannot import TypeScript, and src/docker-hosts.ts builds the same argv for the in-app auto-build — so a parity test pins them together, package list, arg pairs and rendered argv. Their order is pinned too: a different order is a different RUN string and so a needless cache miss between the two build paths. docker/server.Dockerfile is deliberately NOT edited (PRs Ark0N#373 and Ark0N#377 both modify it); its narrower list is asserted as a declared omission list instead, so the divergence is reviewable without touching the file. Also fixes the in-app hint at index.html, which the new coverage test caught still omitting omp. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
…boundary Adds a "Consumers outside the server" section covering the two generated artifacts, why each exists (neither install.sh nor a .mjs can import TypeScript), what is deliberately NOT exported and why, the three-rule install command trust boundary, and the bash 3.2 constraint with the offset/length window shape it forces. The adding-a-CLI checklist gains the regenerate step, since forgetting it is how the installer would keep detecting the old set while the server offers the new one — the drift this change removes, one level out. docs/docker-cases.md gains how CLI_NPM_PACKAGES is derived, why it reads the stock catalogue and not the merged registry, and a table of the four documented Dockerfile special cases with their reasons. CLAUDE.md gains a command row and names the generated block, the bash 3.2 rule and the trust boundary in its install.sh paragraph. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
PR A made every run mode a data-driven
CliEntry, but stopped at the server boundary. Everything outside the Node process still hand-maintains its own CLI list, and those lists have already drifted in ways that shipped:b6d0f1fa— "wire OMP into install.sh's CLI detection (it had none)". A user with onlyompinstalled was told no AI CLI was found, and offered Claude Code.install.shcarried nine*_SEARCH_PATHSarrays, ~18 near-duplicatecheck_*/get_*_pathfunctions, and three separate hand-written enumerations of all nine CLIs.docker/agent.Dockerfilehardcoded its npm list, and theindex.htmlhint describing the image omittedomp.After this, adding a CLI to
stock.tsand runningnpm run generate:cli-catalogwires it into the installer's detection, its install menu, its closing reminder, and the agent image's npm layer.This is the second of the three PRs you staged: "install.sh + docker image reading the stock JSON, with the bash-3.2 fix and an
enabledfilter."Shape
Neither consumer can import TypeScript, so the generator emits two committed artifacts:
config/clis.stock.jsonbuild-agent-image.mjs.mjscannot import the TS registryinstall.shnpm run generate:cli-catalogwrites both;--checkfails on drift and runs in CI, andtest/cli-catalog-sync.test.tsfails if either goes stale.The embedded copy is the full catalogue. An earlier attempt fetched it with a hardcoded two-CLI fallback, which degrades silently on an empty response; there is no degraded mode to fall into now. An optional, opt-in refresh (
CODEMAN_CLI_CATALOGUE_URL/CODEMAN_REFRESH_CLI_CATALOGUE=1) warns loudly on all three failure shapes.Trust boundary, made mechanical
The server still never executes an entry's install command.
install.shexecutes only commands embedded in itself — same file, same TLS fetch, same commit as the one-liner that fetched it — and nothing pulled from the network at install time is ever run.That is now structural rather than a convention:
CLI_INSTALL_CMD_TRUSTED[]is written only by the generated block and is the array the menu executes;CLI_INSTALL_CMD_DISPLAY[]is what a refresh may overwrite and is display-only.test/install-sh-invariants.test.tsasserts the refresh function cannot write the executable array, and that it contains noeval.The
enabledfilterYour §3 finding: the earlier export carried no
enabledfield, so a CLI shipping disabled would still be baked into every image. The generated catalogue carries the flag and the image respects it.It reads the stock catalogue, not the merged registry, so a user's
~/.codeman/clis.jsoncannot change what is inside an image taggedcodeman/agent:base.Every stock entry is enabled today, so that assertion would pass vacuously — a unit test feeds the pure helper a fabricated disabled entry so the fix is genuinely covered now rather than the first time someone ships one.
bash 3.2
macOS ships bash 3.2 and the documented install is a curl pipe into bash under
set -euo pipefail, so nodeclare -A,mapfile, namerefs,${x,,}or here-strings. The catalogue is parallel indexed arrays with offset/length windows into flat arrays rather than delimiters — safe for a$HOMEwith spaces, and a length-0 entry (theshellpseudo-CLI) is never iterated.bash -ncannot see the failure mode that actually bites here: expanding an empty array underset -uis a runtime abort. So CI does both —bash -n, and executing the script inside a realbash:3.2container with detection exercised end to end. That needed a sourcing guard (CODEMAN_INSTALL_SH_LIB) so the library half can be sourced without running the installer.Behaviour changes, all in the installer
These are deliberate and worth your eye:
codeman doctorrows. A suffix map would just be the hand-maintained list again.wget-only host the menu prints the commands instead of running them. The registry's commands callcurl, whereas the two literals they replace went throughdownload_to_stdout. Rewritingcurltowgetinside a string about to be executed is the wrong instinct.CODEMAN_NONINTERACTIVE=1still defaults to Claude Code, unchanged.docker/server.Dockerfileis deliberately untouched#377 is still open and modifies it (#373 has since merged). Rather than edit a file another PR owns, its narrower CLI list is asserted as a declared omission list in
test/docker-agent-image-coverage.test.ts, so the divergence is visible and reviewable without touching the file. Adding a CLI there, or changing the intent, now fails a test and forces the list to be restated.Tests
Written in the order that makes each one non-vacuous, and each guard was proved by perturbing both sides.
test/install-sh-detection-parity.test.tssearchDirs × binariesreproduces them dir-major. They are not uniform (claude has~/.claude/local, opencode has~/go/binand~/.bun/bin), so silent narrowing means a user's installed CLI stops being found. Retargeted at the generated block afterwards.test/cli-catalog-sync.test.tsstock.tstest/install-sh-invariants.test.tsevalin the refresh path, and no leftover per-CLI detection names — derived fromCLI_IDS/CLI_ALL_BINSrather than an allowlist, which had already missed twotest/docker-agent-image-coverage.test.tsindex.htmlhint names every binarytest/agent-image-build-args-parity.test.ts.mjsand the TS mirror produce the same packages, pairs and argv — two producers of one command line is exactly the shape that drifts, and the failure would be two different images under one tagVerification: static gate green;
bash -nclean; executed under a realbash:3.2.57container; verified the Dockerfile ARG really word-splits (4 packages givesARGC=4) and that the default still applies with no arg. Full suite diffed by test name against the master baseline — zero new failures.Open questions for you
I did not want to decide these unilaterally:
enabledfilter" mean the shipped flag only, or shouldinstall.shalso honour a user's~/.codeman/clis.jsondisables? This does the shipped flag only.CliDiscovery.installgrownpmFlags/companionPackagesso pi and deepseek stop being Dockerfile special cases? No for now, pinned with a test that demands a written reason for each.server.Dockerfilebe wired to the catalogue (which would add pi/deepseek to the Compose image) or keep a deliberately narrower list?Not in this PR
The frontend half — injecting
window.__codemanCliCatalogand makingmobile-overview.js/session-ui.jscatalogue-driven — is held back as PR B2. Six of the thirteen open PRs touch those three files; splitting keeps this one conflict-free.Separately,
dsh_banner_probeininstall.shaborts on macOS bash 3.2 (local runner=()then"${runner[@]}"underset -u). It predates this PR and is reported rather than fixed here, to keep the diff reviewable.Commits
e71d954etest(install): pin install.sh's CLI detection paths before generating them3c393196feat(cli-registry): generate a CLI catalogue for install.sh and the Docker build7c242802refactor(install): drive CLI detection, the install menu and hints from the catalogue9a1c1edffeat(docker): derive the agent image's npm CLI list from the cataloguee417b94ddocs(cli-registry): document the catalogue's consumers and the trust boundary🤖 Generated with Claude Code