Skip to content

feat(cli-registry): drive install.sh and the Docker agent image from the CLI catalogue - #380

Open
opticon454 wants to merge 5 commits into
Ark0N:masterfrom
opticon454:feature/cli-catalog-consumers
Open

feat(cli-registry): drive install.sh and the Docker agent image from the CLI catalogue#380
opticon454 wants to merge 5 commits into
Ark0N:masterfrom
opticon454:feature/cli-catalog-consumers

Conversation

@opticon454

@opticon454opticon454 commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

What this does

PR A made every run mode a data-driven CliEntry, but stopped at the server boundary. Everything outside the Node process still hand-maintains its own CLI list, and those lists have already drifted in ways that shipped:

  • b6d0f1fa"wire OMP into install.sh's CLI detection (it had none)". A user with only omp installed was told no AI CLI was found, and offered Claude Code.
  • install.sh carried nine *_SEARCH_PATHS arrays, ~18 near-duplicate check_* / get_*_path functions, and three separate hand-written enumerations of all nine CLIs.
  • The section comment above that code named six of the nine. The comment describing the drift had itself drifted.
  • docker/agent.Dockerfile hardcoded its npm list, and the index.html hint describing the image omitted omp.

After this, adding a CLI to stock.ts and running npm run generate:cli-catalog wires it into the installer's detection, its install menu, its closing reminder, and the agent image's npm layer.

This is the second of the three PRs you staged: "install.sh + docker image reading the stock JSON, with the bash-3.2 fix and an enabled filter."

Shape

Neither consumer can import TypeScript, so the generator emits two committed artifacts:

ArtifactConsumerWhy not fetched
config/clis.stock.jsonthe Docker build, build-agent-image.mjsa .mjs cannot import the TS registry
a marked block in install.shthe installer itselfit runs via curl-pipe-bash before any checkout exists

npm run generate:cli-catalog writes both; --check fails on drift and runs in CI, and test/cli-catalog-sync.test.ts fails if either goes stale.

The embedded copy is the full catalogue. An earlier attempt fetched it with a hardcoded two-CLI fallback, which degrades silently on an empty response; there is no degraded mode to fall into now. An optional, opt-in refresh (CODEMAN_CLI_CATALOGUE_URL / CODEMAN_REFRESH_CLI_CATALOGUE=1) warns loudly on all three failure shapes.

Trust boundary, made mechanical

The server still never executes an entry's install command. install.sh executes only commands embedded in itself — same file, same TLS fetch, same commit as the one-liner that fetched it — and nothing pulled from the network at install time is ever run.

That is now structural rather than a convention: CLI_INSTALL_CMD_TRUSTED[] is written only by the generated block and is the array the menu executes; CLI_INSTALL_CMD_DISPLAY[] is what a refresh may overwrite and is display-only. test/install-sh-invariants.test.ts asserts the refresh function cannot write the executable array, and that it contains no eval.

The enabled filter

Your §3 finding: the earlier export carried no enabled field, so a CLI shipping disabled would still be baked into every image. The generated catalogue carries the flag and the image respects it.

It reads the stock catalogue, not the merged registry, so a user's ~/.codeman/clis.json cannot change what is inside an image tagged codeman/agent:base.

Every stock entry is enabled today, so that assertion would pass vacuously — a unit test feeds the pure helper a fabricated disabled entry so the fix is genuinely covered now rather than the first time someone ships one.

bash 3.2

macOS ships bash 3.2 and the documented install is a curl pipe into bash under set -euo pipefail, so no declare -A, mapfile, namerefs, ${x,,} or here-strings. The catalogue is parallel indexed arrays with offset/length windows into flat arrays rather than delimiters — safe for a $HOME with spaces, and a length-0 entry (the shell pseudo-CLI) is never iterated.

bash -n cannot see the failure mode that actually bites here: expanding an empty array under set -u is a runtime abort. So CI does both — bash -n, and executing the script inside a real bash:3.2 container with detection exercised end to end. That needed a sourcing guard (CODEMAN_INSTALL_SH_LIB) so the library half can be sourced without running the installer.

Behaviour changes, all in the installer

These are deliberate and worth your eye:

  1. The install menu now offers five CLIs rather than two. It is built from the catalogue, so it lists every enabled CLI that is not installed and ships an install command. Gemini had a command in the registry and appeared in no list in the script at all.
  2. Menu entries use the registry's labels — "Claude" rather than "Claude Code". Same trade already made for codeman doctor rows. A suffix map would just be the hand-maintained list again.
  3. On a wget-only host the menu prints the commands instead of running them. The registry's commands call curl, whereas the two literals they replace went through download_to_stdout. Rewriting curl to wget inside a string about to be executed is the wrong instinct.
  4. CODEMAN_NONINTERACTIVE=1 still defaults to Claude Code, unchanged.

docker/server.Dockerfile is deliberately untouched

#377 is still open and modifies it (#373 has since merged). Rather than edit a file another PR owns, its narrower CLI list is asserted as a declared omission list in test/docker-agent-image-coverage.test.ts, so the divergence is visible and reviewable without touching the file. Adding a CLI there, or changing the intent, now fails a test and forces the list to be restated.

Tests

Written in the order that makes each one non-vacuous, and each guard was proved by perturbing both sides.

FileWhat it pins
test/install-sh-detection-parity.test.tsWritten before deleting anything. The nine search-path arrays as literals, then that the registry's searchDirs × binaries reproduces them dir-major. They are not uniform (claude has ~/.claude/local, opencode has ~/go/bin and ~/.bun/bin), so silent narrowing means a user's installed CLI stops being found. Retargeted at the generated block afterwards.
test/cli-catalog-sync.test.tsBoth artifacts match stock.ts
test/install-sh-invariants.test.tsTrust boundary, no eval in the refresh path, and no leftover per-CLI detection names — derived from CLI_IDS/CLI_ALL_BINS rather than an allowlist, which had already missed two
test/docker-agent-image-coverage.test.tsEvery enabled entry reaches the image via the arg or a special case carrying a written reason; no disabled entry does; a hostile package name is refused; the index.html hint names every binary
test/agent-image-build-args-parity.test.tsThe .mjs and the TS mirror produce the same packages, pairs and argv — two producers of one command line is exactly the shape that drifts, and the failure would be two different images under one tag

Verification: static gate green; bash -n clean; executed under a real bash:3.2.57 container; verified the Dockerfile ARG really word-splits (4 packages gives ARGC=4) and that the default still applies with no arg. Full suite diffed by test name against the master baseline — zero new failures.

Open questions for you

I did not want to decide these unilaterally:

  1. Does "an enabled filter" mean the shipped flag only, or should install.sh also honour a user's ~/.codeman/clis.json disables? This does the shipped flag only.
  2. Refresh opt-in (as here), or attempted by default with loud warnings?
  3. Should CliDiscovery.install grow npmFlags / companionPackages so pi and deepseek stop being Dockerfile special cases? No for now, pinned with a test that demands a written reason for each.
  4. Once fix(docker): bind-mount ownership, Compose override discovery, and the default runtime account #377 lands, should server.Dockerfile be wired to the catalogue (which would add pi/deepseek to the Compose image) or keep a deliberately narrower list?
  5. shellcheck in CI now, or a follow-up issue?

Not in this PR

The frontend half — injecting window.__codemanCliCatalog and making mobile-overview.js / session-ui.js catalogue-driven — is held back as PR B2. Six of the thirteen open PRs touch those three files; splitting keeps this one conflict-free.

Separately, dsh_banner_probe in install.sh aborts on macOS bash 3.2 (local runner=() then "${runner[@]}" under set -u). It predates this PR and is reported rather than fixed here, to keep the diff reviewable.

Commits

  • e71d954e test(install): pin install.sh's CLI detection paths before generating them
  • 3c393196 feat(cli-registry): generate a CLI catalogue for install.sh and the Docker build
  • 7c242802 refactor(install): drive CLI detection, the install menu and hints from the catalogue
  • 9a1c1edf feat(docker): derive the agent image's npm CLI list from the catalogue
  • e417b94d docs(cli-registry): document the catalogue's consumers and the trust boundary

🤖 Generated with Claude Code

opticon454and others added 5 commits September 5, 2026 03:24
… them
PR B replaces nine hand-written `*_SEARCH_PATHS` arrays in install.sh with one
block generated from `STOCK_CLIS`. This lands FIRST, against the hand-written
arrays, so the replacement has something to be measured against.
The arrays are not uniform, which is why "generate them from the registry" is a
claim rather than an obvious truth: claude alone has `~/.claude/local`, opencode
alone has `~/go/bin`, opencode/codex/gemini/pi/omp carry `~/.bun/bin` while
dsh/grok/agy do not, and omp's `~/.omp/bin` sits second rather than first. A
generated list that silently narrows leaves a user with that CLI installed being
told no AI CLI was found — upstream `b6d0f1fa` is that bug, fixed for omp by
hand after it shipped.
The test asserts a three-way identity: the pinned literals equal what install.sh
contains today, AND equal `searchDirs x binaries` from the registry, dir-major so
the probe ORDER is pinned too and not just the set. Both halves were verified to
fail independently — dropping one path from install.sh fails the first, changing
one `searchDirs` entry fails the second — because a pin that cannot fail is
worse than no pin. A fourth case asserts every stock CLI with a binary is
covered, which is the omp bug restated so it cannot recur silently.
No production code changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
…ocker build
Two consumers of the registry cannot import TypeScript: `install.sh`, which runs
via `curl | bash` before any checkout exists, and `scripts/build-agent-image.mjs`.
Both currently hand-maintain their own CLI lists, and both have already drifted.
`scripts/generate-cli-catalog.mts` (`npm run generate:cli-catalog`, plus a
`--check` mode) emits from `STOCK_CLIS`:
- `config/clis.stock.json` for the `.mjs` and the tests. It carries `enabled` —
the field the earlier attempt omitted, which is how a disabled CLI's npm
package still got baked into every agent image.
- a marker-delimited block inside `install.sh`, embedded rather than fetched.
The embedded copy is the FULL catalogue on purpose: the earlier design fetched
it and fell back to a hardcoded two-CLI list, degrading silently on an empty
response. There is no degraded mode to fall into now.
The block is bash 3.2 safe: parallel indexed arrays, no associative arrays, no
namerefs, no mapfile. Variable-length lists use OFFSET/LENGTH windows into one
flat array rather than a delimiter, so a $HOME containing a space needs no IFS
handling and `shell` (no binaries) gets length 0 and is never iterated. Search
paths are emitted dir-major, matching the probe order the hand-written arrays
use and `test/install-sh-detection-parity.test.ts` pins.
Only fields the two consumers need are exported. `launch`/`env`/`capabilities`/
`overlays` are spawn-time concerns the server alone interprets, and a test
asserts they never leak into the artifact.
`main()` sits behind an `isMainModule()` guard so the sync test can import the
renderers. Without it, importing the module would rewrite the artifacts as a
side effect of checking them — passing always, guarding never.
This commit adds the block; it does not yet delete the hand-written arrays, so
the detection pin keeps measuring both against each other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
…om the catalogue
install.sh carried nine search-path arrays, eighteen near-identical
check_<cli>/get_<cli>_path functions, and three separately hand-maintained
enumerations of all nine CLIs. They had to agree and did not: upstream b6d0f1f
is "wire OMP into install.sh's CLI detection (it had none)", and the section
comment above the roll-call named six of the nine.
All of it now reads the generated catalogue. `detect_all_clis` resolves every
CLI in one memoized pass into CLI_FOUND_PATH/CLI_FOUND_COUNT; `check_cli` and
`get_cli_path` replace the eighteen pairs; the roll-call, the "no AI CLI found"
gate and the closing reminder become loops. Probe order per CLI is unchanged and
`test/install-sh-detection-parity.test.ts` proves it against the literals
transcribed from the arrays this deletes.
Behaviour changes worth naming:
- The install menu is built from the catalogue, so it offers every enabled CLI
that is not installed and ships a command — five instead of two. Gemini had a
command in the registry and appeared in NO list in this script.
- Its labels are now the registry's ("Claude" rather than "Claude Code"), the
same trade PR A made for `codeman doctor` rows. A suffix map would just be the
hand-maintained list again.
- On a wget-only host the menu prints commands instead of running them. The
registry's commands call curl, whereas the two literals this replaces went
through download_to_stdout; rewriting curl to wget inside a string we are
about to execute is the wrong instinct.
The trust boundary is mechanical, not a promise: CLI_INSTALL_CMD_TRUSTED is
written only from the generated per-platform arrays and is the only thing ever
executed; CLI_INSTALL_CMD_DISPLAY is what the optional, opt-in refresh may
rewrite. The refresh warns on all three failure shapes — empty body, unparseable
content, failed fetch — which is the silent-degradation bug from the review, and
it parses with node into tab-separated records read by `read`, never eval.
Bash 3.2 throughout (macOS ships it): parallel indexed arrays, offset/length
windows instead of delimiters, no associative arrays, namerefs, mapfile or
here-strings. Verified by executing the script under a real bash 3.2 container,
which is also now a CI step alongside `bash -n` and a catalogue `--check` — the
empty-window case (`shell` has no binaries) is a runtime `set -u` abort that
`bash -n` cannot see. Running it that way caught `detect_os` being called inside
the platform loop: ten forks, and ten copies of one error, since a `die` inside
`$( )` can only exit the subshell.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
docker/agent.Dockerfile hardcoded the four npm-published CLIs it installs, one
of the several lists that had to be kept in step with the registry by hand.
It now takes them as `ARG CLI_NPM_PACKAGES`, supplied by
scripts/build-agent-image.mjs from config/clis.stock.json, with the default set
to today's list so a bare `docker build` still produces the same image. The arg
is expanded unquoted because word splitting is what turns the list into several
arguments, which is exactly why every token is validated against
^[@A-Za-z0-9][@A-Za-z0-9/._-]*$ on the producing side; a package name carrying a
space or a metacharacter is refused rather than reaching the RUN line. Verified
by building the layer: four packages in, four arguments out, and the default
still applies with no arg.
The list is filtered on each entry's `enabled` flag — the field whose absence
was the maintainer's §3 finding, where a CLI shipping disabled still got baked
into every image. No stock entry is disabled today, so that assertion would pass
vacuously; a unit test feeds the pure helper a fabricated disabled entry so the
fix is covered now rather than the first time someone ships one.
⚠️ It reads the STOCK catalogue, never the merged registry. A user's
~/.codeman/clis.json must not change what is inside an image tagged
codeman/agent:base, or two machines holding that tag hold different images.
Four CLIs keep hand-written layers because the registry cannot describe what
makes them special: pi's --ignore-scripts, deepseek's pnpm companion and dsh-tui
profile, and the three standalone installers. Rather than extend the schema for
a Docker-only benefit, the coverage test requires each to carry a written reason
AND still be present, so an exclusion cannot quietly become an omission.
There are two producers of this command line and there have to be — the .mjs
cannot import TypeScript, and src/docker-hosts.ts builds the same argv for the
in-app auto-build — so a parity test pins them together, package list, arg pairs
and rendered argv. Their order is pinned too: a different order is a different
RUN string and so a needless cache miss between the two build paths.
docker/server.Dockerfile is deliberately NOT edited (PRs Ark0N#373 and Ark0N#377 both
modify it); its narrower list is asserted as a declared omission list instead, so
the divergence is reviewable without touching the file.
Also fixes the in-app hint at index.html, which the new coverage test caught
still omitting omp.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
…boundary
Adds a "Consumers outside the server" section covering the two generated
artifacts, why each exists (neither install.sh nor a .mjs can import
TypeScript), what is deliberately NOT exported and why, the three-rule install
command trust boundary, and the bash 3.2 constraint with the offset/length
window shape it forces.
The adding-a-CLI checklist gains the regenerate step, since forgetting it is how
the installer would keep detecting the old set while the server offers the new
one — the drift this change removes, one level out.
docs/docker-cases.md gains how CLI_NPM_PACKAGES is derived, why it reads the
stock catalogue and not the merged registry, and a table of the four documented
Dockerfile special cases with their reasons. CLAUDE.md gains a command row and
names the generated block, the bash 3.2 rule and the trust boundary in its
install.sh paragraph.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@opticon454