fix(docker): bind-mount ownership, Compose override discovery, and the default runtime account - #377
fix(docker): bind-mount ownership, Compose override discovery, and the default runtime account#377opticon454 wants to merge 6 commits into
Conversation
57abc1a to
b698b8eComparedocker/agent.Dockerfile hardcoded the four npm-published CLIs it installs, one of the several lists that had to be kept in step with the registry by hand. It now takes them as `ARG CLI_NPM_PACKAGES`, supplied by scripts/build-agent-image.mjs from config/clis.stock.json, with the default set to today's list so a bare `docker build` still produces the same image. The arg is expanded unquoted because word splitting is what turns the list into several arguments, which is exactly why every token is validated against ^[@A-Za-z0-9][@A-Za-z0-9/._-]*$ on the producing side; a package name carrying a space or a metacharacter is refused rather than reaching the RUN line. Verified by building the layer: four packages in, four arguments out, and the default still applies with no arg. The list is filtered on each entry's `enabled` flag — the field whose absence was the maintainer's §3 finding, where a CLI shipping disabled still got baked into every image. No stock entry is disabled today, so that assertion would pass vacuously; a unit test feeds the pure helper a fabricated disabled entry so the fix is covered now rather than the first time someone ships one.⚠️ It reads the STOCK catalogue, never the merged registry. A user's ~/.codeman/clis.json must not change what is inside an image tagged codeman/agent:base, or two machines holding that tag hold different images. Four CLIs keep hand-written layers because the registry cannot describe what makes them special: pi's --ignore-scripts, deepseek's pnpm companion and dsh-tui profile, and the three standalone installers. Rather than extend the schema for a Docker-only benefit, the coverage test requires each to carry a written reason AND still be present, so an exclusion cannot quietly become an omission. There are two producers of this command line and there have to be — the .mjs cannot import TypeScript, and src/docker-hosts.ts builds the same argv for the in-app auto-build — so a parity test pins them together, package list, arg pairs and rendered argv. Their order is pinned too: a different order is a different RUN string and so a needless cache miss between the two build paths. docker/server.Dockerfile is deliberately NOT edited (PRs Ark0N#373 and Ark0N#377 both modify it); its narrower list is asserted as a declared omission list instead, so the divergence is reviewable without touching the file. Also fixes the in-app hint at index.html, which the new coverage test caught still omitting omp. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015EMxQreQUZX5ZyybxAGh12
GitHub's conflict flag traced down to two real conflicts, both caused by #373 (
|
Compose binds CODEMAN_APPDATA_PATH and CODEMAN_CASES_PATH from the host. When either path does not exist yet - a first run, a cleared application-data directory, a restored backup - the Docker daemon creates it owned by root. The server runs unprivileged as CODEMAN_RUNTIME_USER, so it cannot create its own state directory, and the container restarts forever on: Failed to start web server: EACCES: permission denied, mkdir '/home/<user>/.codeman' Start-Codeman.sh already worked around this by preparing the directory on the host, so the failure only appears when Compose is run directly, which the README documents as a supported path. Add docker/entrypoint.sh, which starts as root, corrects the ownership of both bind mounts, then drops to PUID:PGID with setpriv. The Dockerfile's USER instruction is replaced by that entrypoint and CMD is unchanged. docker-compose.yaml adds back only the four capabilities the chown and the privilege drop require, so cap_drop: ALL continues to remove everything else. Two guards keep existing deployments working: - A container started with an explicit `user:` is left alone. The entrypoint execs straight through, with no elevation and no chown. - A chown that fails is a warning, not an error. Bind mounts backed by NFS, CIFS or a rootless daemon can refuse chown while remaining perfectly writable, and those deployments must keep starting. PUID and PGID are also exported as runtime environment defaults so the image behaves correctly when run without Compose, rather than depending on build args alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Naming a Compose file with -f disables Compose's automatic discovery of the override file, so Start-Codeman.sh silently ignored docker-compose.override.yml. Any local customisation placed in the conventional override file was dropped without warning, and the only way to notice was to inspect the running container. Collect the -f arguments into an array, append the override file when one is present, and reuse that array for the final launch so the two cannot drift apart again. Both .yml and .yaml are checked, in Compose's own precedence order, and the chosen file is reported on startup. Document the override file in docker/README.md, including the two things that are easy to get wrong: it is ignored when -f is passed without naming it, and it cannot remove a key such as ports, which Compose concatenates. Add the override file to .gitignore. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CODEMAN_RUNTIME_USER defaulted to `opencode`, which no longer matches the project and is confusing in a deployment whose every other identifier is codeman. Rename the default in .env.example and in the Dockerfile ARG that mirrors it, and correct the example comment that referred to /home/opencode/codeman-cases. Also drop the `Coding/` component from the example application-data path. CODEMAN_APPDATA_PATH and CODEMAN_CASES_PATH now suggest /mnt/user/appdata/codeman and its codeman-cases child, matching the account name and removing a directory level that meant nothing outside the original author's host. README.md is updated to match, including the chown example. The npm package `opencode-ai` and the references to the OpenCode CLI are deliberately left alone: those name a different tool, not this account. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CODEMAN_ALLOWED_HOSTS is a real, documented application setting (the Host- header allowlist in network-auth-policy.ts), but docker-compose.yaml does not forward it from .env into the container - Compose only passes through variables explicitly listed under environment:, and this is not one of them. Set without that passthrough, any request through a reverse proxy is rejected with 403 Forbidden: host not allowed before it reaches any handler, and nothing in the Docker deployment docs said why. Document the variable and the override needed to forward it, using the Local customisation mechanism already described above it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
40d3bce to
0affc10CompareRebased this branch onto current
CI is green on the rebased head and GitHub now reports this as cleanly mergeable. 🤖 Generated with Claude Code |
The four CLIs (claude, gemini, codex, opencode) are npm-installed globally as root during the image build, before the unprivileged runtime account exists. A session running as that account (e.g. a codex-mode terminal) then hits EACCES the moment it tries to update one in place, because npm renames the old package directory aside before installing the new one, which needs write access to the parent (/usr/local/lib/node_modules), not just the target package. Chown that tree plus /usr/local/bin's CLI symlinks to PUID:PGID in the same step that creates/renames the runtime account. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R9ZSTEenc8soSu9bTi8Xru
codeman-node-modules and codeman-dist (docker-compose.yaml) are seeded from the image only while empty, so a rebuilt image's fresh dist/ node_modules sat unused behind old volume content until something cleared it. The in-app self-updater never hit this (it rebuilds INSIDE the running container, into the very volume already in use), but a `docker compose build` triggered from outside it — Start-Codeman.sh, after a manual `git pull` — did: the container came back up looking unchanged, serving stale compiled routes against current source. Start-Codeman.sh now compares the checkout's HEAD commit and package-lock.json hash against a recorded marker (docker-build-source.json) and clears just the affected volume(s) before its own --build when either moved. The in-place self-update path writes that same marker after a successful build, so the two mechanisms agree on what the volumes currently reflect — without it, the next plain Start-Codeman.sh run would see the HEAD self-update just checked out, not recognise it as already accounted for, and wipe the volumes self-update just correctly rebuilt right back to the older baked image. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R9ZSTEenc8soSu9bTi8Xru
Problem
Two independent bugs in the Docker deployment.
1. The container crash-loops whenever a bind source does not already exist
docker-compose.yamlbindsCODEMAN_APPDATA_PATHandCODEMAN_CASES_PATHfrom the host. When either path does not exist yet - a first run, a cleared application-data directory, a restored backup - the Docker daemon creates it owned byroot:root. The server runs unprivileged asCODEMAN_RUNTIME_USER, so it cannot create its own state directory, and the container restarts forever on:Start-Codeman.shworks around this by preparing the directory on the host first, so the failure only appears when Compose is run directly - whichdocker/README.mddocuments as a supported path.2.
Start-Codeman.shsilently discardsdocker-compose.override.ymlThe script passes
-f "$compose_file", and naming a Compose file explicitly disables Compose's automatic discovery of the override file. Any customisation placed in the conventional override file is dropped without warning, and the only way to notice is to inspect the running container.3. The default runtime account is named
opencodeCODEMAN_RUNTIME_USERdefaults toopencode, which no longer matches the project and is confusing in a deployment whose every other identifier iscodeman. The example application-data path also carries aCoding/component that means nothing outside the original author's host.Changes
docker/entrypoint.sh(new) - starts as root, corrects the ownership of both bind mounts, then drops toPUID:PGIDwithsetpriv.docker/server.Dockerfile- theUSERinstruction is replaced by that entrypoint;CMDis unchanged.PUID/PGIDare also exported as runtimeENVdefaults so the image behaves correctly when run without Compose, rather than depending on build args alone.docker/docker-compose.yaml- adds back only the four capabilities the chown and the privilege drop require, socap_drop: ALLcontinues to remove everything else.docker/Start-Codeman.sh- collects the-farguments into an array, appends the override file when present, and reuses that array for the final launch so the two cannot drift apart again. Both.ymland.yamlare checked, in Compose's own precedence order, and the chosen file is reported on startup.docker/.env.example,docker/server.Dockerfile- rename the default runtime account tocodeman, in both the example file and theARGthat mirrors it, and correct the comment that referred to/home/opencode/codeman-cases. Simplify the example paths to/mnt/user/appdata/codemanand itscodeman-caseschild. The npm packageopencode-aiand the references to the OpenCode CLI are deliberately untouched, as those name a different tool.docker/README.md,.gitignore- document the override file, including the two things that are easy to get wrong (it is ignored when-fis passed without naming it, and it cannot remove a key such asports, which Compose concatenates), and ignore it in Git.4.
CODEMAN_ALLOWED_HOSTSis documented nowhere in the Docker deployment docsThe variable is a real, working application setting (the Host-header allowlist in
network-auth-policy.ts), but nothing indocker/README.mdmentions it, anddocker-compose.yamldoes not forward it from.envinto the container - Compose only passes through variables explicitly listed underenvironment:. A reverse-proxied deployment fails with403 Forbidden: host not allowedwith no pointer back to the fix.5. Build-artefact volumes go stale after an externally-triggered rebuild
codeman-node-modulesandcodeman-dist(docker-compose.yaml) are seeded from the image only while empty. The in-app self-updater never hits this - it rebuilds INSIDE the running container, into the very volume already in use - but adocker compose buildtriggered from outside it (Start-Codeman.sh, after a manualgit pull) does: the container comes back up looking unchanged, serving stale compiled routes against current source. This bit a real deployment during this work: a source fix landed,docker compose build && upran cleanly, and the old behaviour persisted until the volume was cleared by hand.Compatibility
Two guards keep existing deployments working:
A container started with an explicit
user:is left alone entirely. The entrypoint execs straight through, with no elevation and no chown.A
chownthat fails is a warning, not an error. Bind mounts backed by NFS, CIFS or a rootless daemon can refusechownwhile remaining perfectly writable, and those deployments must keep starting.docker/README.md- documentsCODEMAN_ALLOWED_HOSTS, whydocker-compose.yamldoes not forward it, and the override needed to do so, using theLocal customisationmechanism already described earlier in the same file.docker/Start-Codeman.sh,scripts/self-update.sh,docs/docker-self-update.md-Start-Codeman.shcompares the checkout's HEAD commit andpackage-lock.jsonhash against a recorded marker (docker-build-source.json) and clears just the affected volume(s) before its own--buildwhen either moved. The in-place self-update path writes that same marker after a successful build, so the two mechanisms agree on what the volumes currently reflect - without it, the next plainStart-Codeman.shrun would see the HEAD self-update just checked out, not recognise it as already accounted for, and wipe the volumes self-update just correctly rebuilt right back to the older baked image.Testing
Verified against a full rebuild on Docker 29 / Compose v5.5, with
PUID=99andPGID=100:corrected ownership of ... to 99:100, container healthyroot:rootUid 99/99/99/99,Gid 100/100/100/100,Groups: 281- the socket group is kept and root's group is dropped-u 1000:1000uid=1000, straight exec, no elevation attempteduid=99- does not abortStart-Codeman.shwith an override present-fflags passedStart-Codeman.shwith no override-fflag, no stray argumentStart-Codeman.shfor real against the live Unraid deployment: correctly detected the source change on first run with no prior marker, resolved the real prefixed volume names viadocker volume ls --filter label=com.docker.compose.volume=..., cleared both volumes, rebuilt, and came back up healthy; marker file confirmed written with the real HEAD commit and lockfile hash afterwardThe five commits are independent; happy to split them into separate PRs if you would prefer that.
🤖 Generated with Claude Code
Folded in from #384 (opened, then closed as superseded by this consolidation once it turned out this identity has only read access to this repo and cannot merge PRs here directly).