Harden assisted update against legacy pinned-symlink Linux installs - #896
Merged
Conversation
On ≤v0.31.x Linux installs whose ExecStart resolves through a symlink hard-pinned to a versioned dist/bun binary, the managed-update-first ordering in migration 0010 extracted the target release but restarted back into the old runtime, whose boot-time release-binary pruning then deleted the new artifact — while validations false-greened because they trusted release.json rather than the running executable. - Reorder migration 0010 for Linux legacy shapes: inspect the entrypoint first and perform the fixed-runtime cutover (stage exact checksum- verified target, preserve .previous, activate fixed path, repoint ExecStart) BEFORE the first restart; macOS keeps the launchd bridge. Adds legacy bin/dispatch symlink + unit-backup hygiene and makes rollback prefer the fixed runtime .previous over legacy unit backups. - New running_version required check that proves the actually running executable via the X-Dispatch-Version header; the assisted framework enforces it implicitly at launch and at check-run time (manifests can't name it yet — pre-v0.33 parsers reject unknown check names). - expected_runtime_artifact now uses lstat and rejects a symlinked fixed runtime path. - Manifest requiredChecks schema accepts unknown well-formed names so future check additions can't silently drop a whole migration on older installs; unknown names fail closed at run time instead. A test guards that shipped manifests stay within the legacy-parser-safe set. - VM fixture script (scripts/vm-fixtures/legacy-pinned-symlink.sh) that converts a healthy Linux install into the pinned-symlink legacy shape for the documented VM validation row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On a legacy host both fixed-runtime-entrypoint (0010) and agent-restart-safety (0011) are pending and run in numeric order, so 0010's new cutover restart happened before 0011 added KillMode=process — terminating the tmux-backed assisted agent inside dispatch.service's control group at exactly that first restart. 0010 now adds KillMode=process + daemon-reload as a required pre-restart step (the only permitted unit change besides the ExecStart repoint), intentionally front-running 0011. 0011's alreadySatisfied is now config-based (systemctl show -p KillMode, no behavioral restart probe), non-systemd hosts are an explicit no-op terminal state, and its instructions acknowledge 0010 may have already applied the setting. Review feedback from the Ubuntu VM validation agent on PR #896. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
python3 was first invoked after the fixture had already pinned the symlink, rewritten the unit, and removed the fixed runtime — a VM without Python would be stranded half-converted with migrations still marked applied. Preflight python3/systemctl/sed/ln/cp and the user systemd manager before touching anything, and make the applied-migrations rewrite atomic (tmp + os.replace). infra-review #592 item #1262. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Architecture review flagged two literal-reading hazards: the Linux staging/activation steps were phrased unconditionally, so an already-fixed Linux entrypoint could be steered into a manual cutover that bypasses the managed update; and the platform-generic .previous preservation step, read after the Linux cutover, would overwrite the run's only rollback artifact with the target binary. Instructions are now explicit branches: Branch A (Linux already-fixed: managed update only), Branch B (Linux legacy pinned: staged cutover before first restart, with "never overwrite the .previous this step created"), Branch C (macOS bridge, including the deleted-inode .previous recovery and plist work), plus shared restart-safety, hygiene, and verification steps keyed to their branches. architecture-review #593 items #1263 #1264. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds idea
legacy-linux-pinned-symlink-update-hardeningfrom the shared brain inbox, prompted by a real Cloudy Linux update failure after v0.32.2 went stable.The failure
On ≤v0.31.x Linux installs whose
ExecStartresolves through a symlink hard-pinned to a versioned binary (e.g.bin/dispatch → dist/bun/dispatch-0.31.4-bun-linux-x64), migration 0010's managed-update-first ordering extracts the target release, writes release state, and restarts — straight back into the old pinned binary. The old runtime's boot-time release-binary pruning then deletes the freshly extracted target artifact, while every validation false-greens becauseservice_restarted/version_convergedtrust release.json andexpected_runtime_artifactaccepted the (symlinked) fixed path.Changes
Migration 0010 reorder (the piece that reaches the legacy population — manifest text ships in the target tarball and is executed verbatim by old installs' assisted agents):
.previous, hardlink/copy to the fixed path as a regular file, repointExecStart, daemon-reload, then restart. macOS keeps the launchd bridge (managed update first).bin/dispatchsymlinks and refresh unit backups so a restore can't reintroduce the legacy entrypoint..previousover pre-update unit backups that name the legacy bin path./proc/<MainPID>/exe), not release.json alone.New
running_versionrequired check (framework-side):X-Dispatch-Versionresponse header — which is baked into the binary at build time — against the target tag. No header, wrong version, or unreachable ⇒ fail.requiredChecks: pre-v0.33 runtimes parse manifests with a strict check-name enum and would silently drop the whole manifest on an unknown name. A new test guards that shipped manifests stay within the legacy-parser-safe set.Check hardening:
expected_runtime_artifactnow useslstatand rejects a symlinked fixed runtime path (the exact legacy shape that false-greened).requiredChecksschema accepts unknown well-formed names at parse time; unknown names fail closed at run time (unknown required check) instead of erasing the migration from the pending set. This removes the trap for all future check additions.VM fixture (idea item 3):
scripts/vm-fixtures/legacy-pinned-symlink.shconverts a healthy fixed-path Linux install into the ≤v0.31.x pinned-symlink shape (pinnedbin/dispatchsymlink, no fixed runtime file, noKillMode=process, fixed-runtime migrations un-applied), wired into the docs' "Existing legacy Linux service" procedure. Per repo policy, no VM was provisioned or run for this PR — say the word if you want the VM row exercised before this ships in a release.Validation
pnpm run check✅pnpm run test✅ (718 tests, includes new coverage forrunning_version, symlink rejection, fail-closed unknown checks, implicit enforcement, and shipped-manifest parser compatibility)pnpm run test:e2e✅ (176 passed, 12 skipped)apps/webchanges.🤖 Generated with Claude Code