Skip to content

Release develop → main - #383

Merged
saadqbal merged 33 commits into
mainfrom
develop
Jul 27, 2026
Merged

Release develop → main#383
saadqbal merged 33 commits into
mainfrom
develop

Conversation

@shujaatTracebloc

@shujaatTraceblocshujaatTracebloc commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Release: promote developmain

Promoting 11 commits from develop to production (main). No staging branch in this repo, so develop promotes straight to prod.

Commits being promoted

Summary

Mostly installer hardening (privilege/capability probing, tiered Docker reuse, root/sudo-aware behavior, PATH-aware CTA, copy catalog), two Helm chart updates (post-upgrade metadata backfill hook, prod ingestor digest pin v0.7.5), and installer UX polish.


Note

Medium Risk
Prod edges will adopt images.ingestor.prodDigest on auto-upgrade (ingestion image behavior fleet-wide), and the new storage seal-check can fail helm test or require cluster-scoped RBAC when clusterScope=true; mitigated by helm unit tests and the strengthened multi-arch CI gate.

Overview
Release promotion bundling client chart 1.9.6 and the accumulated develop work (installer tiering/probing, copy catalog, metadata backfill hook, digest refresh, and related fixes referenced in the release notes).

Prod ingestor reproducibility (backend#1245). The fleet-wide prod pin moves from deleted client/values-prod.yaml into chart defaults as images.ingestor.prodDigest, gated by CLIENT_ENV and prodPin through new tracebloc.ingestorDigest. jobs-manager now sets INGESTOR_IMAGE_DIGEST from that helper; dev/staging keep floating on images.ingestor.tag. CI ingestor-multiarchrequires a non-empty, multi-arch prodDigest. client/MIGRATION.md documents clearing a legacy hand-layered images.ingestor.digest if needed.

Seal check (RFC-0003 §8.2). Adds tracebloc.sealCheckLabels on egress and backend-reachability test hooks, a new storage-assertionshelm test Job (PVC bound/class + dynamic-mode hostPath checks, optional cluster-scoped PV RBAC), sealCheck values/schema, and docs/SEAL-CHECK.md (linked from SECURITY.md).

Installer & supply chain.scripts/check-style.sh drops emoji policing, adds an interim bare curl rule (push toward curl_secure()), and gen-manifest.sh includes scripts/lib/probe.sh. STYLE.md and README note emoji/reinstall data hygiene (#376). New .cursor/BUGBOT.md and PR-template guidance for cross-repo issue links.

Docs. Rootless-Docker spike RFC (docs/rfcs/0001-rootless-spike.md); ingestor hook comments aligned with prod-default pinning behavior.

Reviewed by Cursor Bugbot for commit 37817c8. Bugbot is set up for automated code reviews on this repo. Configure here.

LukasWodkaand others added 11 commits July 21, 2026 20:45
From Lukas's review: emoji are wanted, not stripped.
- Guard's emoji check removed (now enforces hardcoded brand colour + 'workspace'
only); STYLE.md documents emoji as welcome (👋 💚 🚀 🟢/🟡/🔴 ⚠).
- summary.sh: restore the 🟢 traffic-light on the "secure environment is live"
and "🟢 Online" lines (the earlier ● replacement is reverted).
- Manifest regenerated (R8) for the summary.sh change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…71) (#370)
* feat(install): host capability/privilege probe + audit (RFC 0001, #1171)
New scripts/lib/probe.sh: side-effect-free probes (docker info; cgroup v2 +
unprivileged userns; id/sudo/sudo-n trio) that classify the lowest-privilege
install tier (0/1/2), plus render_host_audit. Wired read-only into --diagnose
(a console panel + a plain readout in the support bundle); tier ROUTING is the
follow-up (#1172), so nothing in the install path changes behaviour yet.
Registered probe.sh in both FILES arrays (install.sh + gen-manifest.sh) and
regenerated the R8 manifest. 15 bats tests; full suite (318) green; shellcheck
--severity=warning clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(install): add probe.sh to the bootstrap fixture + manifest lists
Foundation added probe.sh to install.sh FILES; install-bootstrap.bats hard-codes
the fetched-file list in two places (staging + manifest) and must include it, or
the bootstrap happy-path tests fail with "no entry in manifest" for probe.sh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): precise non-Linux tier + wire the --verify probe (Bugbot #370)
classify: split the non-Linux case — macOS => needs-docker-desktop (correct
remedy), any OTHER non-Linux (Git Bash/MINGW) => unsupported-os with a "use
install.ps1 on Windows" note, instead of misdirecting to Docker Desktop.
run_host_probes now actually invokes _probe_verify_runtime under TB_PROBE_VERIFY=1
(it was defined + tested but never wired), so a daemon that answers `docker info`
but cannot run a container is correctly demoted from Tier 0. 2 new bats.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…72) (#374)
Wires the host audit (probe.sh, #1171) into the real install flow and adds the
routing skeleton:
- main() step a now runs host_audit, so the user sees the capability/privilege
probe + chosen tier before install, and INSTALL_TIER is set for routing.
- install_linux calls _route_install_tier first: honours the detected tier
(TB_FORCE_TIER overrides for QA) and FAILS FAST with an actionable remedy when
the host genuinely cannot run containers without admin (Tier 2 = no cgroup v2 /
unprivileged userns) AND we are neither root nor able to sudo — instead of a
cryptic mid-install crash. Tier 0/1 proceed; a stale bootstrap without probe.sh
proceeds exactly as before.
Per-tier body optimisation (Tier 0 skip privileged steps #1175, Tier 1 rootless
#1177) stacks on top. 5 new bats; full suite adds no new failures; shellcheck +
manifest clean.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…nary go/no-go (#1176) (#373)
Deliverable for the rootless-primary spike: kernel-prerequisite coverage by
distro, rootless install mechanics (subuid/subgid + systemd --user), k3d-on-
rootless-socket caveats (cgroup delegation), the fuse-overlayfs push-perf risk,
a concrete validation plan for real hosts (incl. a hardened/userns-disabled node
to exercise the Tier-2 fall-through), and a provisional GO conditioned on those
experiments. The §5 experiments need real target hosts and gate #1177 (Tier 1);
nothing here blocks #1172 (routing) or #1175 (Tier 0).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(install): Tier 0 — reuse an existing usable Docker, zero privileged steps (#1175)
When the host audit selects Tier 0 (docker info OK), install_linux now skips the
entire privileged path — sudo priming, the Docker engine install, system-package
+ kernel-module setup, and the privileged GPU-driver install — and installs only
the user-space tools (kubectl/k3d/helm) before create_cluster reuses the runtime.
The biggest unlock for shared/managed hosts: a researcher in the docker group
installs with no admin at all. The tool install is shared with the full flow via
_install_userspace_tools so they cannot drift. 3 new bats assert the branch
(Tier 0 skips every privileged step; Tier 1 / unset run the full flow). Full
suite adds no new failures; shellcheck + manifest clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): Tier 0 installs tools user-space (no sudo) — honour the zero-root promise (Bugbot #1175)
The Tier-0 fast path claimed "no administrator rights needed" but
install_kubectl/k3d/helm still sudo-wrote /usr/local/bin, so a docker-group user
without root failed (or hit a password prompt hidden under spin_cmd) at that
step. New _set_tools_target: on Tier 0 the tools install into ~/.local/bin with
NO sudo (kubectl mv without sudo; k3d + helm via their own USE_SUDO=false +
*_INSTALL_DIR knobs) and ~/.local/bin is put on PATH so create_cluster finds
them. The full-flow path (/usr/local/bin + sudo, incl. the RHEL secure_path
sudo-env-PATH handling) is unchanged. 2 new bats.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): Tier 0 default tool target + skip privileged docker autostart (Bugbot #1175 r2)
Two follow-ups on the Tier-0 zero-root path:
1. install_kubectl/k3d/helm read TB_TOOLS_DIR/TB_TOOLS_SUDO — module-level
defaults (/usr/local/bin + sudo) so direct callers (bats, e2e) that skip
_set_tools_target keep the system behaviour instead of an empty dir /
spurious no-sudo branch.
2. create_cluster -> ensure_cluster_autostart ran `sudo systemctl enable docker`
on Tier 0 too, prompting for a password on /dev/tty after the zero-privilege
promise. Skip it on Tier 0 (the k3d --restart policy already covers reboot for
the common case); the full flow is unchanged. New cluster.bats + the k3d test
is green again.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): Tier 0 helm chmod without sudo (Bugbot #1175 r3)
_ensure_helm_executable ran `sudo chmod` unconditionally; on the Tier-0 no-admin
path (helm in ~/.local/bin, user-owned) that would prompt on the tty — the same
zero-privilege break already guarded for systemctl. Use a plain chmod when
TB_TOOLS_SUDO is empty (Tier 0), sudo otherwise.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): Tier 0 reuses the NVIDIA k3d GPU flag when the runtime is configured (Bugbot #375)
The Tier-0 early return skipped dispatch_gpu_setup, which is the only place
K3D_GPU_FLAGS=(--gpus=all) was set — so an NVIDIA Tier-0 host silently got a
CPU-only k3d cluster even with the toolkit installed. New _tier0_gpu_flags sets
the flag when Docker's NVIDIA runtime is already configured (reused, no admin);
otherwise it stays CPU-only with guidance (installing the toolkit needs admin).
3 new bats.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* tier0: persist ~/.local/bin to the shell rc so new terminals find the tools
On Tier 0, kubectl/k3d/helm install into ~/.local/bin and _set_tools_target
only prepends that dir to the CURRENT process PATH. Nothing persisted it to
the user's shell rc, so a fresh terminal couldn't find the tools and the
summary's suggested `kubectl …` commands failed — worst when the CLI lives in
/usr/local/bin (already on PATH), so no other step triggered a PATH fix
either (Bugbot #375).
Add _persist_tools_on_path (called from _install_userspace_tools): when the
user-local dir was used (Tier 0), append `export PATH="$HOME/.local/bin:$PATH"`
to the shell-correct rc — idempotent (skips if the rc already references
~/.local/bin) and best-effort (never fatal). _tools_rc_for_shell mirrors the
CLI installer's per-shell rc resolution (zsh/.zshrc, bash+mac/.bash_profile,
bash+linux/.bashrc, else /.profile). No-op on the full flow (/usr/local/bin).
Tests: rc resolution per shell; Tier-0 append; idempotency; full-flow no-op.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* tier0: fix broken GPU prepare-host hint + fish PATH persistence (Bugbot #375)
Two Bugbot findings:
- Broken prepare-host GPU hint: _tier0_gpu_flags told admins to re-run with
`bash -s -- prepare-host`, but that mode doesn't exist on this branch (it
lands in the stacked prepare-host PR), so the arg was ignored and a normal
install ran — the suggested remedy couldn't enable GPU. Point instead to
installing/configuring nvidia-container-toolkit directly, which is correct
and self-contained here.
- Fish PATH persist incomplete: _persist_tools_on_path appended a bash
`export PATH=…` line to ~/.profile for fish users, which fish never loads,
so Tier-0 kubectl/k3d/helm stayed missing in new fish terminals. Detect fish
and hint the fish-correct `fish_add_path "$HOME/.local/bin"` (persists as a
universal var AND applies now) instead of writing a dead POSIX line. Note in
_tools_rc_for_shell that fish is handled separately, never routed there.
Tests: fish path emits fish_add_path and writes no ~/.profile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(tier0): sandbox HOME in the Tier-0 install_linux test (Bugbot #375)
The test ran the real _install_userspace_tools (only leaf steps are stubbed),
so _set_tools_target mkdir-ed ~/.local/bin and _persist_tools_on_path appended
a PATH export to the shell rc in the developer/CI real home. Set
HOME="$BATS_TEST_TMPDIR" like the sibling PATH/tools tests so both side effects
land in the throwaway tmpdir.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): root-aware sudo — no sudo needed when root; honest "sudo missing" vs "no access" (A2, #1173)
The installer primed and ran ~70 privileged steps as `sudo <cmd>`, which broke
two real users: a root container/VM (sudo unnecessary, and often ABSENT on
minimal images, so `sudo <cmd>` died with "command not found") and reported
"sudo not installed" as "no sudo access" (a different, misleading fix).
Shadow `sudo` with a function in common.sh: as root, run <cmd> directly (no sudo
binary needed); else defer to the real sudo; else fail the way real sudo would
(non-zero) so best-effort `sudo … || fallback` sites still fall back. Every
existing call site is fixed with NO edit — the diff is confined to common.sh.
preflight_sudo now short-circuits on root, gives an accurate message when sudo is
genuinely absent, and primes credentials via the real sudo (`_real_sudo -v`).
type -P finds the real binary past the shadow; helper seams (_have_sudo_bin,
_real_sudo) make every branch unit-testable. 6 new bats; full suite adds no new
failures; shellcheck introduces no new findings; manifest regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): export the sudo shadow so bash -c subshells inherit it (review #372)
The sudo() shadow is a shell function; without `export -f`, `sudo <cmd>` inside
setup-linux.sh's `bash -c '…'` blocks (the apt-lock wait and the RHEL-rebuild
Alma/Rocky/OL Docker install) resolved the REAL sudo binary — so a root box
without sudo installed still hit "sudo: command not found" there, the exact case
A2 eliminates everywhere else. Export sudo + _real_sudo + _have_sudo_bin.
Harmless to non-bash children (they ignore BASH_FUNC_*). New bats asserts a
child bash inherits the shadow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): silence type stderr in _have_sudo_bin (Bugbot #372)
type -P sudo prints "type: sudo: not found" to stderr when sudo is absent, which
leaked out right before preflight_sudo`'s clean "not root and sudo isn't
installed" message — undercutting exactly the accurate messaging A2 adds. Add
2>/dev/null.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* probe: detect the real sudo binary, not the A2 sudo() shadow
_probe_privilege used `has sudo` + a bare `sudo -n true`. Both resolve to
the `sudo()` shell function common.sh installs (the A2 shadow), so on a host
with no sudo at all the probe saw "sudo present" and reported the posture as
sudo_pw instead of no_sudo — misrouting the install tier (Bugbot #372).
Switch to the same primitives preflight_sudo uses: _have_sudo_bin (type -P,
ignores functions) and _real_sudo (command sudo, bypasses the shadow). Update
the three privilege bats cases to mock those helpers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* common: make _have_sudo_bin set -e-safe (drop the command substitution)
_have_sudo_bin wrapped `type -P sudo` in `[ -n "$(...)" ]`. On bash <4.4
(Amazon Linux 2's 4.2, and macOS's 3.2) a failing command substitution trips
`set -e` even when the helper is called from an `if`/`elif` — so the non-root,
no-sudo path that preflight_sudo is meant to handle with a clear message could
abort before that message ran (Bugbot #372).
Use `type -P sudo >/dev/null 2>&1` as the whole function body: no substitution,
same result (0 iff a real sudo binary is on PATH, ignoring the sudo() shadow),
still quiet. Add a regression test that runs the helper under `set -e` with no
sudo on PATH and asserts it returns cleanly instead of aborting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…380)
* feat(chart): one-time metadata backfill as a post-upgrade Helm hook
Adds a post-upgrade/post-install hook Job that runs the ingestor image's
`tracebloc-backfill` entrypoint (data-ingestors #393, backend #1166/#1198) once
per chart upgrade. The auto-upgrade CronJob runs `helm upgrade` when a new chart
publishes, which fires the hook — so a client that auto-updates to this release
runs the backfill automatically, no operator step.
The sweep brings every already-registered pre-cutover dataset up to the
federated-alignment metadata shape (enriched schema + feature_stats, recomputed
from the persisted MySQL rows) and re-folds any competition built from them, so
old competitions/experiments keep working under the new backend alignment code.
- Reuses the same credentials Secret (CLIENT_ID/CLIENT_PASSWORD) and in-cluster
MySQL (mysql-client) as jobs-manager, plus tracebloc.proxyEnv for corporate
egress; resolves the same ingestor image (repo/tag/digest) the ingestion path
spawns.
- Best-effort: retries a few times in-pod then exits 0 regardless, so a backfill
hiccup never fails the auto-upgrade (which would strand the cluster on the old
chart). Operators read the outcome via `kubectl logs`.
- One-time/idempotent: before-hook-creation replaces the prior run; the sweep
skips datasets already carrying new-shape metadata and the upsert is a safe
overwrite, so re-runs are cheap. Opt out with metadataBackfill.enabled=false.
- Hardened pod: non-root, no SA-token, read-only rootfs, caps dropped.
Tests: 7-case helm-unittest (renders by default, hook annotations, image
tag/digest modes, cred+MySQL env, hardened pod, disabled-by-flag). Full chart
suite green (274).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chart): nil-guard metadataBackfill for --reuse-values upgrades
The auto-upgrade E2E upgrades from the last published release via --reuse-values,
whose stored values predate the metadataBackfill key — so .Values.metadataBackfill
is nil and `.enabled` failed template rendering ("nil pointer evaluating
interface {}.enabled"), the same hazard the images.ingestor keys already guard.
Guard the whole block with `$mb := default (dict) .Values.metadataBackfill` and
read deadlineSeconds/resources through $mb. Absent key => feature off on the
--reuse-values path; the production auto-upgrade uses --reset-then-reuse-values,
which layers in the new default (enabled: true), so the backfill still runs on
the real upgrade path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chart): stop the backfill hook from failing the auto-upgrade (Bugbot #380)
Two Bugbot findings on the metadata-backfill hook:
- Medium: the ingestor-tag fallback was "0.5", but values.yaml and jobs-manager's
INGESTOR_IMAGE_TAG default to "0.7"; on the fallback path the Job would pull an
ingestor predating the tracebloc-backfill entrypoint and no-op. Aligned to "0.7".
- High: swallowing the entrypoint exit code didn't make the hook best-effort —
activeDeadlineSeconds (1800, and > autoUpgrade.timeout 10m), image pull/OOM, or
Helm's wait timeout still fail the Job and thus `helm upgrade`. Now the
container bounds ITS OWN runtime with `timeout $backfillTimeoutSeconds`
(default 300s, under the 10m Helm timeout) and ALWAYS exits 0, so the pod
completes successfully regardless of how the sweep goes. Removed
activeDeadlineSeconds (a deadline kill = failed Job = failed upgrade) and set
backoffLimit 0. A run that hits the bound stops cleanly and resumes on the next
upgrade (the sweep is idempotent / skips already-backfilled datasets).
Residual: a container that never starts (image pull) or is OOM-killed can still
fail the hook; those are rare, retried on the next hourly auto-upgrade, and
mitigated by the generous memory limit + floating-tag Always pull.
Test: asserts no activeDeadlineSeconds, backoffLimit 0, and the timeout-wrapped
always-exit-0 command.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chart): backfill timeout headroom + values.schema entry (Bugbot #380)
- Medium: backfillTimeoutSeconds was 300s = Helm's default --timeout (5m) that a
manual `helm install/upgrade` uses. Since Helm counts image-pull + scheduling
against its hook wait (outside the in-container `timeout`), a 300s sweep could
push a manual upgrade past 5m and mark it failed. Lowered to 180s, leaving ~2m
headroom under the 5m default (and far more under the auto-upgrade's 10m).
- Low: schematized the new metadataBackfill block in values.schema.json
(enabled: bool, backfillTimeoutSeconds: int>=1, resources), matching the
autoUpgrade/imageRefresh blocks so helm type-checks it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chart): hard-stop the backfill timeout with -k (Bugbot #380)
`timeout "$TIMEOUT" tracebloc-backfill` only sends SIGTERM; if the entrypoint
ignores it or is wedged, timeout waits indefinitely, the wrapper never reaches
exit 0, and the hook hangs until Helm's --timeout — failing the upgrade, the
very mode removing activeDeadlineSeconds was meant to avoid. Add `-k 30` so
timeout escalates to SIGKILL 30s after the SIGTERM. Worst-case wrapper runtime
is TIMEOUT+30s (210s at the default), still well under the 5m manual --timeout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chart): widen backfill Helm headroom to 120s (Bugbot #380)
180s sweep + 30s SIGKILL grace = 210s worst-case in-container, leaving only ~90s
of Helm's default 5m --timeout for a cold ghcr.io pull + scheduling on a manual
install/upgrade. Lower backfillTimeoutSeconds to 120s so worst-case in-container
is 150s, leaving ~150s for pull/scheduling under the 5m default (and ~450s under
the auto-upgrade's 10m). The sweep is idempotent/incremental, so a smaller
per-run budget just spreads a large backlog across upgrades.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chart): backstop ImagePullBackOff with activeDeadlineSeconds (Bugbot #380)
backoffLimit 0 doesn't bound an ImagePullBackOff pod — it stays Pending (never a
failed attempt), so with no activeDeadlineSeconds the Job neither completes nor
fails and Helm waits its full --timeout before failing the release. The
in-container timeout/exit-0 only apply once the container starts, so they don't
cover a pod that never runs.
Restore activeDeadlineSeconds as a Job-level backstop = backfillTimeoutSeconds +
120s (240s at the default), well under Helm's 5m default, so a stuck/Pending pod
(pull failure) fails FAST and the hourly auto-upgrade retries instead of hanging
the whole wait. It does not gate a healthy run: the in-container `timeout` exits
~150s in, before the deadline. A genuinely un-pullable image still fails the
upgrade (irreducible for a synchronous hook running the ingestor image) but now
quickly and self-healing once the registry recovers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chart): make backfill deadline consistent with the pull budget (Bugbot #380)
The Job deadline (backfillTimeoutSeconds + 120 = 240) contradicted the values
comment's ~150s cold-pull budget: a pull of 90-150s plus a full in-container run
(150s) exceeded 240s and DeadlineExceeded a HEALTHY sweep. Since
activeDeadlineSeconds is Job wall-clock and includes the pull, it must exceed
(cold pull + full run).
Re-derive the deadline as backfillTimeoutSeconds + 30s grace + 120s pull = 270s
and reconcile the docs: in-container <=150s, cold-pull budget 120s, deadline
270s. A healthy run (warm pull ~instant, or a cold pull up to ~120s + <=150s
container) fits; a Pending/stuck pod is still bounded (270s). Sized for the
auto-upgrade's 10m --timeout (the target path); manual installs on a cold
cluster should pass --timeout 10m. A pull slower than the budget fails that
upgrade and self-heals on the next (image cached by then).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chart): backfill hook is post-upgrade only, not post-install (Bugbot #380)
The hook was post-upgrade,post-install. On a fresh install there are no
pre-cutover datasets to backfill, yet Helm waits on hooks even without --wait, so
a cold ingestor-image pull / ImagePullBackOff could fail the whole client install
(the bash/Windows installers run `helm upgrade --install` under the default 5m
timeout and assume a ~10-15s apply). Drop post-install so the hook never runs at
install time; a cluster that actually carries pre-cutover data reaches it on its
first auto-upgrade (post-upgrade).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Installer copy catalog: byte-exact golden of what a user sees
Adds scripts/tests/copy-catalog.bats — a golden catalog mirroring the CLI's
(cli repo, internal/cli/testdata/golden/), so the installer's user-facing copy
can be reviewed without running an install. Two files under
scripts/testdata/golden/, ordered like the run:
00-install.golden — the banner (versioned + direct-run variants), the
"2. Installing" roadmap, the six running step headers
(a-f; titles read live from install-k8s.sh so they can't
drift), and --help.
01-outcomes.golden — print_summary's five end states (connected / starting /
bad creds / image pull / crash) + the reboot footer,
incl. the macOS/Windows variant.
Rendered colour-off (NO_COLOR) so the text is byte-exact; the one live read
(_chart_version) is stubbed for determinism, and $HOME collapses to ~, so the
goldens are stable across machines/CI. The tests fail on drift; regenerate with
TB_UPDATE_GOLDEN=1 bats scripts/tests/copy-catalog.bats.
Not in the R8 manifest (tests/ + testdata/ aren't installer scripts); CI already
globs scripts/tests/*.bats and excludes .bats from shellcheck.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(installer): clear TRACEBLOC_BANNER_SHOWN in copy-catalog setup
Bugbot: print_banner no-ops when TRACEBLOC_BANNER_SHOWN is set (the curl|bash
bootstrap exports it after drawing the banner). The copy-catalog setup() never
cleared it, so an inherited value — from the shell or a prior install session —
would blank both banner samples in emit_install, failing the drift check or
regenerating a banner-less 00-install.golden. unset it in setup(), matching the
precedent in common.bats. Golden already has both banners (generated clean), so
no regen needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
…g-gated) (#368)
* feat(installer): node-local storage prototype (RFC-0003 Option C, flag-gated)
TB_STORAGE_MODE=node-local (default: hostpath, unchanged) switches the local
k3d install to store datasets on k3s's built-in local-path provisioner INSIDE
the node instead of bind-mounting ~/.tracebloc. Data then dies with
`cluster delete`, is not a browsable host folder, and needs no chmod 777 —
the RFC-0003 goal for the local install.
- common.sh: TB_STORAGE_MODE flag + validation; C1 forces AGENTS=0 (single-node)
since local-path is RWO/WaitForFirstConsumer and the shared data PVC is
mounted by jobs-manager-spawned Jobs.
- cluster.sh: node-local drops the -v ~/.tracebloc:/tracebloc@all bind-mount,
KEEPS k3s local-storage (removes --disable=local-storage), and skips the
world-writable host-dir pre-create.
- install-client-helm.sh: node-local emits storageClass.create=false +
name=local-path, hostPath.enabled=false; skips _ensure_release_dirs.
- cluster.bats: cover both modes.
Default (hostpath) path is byte-for-byte unchanged; existing installs on
--reuse-values do not migrate. Refs #367, backend#1151,
tracebloc/cli#366.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): gate the second _ensure_tracebloc_dirs call for node-local
install_client_helm() calls _ensure_tracebloc_dirs independently of
create_cluster; the first gate missed it, so node-local installs still
created empty world-writable ~/.tracebloc/{data,logs,mysql} dirs. Now
node-local only ensures the base dir exists (for values.yaml + the install
log) and skips the 777 data subdirs. Verified on a real tb-nodelocal
install. Refs #367.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(installer): document TB_STORAGE_MODE in install-k8s.sh usage header
The node-local storage flag (RFC-0003 Option C) was undocumented in the
env-var overrides list. Add it alongside HOST_DATA_DIR with a one-line note
on what node-local does (k3s local-path, no host dirs, AGENTS=0).
Regenerated manifest.sha256 (install-k8s.sh is manifest-covered).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): guard node-local storage mismatches (Bugbot #368)
Two silent-misroute footguns Bugbot flagged on the node-local path:
1. node-local + HOST_DATASET_DIR: node-local forces hostPath.enabled=false,
so a HOST_DATASET_DIR network export was accepted but ignored — datasets
would land on ephemeral local-path storage (gone on 'cluster delete').
Combining the two is a documented follow-up (backend#743 + RFC-0003);
validate_config now rejects the combo instead of misrouting.
2. Reusing a cluster built for the other storage topology: storage mode is
baked in at create time (hostpath bind-mounts /tracebloc + disables k3s
local-storage; node-local does neither). A node-local install onto a
hostpath cluster requests a local-path StorageClass that was disabled
(PVCs Pending); a hostpath install onto a node-local cluster points
hostPath PVs at an unmounted /tracebloc (ephemeral). New
_check_existing_cluster_storage_mode fails fast both ways, mirroring
_check_existing_cluster_dataset_mount. The /tracebloc bind mount is the
discriminator: present <=> hostpath cluster.
Tests: cluster.bats 35/35 (5 new storage-mode cases), common.bats +1
(node-local + HOST_DATASET_DIR rejected). shellcheck: no new findings.
Regenerated manifest.sha256 (cluster.sh + common.sh are manifest-covered).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): node-local must force SERVERS=1 too, not just AGENTS=0 (Bugbot #368)
The C1 single-node guarantee only forced AGENTS=0. But k3s server nodes are
schedulable (unlike a full k8s control plane), so TB_STORAGE_MODE=node-local
with SERVERS>1 still produced multiple nodes — the RWO local-path data PVC
binds on one while jobs-manager Jobs land on another and stay Pending, the
exact failure C1 exists to prevent. Now clamps both AGENTS=0 and SERVERS=1.
Tests: common.bats +2 (node-local clamps both; hostpath leaves them untouched).
shellcheck: no new findings. Regenerated manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): PATH-aware final CTA — stop telling users to run a CLI this shell cannot find (B2, #1174)
install-cli.sh records TB_CLI_USABLE_NOW (1 only when the CLI resolves in THIS
shell, not just a fresh terminal). summary.sh reads it via _cli_runnable_now
(with a live has-tracebloc fallback) so the final CTA says "Run tracebloc" only
when that is true, and "Open a new terminal, then run tracebloc" otherwise —
matching the honest guidance install-cli already prints for the ~/.local/bin
case. 2 new summary bats; install-cli suite green; shellcheck + manifest clean.
The complementary cli-repo change (installer prefers a writable dir already on
PATH so the same-terminal case happens more often) is a separate PR.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): B2 CTA relies solely on the fresh-shell signal, not the mutated process PATH (Bugbot #371)
_cli_runnable_now dropped its `has tracebloc` fallback: install.sh + provision.sh
prepend ~/.local/bin to THIS process PATH, so `has tracebloc` was true even when
the user's launching shell could not resolve it — so the CTA wrongly said "Run
tracebloc" on the common curl|bash path. Now it trusts only TB_CLI_USABLE_NOW
(set from install-cli.sh's FRESH-shell probe); unset/0 → the honest "open a new
terminal" branch. Also fixed a pre-existing summary.bats test that assumed the
unconditional CTA (pinned TB_CLI_USABLE_NOW=1) — it was RED on Linux CI (masked
locally by my having the CLI installed) — and tightened the new B2 test to assert
the "Run …" branch specifically.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): B2 usable-now gate keys on the CLI install dir, not process PATH (Bugbot #371 r2)
TB_CLI_USABLE_NOW was still set from `has tracebloc` in the installer process,
whose PATH install.sh prepended with ~/.local/bin — so it was true even when the
users returning shell cannot resolve the CLI. New _cli_at_system_dir keys the
flag on WHERE the CLI landed: a system dir (unconditionally on PATH) => usable
now => "Run tracebloc"; a $HOME bin (~/.local/bin, ~/bin) or unresolved =>
conservative => "open a new terminal". Unit test for the gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): B2 step verdict must match the summary CTA (Bugbot #371)
_verify_tracebloc_cli set TB_CLI_USABLE_NOW=0 for a ~/.local/bin install but
then printed the usable-now verdict ("... ready — run tracebloc to use it") and
returned, because `has tracebloc` is always true (install.sh prepends
~/.local/bin to the process PATH). The summary CTA, keyed on
TB_CLI_USABLE_NOW=0, said "open a new terminal" — a direct contradiction on the
least-privilege path, and following the step fails command-not-found.
Gate the usable-now verdict on `has tracebloc && _cli_at_system_dir`; otherwise
fall through to the honest "installed — open a new terminal" guidance and keep
TB_CLI_USABLE_NOW=0 so the summary agrees. Add a regression test; the two
existing usable-now tests now force _cli_at_system_dir (they mocked tracebloc as
a function, so command -v never returned a system path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): summary CTA distinguishes not-on-PATH-yet from new-terminal (Bugbot #371)
TB_CLI_USABLE_NOW=0 covered two different states, and the summary always said
"open a new terminal" for both:
- case A: installed to ~/.local/bin, persisted — a NEW terminal resolves it (only
this login shell does not). "Open a new terminal" is correct.
- case B: _cli_on_fresh_path failed — a new terminal will NOT find it either;
install-cli.sh prints the exact PATH fix and deliberately avoids "open a new
terminal". The summary contradicted that, pointing users at a useless step.
Add a second signal TB_CLI_ON_FRESH_PATH (set 1 on the fresh-path branch, 0 in
the case-B fall-through) and a third summary CTA branch: case B now says "Add
tracebloc to your PATH (see above)", matching install-cli.sh. Update the case-A
test to set the flag; add a case-B CTA test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): unset fresh-path flag defaults to new-terminal, not the see-above PATH fix (Bugbot #371)
The case-B CTA branch fired on `else`, catching BOTH TB_CLI_ON_FRESH_PATH=0
(install-cli.sh ran and printed the exact PATH fix) AND the flag being UNSET
(CLI step skipped/failed, so nothing was printed above). In the unset case the
"Add tracebloc to your PATH (see above)" message pointed at guidance that never
appeared. Gate case B on the explicit "0"; fold unset + case A into the safe
"open a new terminal" default (matching _cli_runnable_now's documented default).
Add an unset-flag test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(install): honest CTA when a system tracebloc pre-exists but the CLI step is skipped (Bugbot #371)
Bugbot (Medium): install_tracebloc_cli hard-defaulted TB_CLI_USABLE_NOW=0, so
if the CLI step is skipped or fails (download/installer/temp-dir miss → early
return, _verify_tracebloc_cli never runs) the connected summary said 'open a
new terminal' even when tracebloc was ALREADY on a system PATH dir and works in
the user's current shell.
Seed the default from the PRE-install state instead: usable-now iff a tracebloc
is already resolvable AND at a system dir. Gate on _cli_at_system_dir, NOT bare
'has tracebloc' — install.sh prepends ~/.local/bin to THIS process, which would
false-positive a ~/.local/bin install the returning shell can't yet see (the
exact trap the earlier #371 findings closed). _verify_tracebloc_cli still
overrides per this run's outcome; summary.sh's _cli_runnable_now is unchanged.
Tests: pre-existing system tracebloc + failed install → stays 1; ~/.local/bin +
failed install → 0. install-cli.bats green; manifest regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(installer): pin TB_CLI_USABLE_NOW in copy-catalog so the CTA golden is stable (#371)
Merging #371's PATH-aware connected-state CTA into a develop that has the #366
copy catalog broke copy-catalog.bats: emit_outcomes renders print_summary, and
with TB_CLI_USABLE_NOW unset the new CTA branch prints 'Open a new terminal…'
while 01-outcomes.golden still has the happy-path 'Run tracebloc to get
started.'. Pin TB_CLI_USABLE_NOW=1 in setup() (mirrors #371's summary.bats) so
the catalog deterministically renders the 'Run' line the golden captures — no
golden regen needed, and it's robust to an inherited flag (same env-stability
class as the TRACEBLOC_BANNER_SHOWN unset).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
…8) (#379)
The prod overlay pinned the 0.7 index digest as resolved on 2026-07-13
(v0.7.0 era). v0.7.4 (di#368 correlation-id, di#369 orphan-row reconcile)
and v0.7.5 have shipped since, so a prod install spawned a pre-fix
ingestor while dev/staging (floating 0.7) already run v0.7.5.
Re-resolved via scripts/resolve-ingestor-digest.sh 0.7 --write per the
release convention documented in the overlay; multi-arch index verified
(linux/amd64 + linux/arm64).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Comment threadscripts/tests/copy-catalog.bats
Comment threadscripts/tests/copy-catalog.bats Outdated
Comment threadscripts/tests/copy-catalog.bats
Comment threadscripts/lib/probe.sh Outdated
LukasWodkaand others added 2 commits July 24, 2026 15:13
…repo (#385) (#389)
The repo presence guard string-matched (helm repo list 2>&1) | Out-String.
On a fresh machine helm reports 'no repositories' on stderr, and Windows
PowerShell 5.1 renders that ErrorRecord with this script's own
...\tracebloc-installer-<n>\install-k8s.ps1 temp path — which contains
'tracebloc' — so the add was skipped on every fresh install and Step 4 died
with 'Error: repo tracebloc not found'.
- always 'helm repo add --force-update' (idempotent, heals wrong URLs,
refreshes the index; the unchecked 'helm repo update' pass is gone)
- add failures abort with helm's actual output
- preflight probes the chart repo index.yaml strictly (ps1 + bash twins):
the site root 404s by design, so plain reachability proved nothing
- manifest.sha256 regenerated (R8)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix: resolve Bugbot findings on promotion PR #383
- Golden step scrape: broaden [a-f] to [a-z] so new steps stay covered
- Golden summary capture: route stdout+stderr through a file for deterministic ordering
- check_golden: point at TB_UPDATE_GOLDEN=1 when the golden is missing
- Host audit: stop promising "no admin" for Tier 1 until rootless lands
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: refresh installer manifest for probe.sh audit-copy change
The supply-chain manifest pins a SHA per sub-script; the Tier 1 host-audit
wording fix changed scripts/lib/probe.sh, so regenerate scripts/manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadscripts/lib/summary.sh
Comment threadscripts/lib/summary.sh
Comment threadclient/values.yaml Outdated
…r install's data (#376) (#384)
* feat(installer): guard against silently adopting leftover data (#376)
A new install (one that creates a fresh cluster) used to mkdir -p its data
dirs and silently adopt whatever was already under HOST_DATA_DIR — data
from an earlier install (different layout, older version, custom dir) got
picked up, so a "fresh" install was not guaranteed fresh.
Add guard_leftover_data (cluster.sh), called from create_cluster only when
creating a NEW cluster (an existing cluster is an in-place reuse/upgrade
whose data stays by design, §3.3). It detects real client data — MySQL or
dataset files — across both the flat and per-release layouts, and forces a
choice instead of adopting it:
- interactive: reuse / wipe / new dir / abort (abort is the default)
- non-interactive: --reuse-data / --wipe-data / --data-dir=PATH
(or TB_LEFTOVER_ACTION); no terminal + no choice => fail-safe abort
- TRACEBLOC_SKIP_LEFTOVER_GUARD=1 bypasses entirely
Scoped to HOST_DATA_DIR only — HOST_DATASET_DIR may be a shared network
mount, so it is never scanned or wiped. Also doubles as the node-local
migration prompt (#367): existing ~/.tracebloc data is never silently
stranded.
Implements RFC-0003 D3 (cli#366, §4). Adds scripts/tests/leftover-guard.bats
(13 tests), README + --help/env docs, and regenerates manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(installer): refresh 00-install golden for the leftover-guard --help lines
The copy-catalog byte-exact golden captures install-k8s.sh --help output; the
new --reuse-data/--wipe-data/--data-dir usage + leftover-data section drifted
it. Regenerated via TB_UPDATE_GOLDEN=1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): pipefail-safe leftover detection + sanitize prompt input (Bugbot #384)
- _leftover_data_dirs: a find|head|grep pipeline SIGPIPEs find once its
output exceeds the pipe buffer (a real multi-table MySQL dir); under the
installer's set -o pipefail that read as "empty" and the guard silently
adopted real leftovers — the exact failure it exists to prevent. Read the
first path from a process substitution instead (find's status stays out of
the check, still short-circuits). Regression test seeds 3000 files + pipefail.
- Interactive reply/newdir reads now go through _read_sanitized
(_strip_paste_garbage + whitespace trim), so pasted CSI escapes can't enter
HOST_DATA_DIR and a spaces-only path no longer slips past the non-empty check.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(installer): refresh manifest.sha256 for cluster.sh (Bugbot #384 fixes)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): wipe fails closed when data can't be removed (Bugbot #384)
_wipe_leftover_data only warned on rm -rf failure and the guard still
returned success + logged "wiped"; create_cluster then adopted whatever
survived — common when container MySQL dirs are root-owned and the host
user can't remove them — silently breaking the "wipe means gone" fail-safe.
Now _wipe_leftover_data verifies each target is actually gone (not just rm's
exit code) and returns non-zero if any survived; the guard's wipe branch
errors out instead of proceeding. Regression test: an unremovable dir makes
wipe abort (exit 1) with the survivor left in place, not adopted.
Regenerates manifest.sha256 for the cluster.sh change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): assert HOST_DATA_DIR non-empty + under $HOME before wipe (Lukas review #384)
Belt-and-suspenders for the destructive path: _wipe_leftover_data now
asserts HOST_DATA_DIR is a non-empty path strictly under $HOME before any
rm. Today _leftover_data_dirs already returns empty when it's unset (so the
wipe never runs), and validate_config enforces the invariant — but placing
the check in the destructive function guards the rm even if a future refactor
changes call ordering (an empty HOST_DATA_DIR would collapse the
"$HOST_DATA_DIR"/* scope pattern to /*). Two tests: empty + outside-$HOME
both refuse.
Regenerates manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): detection fails closed on unreadable data dirs (Bugbot #384)
_leftover_data_dirs used find ... 2>/dev/null, so a root/container-owned
mysql/data dir the host user can't list produced no paths AND swallowed the
Permission-denied error — looking like a clean slate, letting create_cluster
adopt the leftovers (the exact fail-open the guard exists to prevent, and the
same ownership case the wipe path already treats as fatal).
Now capture find's stderr: a file found OR any find error => leftover. Keeps
the pipefail-safe read < <(find) (no find|head pipe). Regression test: a
chmod 000 mysql dir is detected, not skipped. Regenerates manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): drop mktemp from leftover detection — no fail-open path (Bugbot #384)
The previous fix redirected find's stderr to a temp file; if mktemp failed,
stderr went to /dev/null and the unreadable-dir check never fired -> a
root/container-owned leftover dir looked empty and got adopted. Remove the
temp file entirely:
- explicit [[ -r && -x ]] check fails closed on an unlistable top dir (the
common container-owned case),
- a temp-file-free $(find … 2>&1 >/dev/null) fallback (only when no top-level
file was read) catches an unreadable subdir too.
Keeps the pipefail-safe read < <(find). Added a subdir-unreadable regression
test. Regenerates manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): symlink-safe wipe + tilde/empty data-dir handling (Bugbot #384)
Three findings on the new-dir/wipe surface:
- Symlink traversal (Med): detection walked $HOST_DATA_DIR/*/ incl. dir
symlinks, so a planted 'evil -> /etc' gave …/evil/mysql and wipe's rm -rf
followed it outside scope. Detection now skips symlinked candidates
(-L check on both the per-release glob and each candidate); wipe refuses to
rm a symlink as a backstop.
- Tilde paths (Med): --data-dir=~/foo / new-dir '~/foo' were read literally,
so validate_config turned them into $HOME/~/foo and failed. Expand a
leading ~ / ~/ at the validate_config choke point.
- Empty data-dir (Low): --data-dir= set HOST_DATA_DIR='' which resolved to a
surprise dir; validate_config now fails closed on empty.
Tests: symlink-not-a-candidate, wipe-refuses-symlink, tilde-expands,
empty-fails-closed. Regenerates manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): probe TB_TTY openability, not just -r (Bugbot #384)
`guard_leftover_data` decided interactivity with `[[ -r "$TB_TTY" ]]`, but
/dev/tty is world-readable even with no controlling terminal (CI, curl|bash).
Those runs took the interactive branch, `read` failed immediately, and the
guard aborted with the generic abort text instead of the non-interactive
guidance that lists --reuse-data / --wipe-data.
Add `_tty_usable` — the same `{ : <"$TB_TTY"; }` openability probe assess.sh
already uses — and gate both the reuse/wipe/new-dir prompt and the new-dir
read on it.
Regression test: readable-but-unopenable /dev/tty -> non-interactive guidance
(skips in an interactive shell). Regenerates manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): reject HOST_DATA_DIR == $HOME, require a subdirectory (Bugbot #384)
validate_config allowed HOST_DATA_DIR to equal $HOME, and the new tilde
expansion maps a bare `~` to $HOME — so a common new-dir / --data-dir input
would select the whole home directory. The installer would then chmod 777
home-level logs/mysql dirs, bind-mount all of $HOME into the cluster, and
leftover detection would treat any existing ~/data or ~/mysql as install data.
Require HOST_DATA_DIR to be strictly UNDER $HOME (never $HOME itself); this
also closes the same footgun via HOST_DATA_DIR=$HOME / --data-dir=$HOME.
Tests: `== $HOME` rejected, bare `~` rejected. Regenerates manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): don't rescan flat data dirs as release dirs (Bugbot #384)
The per-release scan walked every subdir of HOST_DATA_DIR, including the
flat-layout `mysql`/`data` dirs. A real MySQL datadir always contains a nested
`mysql` system schema, so detection reported both $HOST_DATA_DIR/mysql and
$HOST_DATA_DIR/mysql/mysql as leftover roots — mislabeling layout, confusing
the prompt, and doubling wipe targets.
Skip the flat data dirs in the per-release walk; they are already candidates.
Test: a datadir with a nested `mysql` schema reports exactly one root.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
LukasWodkaand others added 4 commits July 24, 2026 16:02
…te-limit-proof installs (#382)
* fix(installer): pin the k3d release (K3D_VERSION) — no more releases/latest lookup
The upstream k3d install script resolves its version via a GitHub
releases/latest redirect, which breaks under GitHub rate limiting on
shared egress IPs (CI runners, corporate NAT) and failed 2/9 distro CI
jobs on main on 2026-07-21 with a bare 'curl: (22) 404 / Failed to
install k3d'. Retrying can't help — the limit persists per-IP.
Pin it the way K8S_VERSION already pins k3s:
- K3D_VERSION (common.sh, default v5.9.0) — the install script is now
fetched AT the pinned tag (immutable bytes instead of k3d's mutable
main) and TAG=<pin> makes it skip the releases/latest lookup entirely.
Its checksums.txt verification of the binary still runs.
- K3D_VERSION="" or "latest" restores resolve-at-install-time.
- Covers both the Tier-0 (user-space) and sudo install paths.
- Help text + golden catalog + signed manifest regenerated.
Verified: scripts/tests/distro-prereqs.sh passes in fresh
amazonlinux:2023 and ubuntu:24.04 containers (the two failing CI jobs),
installing k3d v5.9.0; setup-linux.bats 47/47 incl. two new pin tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): K3D_VERSION guard + honest latest semantics (Bugbot r1)
- Fail closed on malformed K3D_VERSION before any fetch: the value lands
in a raw.githubusercontent URL path, so a crafted value carrying '/'
could traverse to a script outside k3d-io/k3d and run it privileged.
Accept only vX[.Y.Z…] tags or 'latest'.
- Stop claiming K3D_VERSION="" restores resolve-at-install-time — the
common.sh :- default re-pins on empty (same convention as K8S_VERSION);
only 'latest' opts out. Comments now say so.
- New bats case: malformed tag errors out with nothing fetched/run.
- Manifest regenerated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* k3d: verify the release binary ourselves — upstream install.sh checks nothing (review)
Asad pulled the pinned v5.9.0 install.sh: downloadFile fetches the bare
binary, installFile chmod+cp's it — no checksum is downloaded or verified,
so the previous comment/PR claim ('the script still verifies the binary')
was wrong, on a sudo path.
- Linux: drop upstream install.sh entirely; fetch k3d-linux-<arch> +
checksums.txt straight from the pinned release and sha256-verify before
installing (mirrors _fetch_kubectl, fail-closed). K3D_VERSION=latest now
resolves the tag via the plain /releases/latest redirect and takes the
SAME verified path.
- Windows: the ps1 fetched sha256sum.txt, which has never existed on k3d
releases (asset is checksums.txt) — it 404'd into a fail-open catch, so
that verification silently never ran either. URL fixed.
- preflight: the k3d tool-host probe now points at github.com (the install
script from raw.githubusercontent.com is no longer fetched).
- bats: 5 tests for the new path (verified download / mismatch fail-closed /
missing-asset fail-closed / sudo target / latest-resolve); the 3
script-era tests removed with the script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* k3d fetch: stall protection + honest empty-value comments (Bugbot r2)
- connect-timeout + speed-limit floor on the release downloads (no --max-time:
the ~50 MB binary on a slow-but-healthy link must not be capped), max-time on
the small releases/latest HEAD resolve — a hung transfer under spin_cmd would
otherwise spin forever
- comments: only the literal K3D_VERSION=latest opts out; also corrected the
adjacent pre-existing K8S_VERSION comment that claimed "" resolves latest
(the :- default re-pins on empty)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* k3d (Windows): fail-closed checksum verification (Bugbot r3)
An unfetchable checksums.txt, a missing asset line, or a mismatch now all
abort and remove the download — matching the Linux path and the kubectl
precedent — instead of skipping silently or logging into a fail-open catch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
#391)
* fix(installer): followable elevation steps for the irm|iex flow (#386)
'Right-click > Run as Administrator' is impossible in the documented
one-liner flow (there is no file). Field-observed dead-end: the user did
not know how to open an elevated PowerShell. Give the actual steps
(Win+X -> Terminal (Admin) / Ctrl+Shift+Enter, accept UAC, re-run the
one-liner).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* elevation re-run one-liner: explicit https:// (Bugbot)
Scheme-less irm resolves the first hop of the bootstrap trust root over
cleartext HTTP (and canonical docs use https://). Manifest regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…t hook (RFC-0003 §8.2–8.4) (#378)
* feat(seal-check): label contract + storage-assertions test hook (RFC-0003 S8.2 / backend#1184)
Establish the seal-check enumeration contract and fill the storage gap in
the chart conformance suite:
- Label every conformance helm-test Job (and its pods) with
tracebloc.io/seal-check="true" + tracebloc.io/seal-check-name=<check> --
the contract the tracebloc CLI consumes (cli#393). Applied to the
existing egress-enforcement probe and backend-reachability check via a
new tracebloc.sealCheckLabels helper.
- New storage-assertions helm-test Job (sealCheck.storageAssertions.*,
default on, fully nil-guarded for --reuse-values upgrades): release PVCs
Bound on the expected StorageClass; in dynamic-PVC mode
(hostPath.enabled=false) no release PVC backed by an unmanaged hostPath
PV -- catches the RFC-0003 D3/D4 leftover-PV claimRef capture. k3s
local-path node-local paths tolerated and reported (Option C);
clusterScope=false degrades the PV scan to a name check with an explicit
WARNING, never silently.
- Least-privilege SA/RBAC as negative-weight test hooks (PVC read in the
namespace; PV read only when clusterScope allows a ClusterRole).
- docs/SEAL-CHECK.md: label contract, coverage today, UNSEALED philosophy,
per-substrate coverage view, and the k3d/k3s NetworkPolicy egress
verification runbook (documented honestly as not yet executed).
- 20 new helm-unittest cases; 287/287 pass, helm lint --strict clean on
all four platform values, check-drift clean.
Part of tracebloc/backend#1184
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(seal-check): segment-bounded node-local prefixes; drop misleading WARNING after legacy FAIL
Two Bugbot findings on the storage-assertions check (PR #378):
- The pv-hostpath allowlist matched entries with a bare "$prefix"* glob,
and the default /opt/local-path-provisioner entry carries no trailing
slash, so sibling trees like /opt/local-path-provisioner-evil/... were
accepted as node-local and could pass the storage seal. Entries are now
normalized (trailing slash stripped) and matched on whole path
segments: a prefix admits exactly itself or paths under it, never
siblings. The default entry in values.yaml / template dig fallback /
schema / docs is normalized to the trailing-slash style; either entry
style now behaves identically.
- With clusterScope=false (no PV read), a claim captured by a chart
legacy hostPath PV printed FAIL and then still printed the PARTIAL
verification WARNING claiming the claim was "verified not bound" to a
legacy PV. The WARNING is now suppressed for a claim whose legacy-name
check just FAILed - the FAIL alone is the honest verdict there; claims
that pass the name check keep the WARNING unchanged.
Validated: helm lint --strict on the 4 CI platform values; helm template
(near-default / node-local / flag off / sealCheck: null); helm unittest
289/289 (2 new cases pin both guards); sh -n + shellcheck on the rendered
script; mocked-kubectl runs covering sibling-path FAIL (both entry
styles), exact-prefix-root OK, legacy FAIL without WARNING, no-legacy
WARNING x3, unmanaged-tree FAIL, PV-read-error FAIL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* seal-check docs: literal job names for --filter (review F1)
The generic '--filter name=<release>-<check>-check' formula silently
no-ops for backend-reachability (its Job is egress-reachability-check):
helm test matches zero hooks and exits 0 -- a silent pass. List the
literal Job name per check instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…-aware memory hints (#387) (#392)
* feat(installer): fail fast on disabled firmware virtualization + WSL2-aware memory hints (#387)
- Step-1 preflight: new Get-PfVirtualization reader (HypervisorPresent
first -- Hyper-V owns VT-x and firmware reads false then -- else
VirtualizationFirmwareEnabled). Disabled -> hard fail with the BIOS fix
+ Task Manager confirmation tip, BEFORE Docker is installed/launched.
Field case: Docker Desktop's own 'Virtualization support not detected'
appeared mid-install with zero guidance.
- The three memory hints pointed at Docker Desktop -> Resources -> Memory,
a slider that does not exist on the WSL2 backend (the default) -- now
give both levers (.wslconfig + wsl --shutdown / Hyper-V slider).
- Docker wait-loop: name the error-window case instead of only 'still
starting up'.
- GPU: soft VRAM note for entry-level cards (a 2 GB GT 710 passes every
check but cannot fit a model) -- Hint only, never a gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* memory hints: the Hyper-V slider lives at Settings -> Resources -> Advanced
Verified against Docker's settings docs ('Memory limit ... Mac, Linux,
Windows Hyper-V' under Resources > Advanced; WSL2 explicitly deferred to
the WSL utility VM). Review question by Lukas.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadscripts/lib/setup-linux.sh
shujaatTraceblocand others added 2 commits July 24, 2026 16:48
…hook docs) (#394)
* fix: resolve Bugbot findings on promotion PR #383 (summary copy + backfill docs)
- Reboot note: only promise auto-restart when docker.service autostart was
actually enabled; Tier 0 (zero-privilege) leaves it to the user, so be honest
- Connected summary Data path: branch on TB_STORAGE_MODE — node-local keeps
datasets in-node on k3s local-path, so don't point at a host /tracebloc
- Backfill docs: correct values.yaml + values.schema.json to say post-upgrade
ONLY (not post-install) to match the Job's helm.sh/hook annotation
- Refresh scripts/manifest.sha256 for the summary.sh + cluster.sh changes
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: seed reboot auto-restart promise from pre-install docker.service state
Bugbot: on Tier 0, ensure_cluster_autostart deliberately never runs
`systemctl enable docker`, and TB_DOCKER_AUTOSTART was set only when *this*
run enabled the service. A host where docker.service is already enabled on
boot (a normal package install) therefore kept the flag unset, so _reboot_note
told the user to start Docker manually even though the cluster returns on its
own after a reboot.
Seed TB_DOCKER_AUTOSTART from the current on-boot state via an unprivileged
`systemctl is-enabled docker` read (matching only the persistent "enabled"
state, not the transient "enabled-runtime"). Add cluster.bats coverage for the
pre-enabled, disabled, and enabled-runtime cases; refresh manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…kend#1236, option A) (#393)
* feat(installer): size the default training budget to the machine (backend#1236, option A)
The static cpu=2,memory=8Gi default was wrong at both ends: dead on
arrival on nodes under 8 GiB (the WSL2 field case -- doctor 'not ready',
nothing could ever schedule) and ~12% utilisation of a 64 GiB box, while
the wizard itself calls max 'recommended'.
Both installers now write RESOURCE_REQUESTS/LIMITS sized to this machine:
largest node allocatable minus the ~1 CPU / 3 GiB platform overhead
(largest, not summed -- k3d's server+agent are one machine, cli#399).
Precedence: TRACEBLOC_TRAINING_RESOURCES override > the installed
release's existing choice (a 'tracebloc resources set' decision now
survives re-install instead of being clobbered back to a default) >
machine sizing > the historic static fallback (tiny/unreadable machines).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* machine-sizing probe: bound kubectl with --request-timeout (Bugbot)
A wedged API server must degrade to the static default, never hang values
generation. Both installers; the sizing tests' stubs now only answer a
bounded call, so dropping the flag fails them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* carry-lookup: gate helm get values behind a bounded probe (Bugbot)
helm get has no request timeout, so the existing-choice lookup now runs
only after 'kubectl get namespace --request-timeout=5s' succeeds -- a
wedged API degrades to machine sizing / the static default instead of
hanging values generation. Both installers; stubs answer only bounded
calls.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* carry-lookup: never carry the historic static default (Bugbot)
An older install stored the chart default cpu=2,memory=8Gi; carrying it as
a 'choice' would keep the unschedulable budget on exactly the machines
this sizing exists to fix. The static default was the absence of a choice
-- only a differing value survives re-install. Both installers + tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ps1 sizing: jsonpath cpu/memory lines, no full-JSON parse (Bugbot r5)
Extract exactly the two allocatable fields via jsonpath (one 'cpu memory'
line per node), mirroring the bash twin -- a parse hiccup on unrelated
node JSON (e.g. long image-digest fields under PS 5.1 string handling)
can no longer fall into the catch and silently reinstate the static
default this change exists to remove. Pester mocks moved to the line
contract.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…before Helm (#388) (#397)
* feat(installer): Windows provisioning parity — login + client create before Helm (#388)
Port the bash provisioning sequence (scripts/lib/provision.sh) to
install-k8s.ps1, ending the legacy hand-copied Client-ID/password flow:
- Step reorder: 3 = install the tracebloc CLI (was 5), 4 = register this
machine, 5 = install the client via Helm. Roadmap updated.
- Invoke-ProvisionClient (Step 4): browser sign-in (tracebloc login,
device flow, attached to the console) + tracebloc client create --yes
--credential-file — the credential never reaches the terminal; the file
is parsed (KEY=value, first-'=' split) and deleted immediately. The
minted slug becomes the namespace (bash Q2 parity). Includes the
one-client-per-machine pre-flight (fail-closed on inconclusive reads;
legacy 'tracebloc'-namespace deferral) and Print-CreateFailure (real
failure reason incl. the carbon-zone special case).
- Adopted re-runs reconcile in place: reuse the previous values password,
heal a stale clientId to the adopted UUID; honest terminal error when
the local values file is gone.
- TRACEBLOC_CLIENT_ID/PASSWORD env pair stays as the unattended path
(verified once, non-interactively). The old interactive prompts survive
ONLY as the fallback for a missing/too-old CLI.
- Get-InstalledClientInfo factors the one-client enumeration (#200
fail-closed semantics) so the pre-flight and the Helm guard share one
source. CLI-install failure copy no longer claims the client is set up.
Closes#388.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): adopted heal passes the one-client guard + paste-sanitize the name (Bugbot r1)
- The one-client guard now lets ADOPTED mode through on an id mismatch —
that mismatch IS the heal the mode exists for (helm still stores the
cli#125-era numeric dashboard id while the backend anchored this
cluster to the adopted UUID). Every other mode still refuses. The
adopted Pester test now drives the realistic stale-id scenario, plus a
negative test proving the guard is intact outside adopted mode.
- ConvertTo-SanitizedInput strips CSI sequences, bracketed-paste markers,
and control chars from the Step-4 name prompt (port of common.sh
_strip_paste_garbage; customer-reported on the bash flow 2026-07-20).
Pester: 118 passed / 0 failed / 8 skipped (Windows-only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): verify minted creds, surgical adopted reconcile, signal-safe credential file (Bugbot r2)
- Minted credentials now verify via api-token-auth like every other mode
(never skip verification by provisioning method): backend skew or a
mid-flow deactivated account fails at install, not as a crash-looping
pod. Unreachable backend stays warn-and-continue.
- Adopted mode with a LIVE release reconciles surgically: helm upgrade of
THAT release, in ITS namespace, with --reuse-values --set-string
clientId=<uuid> — preserving the deployed configuration and secret
(bash parity) instead of regenerating values.yaml with fresh defaults.
Needs no local password at all; the local values file gets only its
clientId line healed. A rebuilt cluster (adopted anchor, no release)
keeps the full values write and its honest no-password error.
Wait-ForClientReady now watches the live release's namespace.
- The credential file is removed in a try/finally spanning mint->parse,
so Ctrl-C / terminating errors / Err exits can't leave the secret on
disk (ps1 analogue of _PROVISION_CRED_FILE + install_cleanup).
Pester: 119 passed / 0 failed / 8 skipped (Windows-only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
LukasWodkaand others added 3 commits July 27, 2026 09:14
… (backend#1252) (#400)
* fix(installer): make the curl TLS floor structural, not per-call-site (backend#1252)
`CURL_SECURE` was a bare constant every call site had to splice in by hand, so
call sites kept losing it: seven live `curl` invocations ran with no minimum TLS
version, including the POST in `verify_credentials()` that carries the client's
password. These installs run on customer-managed hosts and behind TLS-inspecting
proxies, which negotiate down to whatever the client permits — the reason this
repo adopted an explicit floor instead of trusting curl's defaults.
Add `curl_secure()` in `scripts/lib/common.sh` and route every fetch in
`scripts/lib/*.sh` through it (18 call sites). The wrapper always passes
`--tlsv1.2` and supplies default `--connect-timeout 30` / `--max-time 300`.
Defaults are injected before `"$@"`, so a call site that wants a tighter bound
still wins (curl honours the last occurrence), and a transfer that bounds itself
with `--speed-limit`/`--speed-time` gets no injected `--max-time` — a hard
deadline would fail a slow-but-healthy link on a large binary download. Every
existing site therefore keeps its effective behaviour; seven gain the floor and
nine previously unbounded ones gain a deadline.
Also fixed while here:
- `gpu-amd.sh` had the least-bounded curl usage in the repo — no TLS floor, no
timeout, no retry. Both calls now go through the wrapper; the `.deb` download
is retry-wrapped. The listing scrape deliberately is not: `retry()` reports
attempts on stdout, which is that function's return value.
- `install-k8s.ps1`'s WSL2 here-string had the same two nvidia-container-toolkit
fetches bare. It cannot source `common.sh`, so it spells the flags out inline
the way the bootstrap does.
`scripts/install.sh` keeps its seven hardcoded literals: it is the trust root
that fetches `common.sh`, so it cannot source the wrapper.
`CURL_SECURE` stays defined and unchanged for out-of-tree callers, but nothing
in the repo reads it now — the wrapper names the flag itself, so the constant can
never silently reshape every fetch in the installer.
Enforcement: an INTERIM third check in `scripts/check-style.sh` fails on a bare
`curl`. tracebloc/.github#65 already implements this properly (a shell-aware
lexer, not a grep) in a shared reusable workflow, but that workflow is not on
`main` yet and cannot be referenced from here until it is. The check is marked
for retirement the moment this repo adds that caller.
Regenerated `scripts/manifest.sha256` (R8 supply-chain gate).
Found in #399.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): stall-bound the kubectl fetch, don't give it a deadline (Bugbot)
`_fetch_kubectl` had no time bound at all, so routing it through `curl_secure`
handed it the wrapper's default `--max-time 300`. kubectl is a ~50 MB binary, and
this repo already documents (at `_fetch_k3d_release`, same file) that a fixed
ceiling fails a slow-but-healthy link at that size — so the wrapper would have
made every retry fail where the fetch previously completed.
Give both fetches the same `--connect-timeout 15 --speed-limit 1024
--speed-time 60` as the k3d pair. That is also how `curl_secure` knows to skip its
default deadline, and it is strictly better than before: the fetch was previously
unbounded in both directions, so a mid-stream stall hung the step indefinitely.
Audited the other 7 sites that now inherit the 300s default — get.docker.com,
get-helm-3, the Homebrew script, stable.txt, the DMG checksum, the device-plugin
manifest and the amdgpu-install package are all small text/script payloads. The
only large downloads in the repo are kubectl, k3d and the Docker Desktop DMG; the
latter two were already stall-bounded.
Adds a bats test pinning it, since nothing covered `_fetch_kubectl` before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
…hes edges (#398)
The prod ingestor digest pin reached no customer edge, for two independent
reasons.
1. Standard installs never applied the overlay. `client/values-prod.yaml` was
an install-time `-f` overlay, but the installer runs `helm upgrade --install
… --values "$values_file"` with only its generated values file. No installer
path ever passed the overlay, so a normal prod install kept the base default
`images.ingestor.digest: ""` and floated on tag `0.7` with
imagePullPolicy=Always — exactly like dev and staging.
2. Where it was applied by hand it could never be updated. The fleet
auto-upgrade CronJob runs `helm upgrade --reset-then-reuse-values` with no
`-f` and no `--set`: that resets to the new chart's `values.yaml` defaults,
then re-applies the release's stored user-supplied values. An overlay value
is user-supplied, so it was replayed verbatim forever — and Helm only
auto-reads `values.yaml` from a chart, so the updated overlay shipped inside
the new chart archive was never read.
Chart defaults propagate through that upgrade; user-supplied values freeze.
So the pin has to be a chart default, which is how the egress-proxy squid
image has always been pinned.
* `images.ingestor.prodDigest` (new, chart default) carries the pin.
* `images.ingestor.prodPin` (new, default true) gates it. A new
`tracebloc.ingestorDigest` helper resolves one effective digest for every
consumer: explicit `images.ingestor.digest` wins in any environment, else
`prodDigest` when prodPin is on and the resolved CLIENT_ENV is prod, else
empty (float on `tag`). dev/staging carry `env.CLIENT_ENV: dev|stg` in
user-supplied values while the template defaults it to prod, so prod pins
and non-prod floats with zero per-edge action. `prodPin: false` floats a
canary prod edge.
* The metadata-backfill hook now shares the same helper, so it can never run
a different build than the ingestions it reconciles.
* The consuming contract in client-runtime `_build_image_reference` is
unchanged: digest set -> repo@digest + IfNotPresent, empty -> repo:tag +
Always.
`client/values-prod.yaml` is removed rather than kept as a shim: passing
`-f values-prod.yaml` is what created the frozen-value bug, so leaving the
file would invite operators to re-create it. `MIGRATION.md` documents the move,
the canary knob, and how to clear a hand-layered stored digest (which still
takes precedence and would otherwise stay frozen).
`resolve-ingestor-digest.sh` keeps its resolution mechanism (multi-arch index
digest + guard) and now writes `prodDigest` in `client/values.yaml`, keyed on
that unique name with a single-match assertion so it can never rewrite a
sibling image's `digest:` leaf, and reminding the operator to bump Chart.yaml.
CI's `ingestor-multiarch` guard now validates `prodDigest` too — it previously
read only values.yaml and so never checked the overlay it was meant to guard —
and hard-fails an empty pin, which would silently un-pin prod.
Verified on a throwaway k3d cluster: installing the published 1.9.5 chart
renders an empty digest (prod floats today), and `--reset-then-reuse-values` to
this tree lands the chart-default pin on the already-installed edge, while the
same upgrade leaves an overlay-style user-supplied pin frozen on its
install-day value. `scripts/tests/e2e-auto-upgrade.sh` now asserts all of that
plus the prodPin opt-out surviving the next auto-upgrade.
Chart bumped 1.9.5 -> 1.9.6 — the chart only publishes on a version change, so
an unbumped pin refresh reaches no edge.
Fixestracebloc/backend#1245
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…395)
The Tier 0 fast path skips install_system_deps but still runs install_helm,
which shells to get-helm-3 — that needs openssl (checksum) and tar (unpack).
On a minimal docker-group host lacking them, the "zero-privilege" install
promised no admin, then failed mid-Helm-install.
Add _ensure_helm_prereqs: verify openssl+tar are present before install_helm on
the Tier 0 path. We cannot install them there without sudo (Tier 0 promises no
admin, and setup_pm/$PM_INSTALL aren't even run), so surface the real constraint
with an actionable fail-fast instead of silently sudo-ing or dying cryptically.
The full flow is unchanged (install_system_deps still installs them).
Regenerate scripts/manifest.sha256 for the setup-linux.sh change.
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Lukas Wuttke <lukas@tracebloc.io>
Comment threadREADME.md
LukasWodkaand others added 7 commits July 27, 2026 09:48
…load, no get-helm-3 (#396)
* feat(installer): self-serve Helm prerequisites — verified direct download, no get-helm-3 (#395)
The installer now takes care of its own requirements instead of erroring
out and telling the user to install tools:
- install_helm fetches the pinned Helm release directly from get.helm.sh
and verifies it against the published .sha256sum (fail-closed), exactly
like the k3d direct download (#382). helm's get-helm-3 script is gone —
it floats on the mutable helm/helm@main, performs its checksum step with
openssl (absent on minimal cloud images, Bugbot #383), and its fetches
are unbounded. openssl is no longer needed anywhere in the flow.
- HELM_VERSION pin in common.sh (v4.2.3; 'latest' resolves at install
time via get.helm.sh/helm-latest-version, same verified path; malformed
tags fail closed before any fetch).
- _ensure_unpack_tools: when tar/gzip are genuinely missing on the Tier 0
fast path (which skips install_system_deps), install them via the
package manager — quietly as root/passwordless sudo, with an honest
one-line reason when a password is needed — rather than aborting with
'go install tar'. install_system_deps drops openssl, adds gzip.
Supersedes the fatal-error approach in #395 (product call: never tell the
user to install tools we can install ourselves). Windows already fetches
Helm directly from get.helm.sh, so the ps1 path is untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): route sudo probes through _real_sudo/_have_sudo_bin (Bugbot r1)
The A2 sudo shadow runs '-n true' as a command when root, and 'has sudo'
is always true because the shadow function exists — use the #372
primitives like preflight_sudo/_probe_privilege do. The root-without-
sudo-binary strip is dropped entirely: the shadow already handles root by
executing the command directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): Tier 0 unpack install — sudo keepalive + dpkg-lock wait + one combined install (Bugbot r2)
_ensure_unpack_tools ran package installs on the Tier 0 path without the
full flow's guards: apt could sit on the dpkg lock invisibly behind the
spinner, and a long wait could outlast the just-primed sudo timestamp so
the next sudo re-prompts behind the spinner and hangs. Prime, then keep
the ticket warm (preflight_sudo's pattern; killed right after the install
— the zero-privilege tier shouldn't hold a warm admin ticket), wait out
the dpkg lock (bounded + visible), and install everything in ONE package-
manager call so there's a single sudo consumer right after priming.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): keepalive ownership + latest-tag isolation (Bugbot r3)
- _ensure_unpack_tools kills only the keepalive IT started: on the Tier
1/2 recovery path SUDO_KEEPALIVE_PID belongs to preflight_sudo, and
killing it would let later privileged steps re-prompt behind a spinner.
The global is only claimed when empty (install_cleanup coverage) and
only cleared when it is ours.
- HELM_VERSION=latest: isolate the endpoint body with tail -1 — retry's
attempt notices go to stdout and a failed-then-successful fetch would
concatenate them into the captured tag, failing the anchored regex with
a false 'couldn't resolve'. (install_k3d's resolver is immune: its
${var##*/} strip discards anything before the redirect URL.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(copy-catalog): regenerate 00-install golden for the HELM_VERSION knob
The PR adds a HELM_VERSION env var (common.sh), which surfaces one line in
install-k8s.sh --help. emit_install reads --help live, so the golden drifted
by exactly that line. Regenerated via TB_UPDATE_GOLDEN=1; verified locally
(bats copy-catalog now green).
* fix(installer): drop obsolete _ensure_helm_prereqs — Tier 0 no longer blocks on openssl (Bugbot #396)
get-helm-3 is gone (replaced by a sha256-verified direct download), so Helm
no longer needs openssl. The Tier-0 preflight still demanded openssl+tar and
failed fast, aborting minimal-image installs for a dependency Helm doesn't use
— and pre-empting _ensure_unpack_tools, which installs tar/gzip Tier-0-aware.
Remove the function and its Tier-0 call; drop the tests that encoded the old
openssl preflight (tar/gzip stays covered by _ensure_unpack_tools' own tests).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
… client) (#381)
* feat(install): prepare-host — one-time admin step so researchers install unprivileged (#1178)
Adds the admin-only prepare-host path: installs the container runtime +
prerequisites and grants a named non-admin user docker-group access, so they
later install tracebloc at Tier 0 with no root. Rebased onto the current
feat/lpi-tier0 (which now targets develop after routing #374 merged); collapsed
from the prior 4-commit history to avoid replaying an intermediate that carried
resolved conflict markers.
Incorporates the three Bugbot fixes from that history:
- reap the sudo keepalive on EXIT (no lingering background sudo -v loop);
- grant docker-group ONLY to the explicit TB_PREPARE_USER, never $SUDO_USER
(the admin who ran prepare-host);
- only print the "researcher can now install with no admin" message when a
docker-group grant actually succeeded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* prepare-host: sudo-verified engine gate; never sg-re-exec or grant the admin (Bugbot)
install_docker_engine's tail assumed the invoking user is the end-user:
on the documented non-root prepare-host path it added the ADMIN to the
docker group and the sg re-exec re-ran the script WITHOUT the
prepare-host argument -- a silent FULL provision as the admin -- and a
socket-less admin aborted before the TB_PREPARE_USER grant. In
TB_PREPARE_HOST_MODE the daemon check runs via sudo, the group-add and
re-exec are skipped, and only the researcher named by TB_PREPARE_USER is
ever granted the socket (same least-privilege rule as #377). Two bats
regressions pin both escapes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(prepare-host): reach the sub-script on healthy machines + terminal active-daemon corner (Bugbot r2)
Two follow-ups on the prepare-host re-land:
- install.sh: prepare-host/--prepare-host now skip the healthy-install
bailout — the argument is useful precisely on a machine that is already
set up (grant ANOTHER researcher docker-group access), and the bailout
would open the home screen and silently skip the TB_PREPARE_USER grant.
It skips ONLY the bailout: the force flag is split out (_tb_force), so
host-prep is never mislabeled as a forced reinstall — a stale sub-script
without the prepare-host dispatch would otherwise treat the run as a
full forced provision.
- setup-linux.sh: in prepare-host mode, a daemon that systemd reports
ACTIVE but that doesn't answer 'sudo docker info' is now terminal with
prepare-host-appropriate guidance, instead of falling through to the
docker-group 'log out and back in' abort meant for regular users.
Regression tests: bootstrap bats stamps DEFAULT_REF like the release
pipeline (assignment line only) to prove prepare-host passes the bailout
with TB_FORCE_REINSTALL unset; setup-linux bats covers the active-but-
unresponsive daemon corner. 16/16 + 58/58.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(prepare-host): terminal daemon-down path + trim TB_PREPARE_USER (Bugbot r3)
- Daemon DOWN in prepare-host mode: start it via sudo (starting Docker IS
host preparation), re-verify, and stay terminal in prepare-host wording
on every failure — the shared diagnostics end with 're-run this
installer', which for the admin means a full provision as themselves,
the exact outcome prepare-host exists to prevent. Reboot-required case
gets prepare-host wording too.
- TB_PREPARE_USER is trimmed before the non-empty gate and the grant, so
a pasted value with stray spaces doesn't fail usermod while skipping
the honest no-grant messaging.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(prepare-host): dispatch at any argument position (Bugbot r4)
install.sh's bailout exemption scans all of "$@" for prepare-host, but
the sub-script only dispatched on $1 — so '--force prepare-host' skipped
the bailout, exported the force flag, and then fell through into a FULL
FORCED provision as the admin. The dispatch now scans every argument,
matching the bootstrap's contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(prepare-host): enable docker on boot in the already-running path (Bugbot r5)
Running NOW isn't host-prep: after a reboot the Tier-0 researcher can't
start the daemon themselves. The success path now ensures docker.service
is enabled on boot (best-effort), matching the fresh-install and
daemon-down recovery paths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(prepare-host): don't let diagnostics pipelines abort before the guidance (Bugbot r6)
systemctl status exits 3 for an inactive unit (and the shared block's
grep exits 1 on no match) — under set -e -o pipefail the failing
diagnostics pipeline aborted BEFORE the terminal error message, a silent
death with no re-run guidance. Both the prepare-host block and the shared
daemon-down block now guard the pipeline with || true; the bats stub
returns the real rc 3 to exercise the path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…ler (#401)
* fix(installer): make the curl TLS floor structural, not per-call-site (backend#1252)
`CURL_SECURE` was a bare constant every call site had to splice in by hand, so
call sites kept losing it: seven live `curl` invocations ran with no minimum TLS
version, including the POST in `verify_credentials()` that carries the client's
password. These installs run on customer-managed hosts and behind TLS-inspecting
proxies, which negotiate down to whatever the client permits — the reason this
repo adopted an explicit floor instead of trusting curl's defaults.
Add `curl_secure()` in `scripts/lib/common.sh` and route every fetch in
`scripts/lib/*.sh` through it (18 call sites). The wrapper always passes
`--tlsv1.2` and supplies default `--connect-timeout 30` / `--max-time 300`.
Defaults are injected before `"$@"`, so a call site that wants a tighter bound
still wins (curl honours the last occurrence), and a transfer that bounds itself
with `--speed-limit`/`--speed-time` gets no injected `--max-time` — a hard
deadline would fail a slow-but-healthy link on a large binary download. Every
existing site therefore keeps its effective behaviour; seven gain the floor and
nine previously unbounded ones gain a deadline.
Also fixed while here:
- `gpu-amd.sh` had the least-bounded curl usage in the repo — no TLS floor, no
timeout, no retry. Both calls now go through the wrapper; the `.deb` download
is retry-wrapped. The listing scrape deliberately is not: `retry()` reports
attempts on stdout, which is that function's return value.
- `install-k8s.ps1`'s WSL2 here-string had the same two nvidia-container-toolkit
fetches bare. It cannot source `common.sh`, so it spells the flags out inline
the way the bootstrap does.
`scripts/install.sh` keeps its seven hardcoded literals: it is the trust root
that fetches `common.sh`, so it cannot source the wrapper.
`CURL_SECURE` stays defined and unchanged for out-of-tree callers, but nothing
in the repo reads it now — the wrapper names the flag itself, so the constant can
never silently reshape every fetch in the installer.
Enforcement: an INTERIM third check in `scripts/check-style.sh` fails on a bare
`curl`. tracebloc/.github#65 already implements this properly (a shell-aware
lexer, not a grep) in a shared reusable workflow, but that workflow is not on
`main` yet and cannot be referenced from here until it is. The check is marked
for retirement the moment this repo adds that caller.
Regenerated `scripts/manifest.sha256` (R8 supply-chain gate).
Found in #399.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): stall-bound the kubectl fetch, don't give it a deadline (Bugbot)
`_fetch_kubectl` had no time bound at all, so routing it through `curl_secure`
handed it the wrapper's default `--max-time 300`. kubectl is a ~50 MB binary, and
this repo already documents (at `_fetch_k3d_release`, same file) that a fixed
ceiling fails a slow-but-healthy link at that size — so the wrapper would have
made every retry fail where the fetch previously completed.
Give both fetches the same `--connect-timeout 15 --speed-limit 1024
--speed-time 60` as the k3d pair. That is also how `curl_secure` knows to skip its
default deadline, and it is strictly better than before: the fetch was previously
unbounded in both directions, so a mid-stream stall hung the step indefinitely.
Audited the other 7 sites that now inherit the 300s default — get.docker.com,
get-helm-3, the Homebrew script, stable.txt, the DMG checksum, the device-plugin
manifest and the amdgpu-install package are all small text/script payloads. The
only large downloads in the repo are kubectl, k3d and the Docker Desktop DMG; the
latter two were already stall-bounded.
Adds a bats test pinning it, since nothing covered `_fetch_kubectl` before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): stop the ROCm package lookup from aborting the installer
`_find_package_name` ran `curl … | grep … | head -1` as a single pipeline and
returned its status. `install-k8s.sh` sources this lib under `set -euo pipefail`,
so that pipeline could kill the installer two different ways:
1. A failed fetch (404, timeout, proxy block) made the command substitution
non-zero, the caller's assignment inherited it, and `set -e` aborted BEFORE
the friendly `[[ -z "$name" ]] && error "No amdgpu-install …"` on the next
line could run. The user got a silent abort mid-GPU-step instead of an
actionable message, and the RHEL major-version fallback was unreachable for
the same reason.
2. `head -1` can close the pipe while grep is still writing, so grep takes
SIGPIPE (141) and `pipefail` propagates that as a pipeline failure even
though a filename WAS found. It only triggers when the directory index
exceeds the pipe buffer, so it fails on large mirrors only.
Capture the fetch and the match separately and let neither fail the function,
then take the first match with `${var%%…}` so `head` leaves the pipeline
entirely. The contract is unchanged — filename on stdout, nothing when not
found — so emptiness remains the single signal all three callers already test.
Adds scripts/tests/gpu-amd.bats (first coverage for this lib): the contract,
both hazards, and a caller-shaped regression test under `set -euo pipefail`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: re-trigger standard-checks after base retarget to develop (#401)
Retargeting the PR base from #400's merged branch to develop doesn't fire a
pull_request event, so standard-checks (Unit tests + Lint) never ran on this
head. Empty commit fires synchronize so the required checks run against develop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Cursor Bugbot reviews this repo with zero project context today — no repo
in the org has a BUGBOT.md. On `client` Bugbot writes ~21.6x the human
inline review volume, so it is effectively the whole code review here;
every house rule it has to re-derive is a pass it doesn't spend on a hard
finding.
Encodes the invariants that recurring findings actually cluster on, each
with the reason and a real reference in this repo: the `$CURL_SECURE`
TLS floor being a constant rather than a wrapper (so new curl calls lose
it silently), kubectl/helm/curl timeout conventions, version-before-URL
validation, fail-closed guards (`detect_installed_client`), never
reporting success you haven't verified (SEAL-CHECK), Helm nil-guards on
new values keys across both `--reuse-values` and
`--reset-then-reuse-values`, the `resource-policy: keep` stored-manifest
trap that cost a PVC set, digest pinning incl. the prod-overlay CI blind
spot, and the R8 manifest regeneration step.
Also records verified non-issues so Bugbot stops re-reporting them —
most importantly that `scripts/lib/*.sh` omit `set -euo pipefail` by
design (sourced after install-k8s.sh sets it), the deliberate `set -uo
pipefail` guard scripts, and the known cross-file SC2034 false positive.
Item 4 of tracebloc/backend#930; improves on RFC 0001's Appendix A draft.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…oc/backend#930) (#402)
GitHub auto-closes an issue in another repository only when the PR body names
it owner-qualified. A bare `repo#N` merely cross-references and closes nothing
-- and the template's own hint taught `Ref tracebloc/other-repo#456`, which is
not a closing keyword at all.
Eight code-complete issues stayed open for days-to-weeks this way
(tracebloc/backend#1171-#1176, #376, tracebloc/cli#393),
dragging two epics to 0% and 14% when the true figures were 67% and 24%.
Someone had to notice and close all eight by hand.
Makes the qualified form the visible example in Related, and adds one
checklist item. No other changes -- a template nobody reads because it grew
is worse than the bug.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…back-link (#403)
RFC numbers are assigned per repo, so a bare "RFC 0001" names four different
documents across the org. Add the qualified ID (`RFC-CLIENT-0001`) to the
header, in the bold-label style this document already uses, and give it an
explicit `Draft` status.
Also repoint the `**RFC:**` back-link. It pointed at
`./0001-least-privilege-install.md`, which (a) does not exist on develop and
(b) collided with this very file's number. That document is being renumbered
to `0002-least-privilege-install.md` in #369, so the link now targets 0002.
The link is dead either way until #369 merges — it is dead on develop today.
If this lands first it stays dead for the gap; if #369 lands first it resolves
immediately. No ordering requirement.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…lly (#404)
The post-upgrade Helm hook (#380) coupled a one-time, best-effort data
migration to every chart upgrade, which repeatedly surfaced timing and
--timeout edge cases. The backfill is a one-time-per-cluster operation, so
run it manually as a standalone Job instead. The tracebloc-backfill runner
(data-ingestors #393/#395) is unchanged and remains the mechanism.
Removes:
- templates/metadata-backfill-hook.yaml
- tests/metadata_backfill_hook_test.yaml
- metadataBackfill block in values.yaml + values.schema.json
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadscripts/lib/probe.sh
Comment threadscripts/lib/setup-linux.sh
…e-local reuse) (#405)
* fix: Bugbot findings from promotion PR #383 (helm probe timeout + node-local reuse)
A) Ungated Helm in client probe: Get-InstalledClientInfo ran helm list/get with
no bounded kubectl probe, so a wedged API server hung Step 4/5 (helm has no
request timeout). Add Test-ApiReachable (5s-bounded kubectl probe, mirrors
Get-TrainingResources) and gate the enumeration behind it — an unreachable
API now degrades to ListUnknown instead of hanging.
B) Reuse ignores node-local data: under TB_STORAGE_MODE=node-local there is no
/tracebloc host bind-mount, so "reuse = keep and adopt" is false. Make the
leftover-guard prompt option and the reuse branch honest under node-local
(data left on disk, NOT adopted; cluster starts empty in-node). Hostpath
behavior unchanged.
Regenerate scripts/manifest.sha256 for the cluster.sh + install-k8s.ps1 changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: node-local leftover-guard follow-ups from Bugbot on #405
A) Keep reply rejected under node-local: the node-local prompt shows "[r] keep",
but the parser only accepted r/reuse, so typing "keep" aborted the install.
Lowercase the reply (tr, bash 3.2-safe) and accept r/reuse/k/keep -> reuse.
B) Non-interactive reuse still claimed adopt: the no-TTY recovery guidance
advertised --reuse-data as "adopt the existing data" for every mode. Make it
storage-mode-aware — under node-local it keeps data on disk, NOT adopted
(cluster starts empty in-node), matching the interactive reuse branch.
Regenerate scripts/manifest.sha256 for the cluster.sh change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0331802. Configure here.

Comment thread.cursor/BUGBOT.md Outdated
Comment thread.cursor/BUGBOT.md Outdated
shujaatTracebloc pushed a commit that referenced this pull request Jul 27, 2026
…ugbot #383
A) Curl TLS section described CURL_SECURE as a hand-spliced constant with a stale
"already missing" list. The real rule is the curl_secure() wrapper in
common.sh, enforced by check-style.sh rule 3 ("no bare curl"). Rewrote the
bullet to flag bare curl bypassing curl_secure(), with the real exemptions.
B) Prod overlay section told reviewers CI ignores client/values-prod.yaml and to
flag overlay digest edits. That overlay was deleted; the fleet-wide prod pin
moved to the chart default images.ingestor.prodDigest, which
ingestor-multiarch (helm-ci.yaml) reads and hard-fails on (empty or
single-arch). Rewrote to the real contract.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…version + Windows leftover-data guard) (#406)
* fix: Bugbot findings from promotion PR #383 (probe timeout + kubectl version + Windows leftover-data guard)
A) Probe hangs on wedged Docker (High): _probe_runtime_usable ran bare
`docker info` (no timeout) on every install, so a wedged daemon hung a
headless SSH install forever. Bound it with timeout/gtimeout when present
(fallback to bare call); read-only and never fatal.
B) Retry notices corrupt kubectl version (High): KUBE_VER captured retry's
stdout notices, polluting the version and breaking the download URL. Isolate
it the way the Helm resolver does — tail -1 + tr + a version-tag regex that
fails closed on an unresolvable value.
C) Windows missing leftover-data guard (Medium): New-K3dCluster silently
adopted prior data. Port guard_leftover_data to PowerShell (detect flat +
per-release layouts, prompt reuse/wipe/new/abort, honor TB_LEFTOVER_ACTION /
HOST_DATA_DIR / TRACEBLOC_SKIP_LEFTOVER_GUARD, non-interactive fail-safe
abort). Windows is hostpath-only (node-local has no Windows path). Refactored
HOST_DATA_DIR validation into a shared Confirm-DataDir.
Regenerate scripts/manifest.sha256 (probe.sh, setup-linux.sh, install-k8s.ps1).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: cross-platform Pester paths + nested-junction-safe wipe (#406 follow-ups)
1) Pester (ubuntu-latest) failed: the new leftover-data-guard tests hardcoded
Windows '\' separators, which are literal chars (not separators) under Linux
pwsh, so Get-LeftoverDataDirs (Join-Path -> '/') never matched the expected
paths. Build all test paths with Join-Path / [IO.Path]::Combine so they pass
on both Windows and Linux pwsh. Behavior/coverage unchanged.
2) Wipe could follow nested junctions (Bugbot r3655703571): Remove-LeftoverData
guarded only the TOP-LEVEL reparse point, then `Remove-Item -Recurse`, which
on Windows PowerShell 5.1 descends into nested junctions and can delete
targets OUTSIDE HOST_DATA_DIR. Add Remove-TreeNoFollow (rm -rf semantics):
walk depth-first and unlink any reparse point without descending, so a nested
junction is removed but its target is never touched. Add a Pester test for it.
Regenerate scripts/manifest.sha256 for the install-k8s.ps1 change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: refresh stale BUGBOT.md guidance (curl_secure + prodDigest) — Bugbot #383
A) Curl TLS section described CURL_SECURE as a hand-spliced constant with a stale
"already missing" list. The real rule is the curl_secure() wrapper in
common.sh, enforced by check-style.sh rule 3 ("no bare curl"). Rewrote the
bullet to flag bare curl bypassing curl_secure(), with the real exemptions.
B) Prod overlay section told reviewers CI ignores client/values-prod.yaml and to
flag overlay digest edits. That overlay was deleted; the fleet-wide prod pin
moved to the chart default images.ingestor.prodDigest, which
ingestor-multiarch (helm-ci.yaml) reads and hard-fails on (empty or
single-arch). Rewrote to the real contract.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@shujaatTraceblocshujaatTracebloc added the skip-fr-gate Bypass FR gate for this PR (use only for bootstrap or emergencies — visible in audit) label Jul 27, 2026
@saadqbal
saadqbal merged commit 2b64d60 into mainJul 27, 2026
143 of 147 checks passed
LukasWodka added a commit that referenced this pull request Jul 28, 2026
…ase now carries the prod pin
The path-1 assertion hardcoded the pre-#398 era: it expected NO ingestor
pin after a --reuse-values upgrade because the published release's
computed values predated images.ingestor.prodDigest. The #383 promotion
(2026-07-27) published a release that includes the pin, so replayed
computed values now carry it and the assertion fails on every
chart-touching PR (first hit: #454).
The expectation is now read from the baseline release itself
(helm get values --all): pin absent => must not arrive (old behavior);
pin present => the SAME digest must be replayed verbatim. Both eras keep
asserting the actual limitation: --reuse-values never injects new chart
defaults. Path-2's era-stale comment corrected, with a note on the
replay-contamination signal a future pin bump will surface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka added a commit that referenced this pull request Jul 28, 2026
…the path-2 tripwire in #459 (review)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka added a commit that referenced this pull request Jul 28, 2026
…the path-2 tripwire in #459 (review)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka added a commit that referenced this pull request Jul 28, 2026
…ge prod image (#454)
* fix(chart): pin mysql-client by digest — keep fleets on the 5.7-lineage prod image (backend#723)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(e2e): era-aware --reuse-values pin assertion — the published release now carries the prod pin
The path-1 assertion hardcoded the pre-#398 era: it expected NO ingestor
pin after a --reuse-values upgrade because the published release's
computed values predated images.ingestor.prodDigest. The #383 promotion
(2026-07-27) published a release that includes the pin, so replayed
computed values now carry it and the assertion fails on every
chart-touching PR (first hit: #454).
The expectation is now read from the baseline release itself
(helm get values --all): pin absent => must not arrive (old behavior);
pin present => the SAME digest must be replayed verbatim. Both eras keep
asserting the actual limitation: --reuse-values never injects new chart
defaults. Path-2's era-stale comment corrected, with a note on the
replay-contamination signal a future pin bump will surface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(e2e): fail fast when jq is missing (review: local runs, not just CI runners)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(e2e): one era boundary, stated once (the #383 promotion); track the path-2 tripwire in #459 (review)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(chart)+test: single-arch pin note + render assertion for the digest path (review)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
LukasWodka added a commit that referenced this pull request Jul 28, 2026
…ow carries the prod pin (#455)
* fix(e2e): era-aware --reuse-values pin assertion — the published release now carries the prod pin
The path-1 assertion hardcoded the pre-#398 era: it expected NO ingestor
pin after a --reuse-values upgrade because the published release's
computed values predated images.ingestor.prodDigest. The #383 promotion
(2026-07-27) published a release that includes the pin, so replayed
computed values now carry it and the assertion fails on every
chart-touching PR (first hit: #454).
The expectation is now read from the baseline release itself
(helm get values --all): pin absent => must not arrive (old behavior);
pin present => the SAME digest must be replayed verbatim. Both eras keep
asserting the actual limitation: --reuse-values never injects new chart
defaults. Path-2's era-stale comment corrected, with a note on the
replay-contamination signal a future pin bump will surface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(e2e): fail fast when jq is missing (review: local runs, not just CI runners)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(e2e): one era boundary, stated once (the #383 promotion); track the path-2 tripwire in #459 (review)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-fr-gateBypass FR gate for this PR (use only for bootstrap or emergencies — visible in audit)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@shujaatTracebloc@saadqbal@LukasWodka@divyasinghds