Uh oh!
There was an error while loading. Please reload this page.
release-train: develop -> staging - #869
Merged
Merged
Conversation
#859) * sec(2571): resolve clientId/clientPassword from the existing Secret Both were `required` in the template AND in values.schema.json's top-level `required` with `minLength: 1`. So they had to be supplied as values on every install and every upgrade -- which meant they were necessarily written into the Helm release's user-supplied values, in cleartext, in EVERY retained revision (10 by default). Rotating the credential did not clear the older revisions, and anyone with `get secret` in the namespace could read all of them. Measured: 3 fleets x 10 revisions. They are now resolved in three tiers, the shape this file already uses five times over (podTokenSigningSecret, credmgrPassword, tbMetaPassword, tbIngestPassword, bootstrapDbPassword): 1. explicit values, else 2. the CLIENT_ID / CLIENT_PASSWORD keys already in the live Secret, else 3. fail, naming both remedies. Tier 3 is a HARD FAILURE, not `randAlphaNum`: the backend issues these, so the chart must never invent one. An operator can now pre-create the Secret, or install once with values and drop them afterwards, and the credential never enters release values. THE SCHEMA WAS THE REAL ENFORCEMENT. Changing only the template was a no-op -- JSON Schema validation runs BEFORE rendering, so an absent or empty value was rejected before `lookup` could fire and the template's own refusal was dead code. Removing clientId/clientPassword from `required` and dropping `minLength: 1` is what actually makes tier 2 reachable. Enforcement moves to the template, which is the only layer that can see the Secret; placeholder rejection stays in the schema and is now also applied to the RESOLVED value, so a badly pre-created Secret is caught too. Tests. The unit suite CANNOT observe tier 2: `lookup` is inert under helm-unittest, and deleting the whole tier-2 branch leaves all 30 unit tests green. So the mechanism is covered where it can be -- a new path 5 in scripts/tests/e2e-auto-upgrade.sh upgrades a real k3d install with `--reset-values` (every user value discarded, so the credentials are genuinely absent) and asserts they survive in the Secret. If they do, the lookup is the only thing that could have supplied them. Also fixes vacuous assertions found while doing this: * `failedTemplate` with `errorPattern` is SILENTLY IGNORED by helm-unittest 0.5.2 -- a pattern appearing nowhere in the output still passes. Proven by mutating two messages that such tests claim to assert: 49 tests stayed green. Converted this file's two to `errorMessage` (honoured, exact) and re-ran the same mutation: it now reddens. Six more live in five other test files and are filed separately rather than dragged into this PR. * "should reject empty credentials" was passing on SCHEMA validation, not on anything in secrets.yaml -- its bare `failedTemplate: {}` could not tell the difference, and this change moved which layer refuses it. * Placeholder tests stay bare, deliberately: schema failures surface as a chart-load error that no message assertion can match. Commented rather than dressed up. Evidence: 579/579 unit tests pass; helm lint clean; offline render without credentials fails with the tier-3 message at secrets.yaml:40; with credentials renders 56 documents carrying CLIENT_ID/CLIENT_PASSWORD. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * sec(2571): a pre-created Secret is adopted, an empty key is not a credential Three Bugbot findings on #859, all real. Each one is a consequence of the same change -- dropping `required`/`minLength` so tier 2 could run -- so they are fixed together rather than singly. 1. HIGH: the preferred remedy could not work on a first install. secrets.yaml ALWAYS emits `<release>-secrets`, so the values.yaml advice to `kubectl create secret` before installing produced a Helm ownership error, and the tier-3 message named that same broken remedy. Helm adopts a pre-existing object carrying its three ownership fields, so values.yaml now gives the label + annotate commands alongside the create, says the names must match the install exactly, and leads with the simpler install-once-then-drop path. The fail message points at it. 2. HIGH: dropping clientId from values hid the live client. detect_installed_client read "values readable, no clientId" as NOT A CLIENT -- true while clientId was `required`, false the moment this chart told operators to drop it. The one-client guard compares on a non-empty id, so a client installed the new way was invisible and a re-run could re-point the machine. The id is now read from the release Secret when values do not carry it, and a client-chart release naming an id in NEITHER place is UNKNOWN (fail closed), not absent. Same fix in the PowerShell peer, which had the identical null-clientId `continue`. 3. MEDIUM: an empty Secret key counted as resolved. Tier 2 keyed off `hasKey`, so `CLIENT_ID: ""` resolved and helm shipped blank credentials -- the case `minLength: 1` used to catch before it had to come out. Tier 2 now tests the DECODED value, so an empty key reads as absent and falls to tier 3. TESTS. detect_installed_client's own scanning loop had no test at all -- every existing test stubs the function out, which is why nothing caught (2). Four added, and each is mutation-proven: removing the Secret fallback reddens 2 of them, dropping the fail-closed reddens the other 2, and restoring returns all four green. The empty-key path is unobservable to helm-unittest (`lookup` is inert there, as this PR already documents), so it is asserted in e2e-auto-upgrade.sh where tier 2 is real: blank the key, upgrade, require the failure AND require it to name clientId. Chart.yaml 1.9.72 -> 1.9.73 (version + appVersion) for the version-bump gate; manifest.sha256 regenerated for the two installer scripts. Verified: helm-unittest 579/579, make lint 0, make drift 20/20 guards green, helm-lint/vocab/template clean, the 4 new bats tests green + both mutations. NOT verified locally: Pester and the full bats file both hang on this Mac at `DEFAULT (TB_STORAGE_MODE unset)` -- reproduced on the UNMODIFIED PR head, so it is environmental, and neither runs on darwin in CI (ubuntu + windows only). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(secrets): name the renderer tier 2 cannot reach, and what it costs there Saqlain's third thread on #859, the one no code change answers: tier 2 is a `lookup`, so it needs a live cluster. The template already said that about `helm template`/`--dry-run`; it did not say the same limit applies to every CLIENT-SIDE RENDERER -- ArgoCD's default renderer and Flux post-render among them -- and that is where it actually bites someone. The asymmetry is the part worth writing down. The chart's other five credentials survive an inert `lookup` because their tier 3 is `randAlphaNum`, so a value-free render degrades to a generated secret. clientId/clientPassword cannot: the backend issues them, so tier 3 is a hard `fail` and the render STOPS rather than degrading. A GitOps install therefore has to keep both in its values -- which means the cleartext-in-revisions problem this PR fixes is NOT fixed for that deployment style. Said plainly in both places an operator reads, because a limitation discovered at install time is a support ticket: values.yaml (planning) and secrets.yaml (the mechanism). Both name what makes the value-free path reachable -- render against a live cluster -- and secrets.yaml names the out-of-scope remedy for the rest (an external secrets operator writing the Secret before render) so the gap is bounded rather than open. Docs only; no template logic changes, so the resolution tiers and their tests are untouched. Verified: make check green (parse, shellcheck, drift 20/20, helm-lint, chart-env-vocabulary 50/50) and helm-unittest 579/579 on this tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(installer): the Pester peer asserted the contract this PR inverted Pester went red on both ubuntu-latest and windows-latest at e0981f4 and it was a real regression, not the environment. The commit that fixed Bugbot's fail-open finding changed detect_installed_client's contract and left the PowerShell test asserting the OLD one: [-] values without a clientId key do not trip the guard That test read: a readable client release carrying no `clientId` is NOT a client, so the installer upgrades over it. True only while `clientId` was `required`, which made a clientId-free client release impossible. This PR drops that requirement and tells operators to remove clientId from values once the Secret holds it -- so the test was pinning the exact fail-open the finding was about. Both earlier attempts at this fix missed it, because Pester does not run on darwin. REPRODUCED LOCALLY FIRST, rather than inferred from the CI log. pwsh 7.5.2, filtered to the one test: the installer prints "Refusing to replace an unidentifiable existing client" and refuses -- which is the new contract working exactly as intended, failing a test that wanted the old one. The test is rewritten to the new contract, not the fix reverted: * renamed to what it now checks -- fails CLOSED on a readable client release with no clientId in values or Secret. `kubectl` has no cluster under Pester, so Get-ClientIdFromSecret returns "" and the both-places-empty path is what gets exercised, which is the case worth pinning. * NAMES THE REFUSAL (CLAUDE.md rule 10). Every other fail-closed path in this Describe also throws, so a bare `Should -Throw` would have passed on the wrong refusal -- an unreadable-values or garbage-list abort is indistinguishable from the one the test is named for. It asserts `Should -Invoke Err -ParameterFilter { $m -match 'unidentifiable existing client' }` as well as the throw and the absent upgrade. * carries the why above it, so the next reader does not "fix" it back. MUTATION-PROVEN, and the anchor was asserted rather than assumed: restoring the pre-fix `if ($null -eq $vals -or $null -eq $vals.clientId) { continue }` reddens it (PASSED=0 FAILED=1); restoring the fix greens it. The mutation script aborts if its anchor does not match, so an inert mutation cannot read as coverage. Also dropped the trailing bare `continue` e0981f4 added. It was the last statement of the loop body, so it bought nothing -- and PowerShell reported it escaping as an unmatched loop label (pester/Pester#2669), which aborts the whole run rather than failing one test. The two `continue`s above it are load-bearing and stay. manifest.sha256 regenerated for install-k8s.ps1. Verified on this tree: full Pester suite 784 total / 770 passed / 0 failed / 14 skipped (was 1 failed), and `make check` green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(installer): bound the Secret read that is now on the common path Bugbot Medium on #859, and it is right that this PR is what makes it matter. `_client_id_from_secret` / `Get-ClientIdFromSecret` issue a fresh `kubectl get secret` with no `--request-timeout`, and kubectl's default is no timeout at all -- it waits forever. WHAT CHANGED THE RISK. Before this PR the fallback did not exist; the id was always in release values. This PR tells operators to DROP clientId from values, so once they do, every scanned client release reaches this call on every detect_installed_client / Get-InstalledClientInfo. That runs in the pre-provision pre-flight and the Helm-step one-client guard, both of which can run headless -- so a wedged API server hangs the install with no further output and no way to tell what it is waiting on. The enumeration around it is already bounded (Test-ApiReachable, and the `helm list` gate that documents exactly this hazard); this one call was the hole left in it. --request-timeout=5s on both twins. 5s is this repo's existing figure for a cheap existence probe rather than a new number -- install-k8s.ps1 already uses it for the namespace, daemonset and allocatable-GPU reads. A TIMEOUT LANDS IN A PATH THAT ALREADY EXISTS, which is why this is a one-flag change: kubectl exits non-zero, which both twins already treat as "could not read" (`|| return 0`, `return ""`). The caller turns that into an UNIDENTIFIABLE client, not an absent one, so a timeout fails CLOSED -- the guard refuses rather than waving through an install that re-points a live machine. The fail-open this PR set out to close is not reopened by the fix for it. Fixed in both languages. The bash and PowerShell readers are twins by design (they name each other), and fixing one would split them -- the divergence class backend#2220 found five of. VERIFIED THE STUBS STILL MATCH, because that was the real risk of adding an argument. install-client-helm.bats:875 keys its kubectl stub on POSITION -- `[ "$5" = "liverel-secrets" ]` -- so a flag inserted mid-vector would have silently broken three tests. The flag is appended last, leaving $1..$7 intact; exercised against a copy of that exact stub, `_client_id_from_secret liverel munich` still returns `uuid-from-secret`. Both no-kubectl and non-zero-kubectl (the timeout shape) still return empty with rc 0, as the contract requires. manifest.sha256 regenerated for both scripts. Verified: full Pester 784 total / 770 passed / 0 failed / 14 skipped; `make check` green; `bash -n` + `shellcheck -S warning -x` clean on the bash twin; install-k8s.ps1 parses clean. The full bats file hangs on this Mac before reaching these tests (reproduced on the unmodified PR head by a previous session, and bats runs on ubuntu in CI, not darwin) -- hence the direct stub-fidelity check above rather than a claim I did not make. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…kend#2620) (#860) * fix(cronjobs): a failing tick must leave a Pod somebody can read (backend#2620) Both of the chart's CronJobs used `restartPolicy: OnFailure`. Kubernetes DELETES an OnFailure Job's Pod once `backoffLimit` is exhausted, and `failedJobsHistoryLimit` retains the JOB — it cannot retain a Pod that no longer exists. So a repeatedly-failing CronJob leaves a row of `Failed` Jobs and no logs at all. The upstream docs say it outright: "your Pod running the Job will be terminated once the job backoff limit has been reached. This can make debugging the Job's executable more difficult. We suggest setting restartPolicy = 'Never'". MEASURED, NOT THEORETICAL. A customer prod edge's auto-upgrade had been failing hourly for 2.7 days (backend#2620). Five `Failed` Jobs were retained; every one of their Pods was gone and the namespace event window had rolled past the first failure, so the reason was UNRECOVERABLE FROM THE CLUSTER. The only surviving evidence that anything was wrong was the CronJob's `lastSuccessfulTime`, which nothing watches. Each Job failed 39 seconds in, so this was never a timeout — the logs would have said what it was. BOTH CronJobs, not just the one that broke. `image-refresh` carried the same policy and is on a 15-minute schedule, so it had the same blind spot with four times the frequency. Fixing only the instance that happened to fail would have left the class. `scripts/tests/cronjob-failures-are-readable.sh` sweeps every CronJob out of the RENDERED manifests, so a third one is covered the day it lands rather than when someone remembers. It holds no list of CronJob names. Fails closed twice: zero CronJobs rendered is a refusal, not a clean sweep, and a template declaring `kind: CronJob` that no value combination reaches is reported UNREACHED by name. Wired into `DRIFT_GUARDS` (21 entries; the target counts its iterations and refuses to report green on fewer). Scoped to CronJob deliberately, and the argument is at the declaration: a Helm hook Job loses its Pod the same way, but its failure fails the release and is reported to whoever ran it — somebody is already looking. A CronJob's failure is reported to nobody and repeats forever. Four mutations run, each with the baseline restored green afterwards: one template reverts to OnFailure -> FAIL, names the template restartPolicy removed entirely -> FAIL, "silence is the defect" render matrix produces no CronJobs -> FAIL, refuses to check nothing a new CronJob template goes unreached -> FAIL, names it as UNREACHED The two suites that pinned `OnFailure` are updated rather than deleted — they were right to pin it, and the comment now records why the pinned value changed. make helm-unittest: 579 passed, 34 suites. make drift: all 21 guards green. shellcheck clean. Ticket: tracebloc/backend#2620 Parent epic: tracebloc/backend#1872 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(chart): bump to 1.9.74 for the version-bump gate (backend#2620) `chart content ⇒ Chart.yaml version bump` is a REQUIRED check on this repo and was red: the branch changes chart content while `version`/`appVersion` both still read 1.9.73, which is what `develop` carries. The gate exists because the chart's image-refresh CronJob resolves the published tag every 15 minutes, so an unbumped chart is not a cosmetic omission -- nothing downstream can tell the new content from the old. Both fields moved together to 1.9.74, one patch above develop, which is the shape the gate asks for. They are kept EQUAL deliberately: `appVersion` is what the rendered image tags follow, so a bump of `version` alone would advertise a release that pulls the previous images. Verified: `helm lint` clean (only the pre-existing "icon is recommended" note), and the WHOLE chart suite rather than the file touched -- `helm unittest client`, 34 suites, 579 tests, all passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…oses#835) (#852) * fix(installer): wire GPU-capable k3d node image on Linux (client#835) On a Linux GPU host the installer detected the GPU, set the host Docker runtime and deployed the NVIDIA device plugin, but cluster.sh always created the k3d node from stock rancher/k3s — which has no in-node NVIDIA container toolkit. The in-node containerd could never hand the GPU to pods: the device plugin reported "No devices found", the node never advertised nvidia.com/gpu, GPU pods couldn't schedule — yet the installer claimed "GPU access". This brings the Linux path to parity with the Windows/WSL2 twin (which already uses the k3s-cuda image). - cluster.sh: _gpu_node_image derives the ghcr.io/tracebloc/k3s-cuda pull ref (k3s pin + CUDA base, mirror/override aware); _create_new_cluster swaps --image to it when GPU is wired; _generate_node_cdi_specs writes the native CDI spec inside each node (the image is CDI-mode and its boot drop-in only covers WSL2); _check_existing_cluster_gpu falls back to CPU on a reused stock node; the drift check recognises the k3s-cuda tag. - install-client-helm.sh: gate the GPU request + device plugin on GPU actually wired (not bare detection) and set RUNTIME_CLASS_NAME=nvidia. - chart: device plugin runs under runtimeClassName: nvidia so it can init NVML on native Linux (new gpu.devicePlugin.nvidia.runtimeClassName). - summary.sh: "NVIDIA GPU" mode + GPU test hint only when actually wired, so the claim is no longer false on a CPU fallback. - common.sh: TB_CUDA_BASE_TAG pin + _gpu_wired predicate; check-facts.sh enforces the new pin and the GPU --image wiring across both installers. Tests: cluster.bats (+16), install-client-helm.bats (+2), summary.bats (+2), gpu_device_plugin_test.yaml (+3). check-facts, shellcheck (error), helm-unittest and the existing GPU/cluster suites pass. End-to-end GPU-pod scheduling verified via the documented runbook in docker/k3s-cuda/README.md (no Linux GPU host in this environment). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): bump chart to 1.9.72 + refresh manifest for GPU wiring (client#835) Chart templates (gpu-device-plugin runtimeClassName) and values changed, so the chart-version gate requires a version bump; and the installer libs changed, so the signed-bootstrap manifest must be regenerated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): CDI spec presence, not generate exit, gates GPU (client#835) Bugbot High: _generate_node_cdi_specs cleared K3D_GPU_FLAGS (downgrading to CPU) whenever `nvidia-ctk cdi generate` failed — even on a reused cluster whose /etc/cdi/nvidia.yaml already existed from a prior install, or when `docker ps` returned an empty list. Make the spec's PRESENCE the authority: regenerate best-effort, then count a node on `test -s /etc/cdi/nvidia.yaml` (fresh or pre-existing). And treat a node-listing failure as "cannot tell" (leave the request as-is), not "no GPU". +2 cluster.bats cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): pre-pull GPU node image, fall back to CPU if unpullable (client#835) Bugbot High: _create_new_cluster handed the derived k3s-cuda --image straight to k3d with no pullability check, so a blocked/unpublished ghcr.io tag or a private TRACEBLOC_IMAGE_REGISTRY without a docker login would HARD-FAIL cluster create on a host that could still run CPU-only. Pre-pull with the host daemon first (bounded + spinner); on failure drop the GPU request and continue CPU-only with an actionable hint. k3d reuses the cached image, so it is not wasted work. Mirrors the Windows twin's Confirm-GpuImagePullable. TB_SKIP_GPU_IMAGE_PREPULL bypasses it. +2 cluster.bats cases (pull-fail fallback, and the pre-pull itself). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(installer): teach check-facts.bats the new GPU facts (client#835) The check-facts.bats fixture builds a synthetic consumer tree; it didn't carry the additions this PR made to check-facts.sh, so its baseline "everything agrees" case went red (and the wiring-message test pinned the old string). Seed the fixture common.sh with TB_CUDA_BASE_TAG, add the GPU --image pin literal to the fixture cluster.sh + install-k8s.ps1 (the new #835 wiring guards), and update the missing-pin test to assert the generalized ${K8S_VERSION}/$K8S_VERSION hint (it now covers both the k3s pin and the GPU node image). Whole check-facts.bats green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): GPU pre-pull logs into private registry + verifies k3s (client#835) Two follow-on Bugbot findings on the new pre-pull: - High: it never authenticated the host daemon, so credentials the operator set (TRACEBLOC_REGISTRY_USERNAME/PASSWORD) went unused on a private registry and the hint telling them to set those was false. Now `docker login` the image's registry host (via --password-stdin) before the pull — the node image is host-pulled, not kubelet-pulled, so a chart imagePullSecret can't help. New _registry_host_for mirrors the Windows Get-RegistryHost (docker.io for a bare owner/name ref). - Medium: a pulled-but-broken image (mis-tagged override / mirror copy) passed the gate then hard-failed k3d create. Now verify it runs k3s (`docker run --gpus all <img> --version`, capture-then-match) and fall back to CPU otherwise. Mirrors Connect-GpuRegistry + Test-GpuImageRunsK3s. +2 cluster.bats cases; setup mock now answers the verify. cluster.bats 162 green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): strip mirror trailing slash + cover _registry_host_for (client#835) - Bugbot Medium: _gpu_node_image stripped the URL scheme but not a trailing slash from TRACEBLOC_IMAGE_REGISTRY, so https://mirror.corp/ became mirror.corp//… — a double slash that fails the host pre-pull and drops a credentialed GPU install to CPU. Strip trailing slash(es) too, matching the Windows twin's `-replace '/+$',''`. - Review (LukasWodka): _registry_host_for's docker.io branch (a bare owner/name ref must log into docker.io, not the owner segment) was untested — the only caller test used a dotted host, so the naive ${1%%/*} passed too. Add a 4-shape unit test (ghcr.io / mirror.corp / localhost:5000 / owner→docker.io) and a trailing-slash test. cluster.bats 164 green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): flag GPU-inconsistent cluster on the healthy fast path (client#835) Bugbot High: the healthy assess fast path (assess.sh) hands off and exits before the create/reuse GPU reconcile AND before detect_gpu, so a cluster whose live release requests a GPU its node can't schedule — a pre-#835 install that wrote GPU chart values onto a stock rancher/k3s node, or a k3s-cuda node whose device plugin died — kept every GPU job Pending while the control plane looked healthy, with no signal (same gap the drift check already covers on this path). Add _check_healthy_cluster_gpu_consistent: ask the LIVE cluster (does any release request a GPU? does the node advertise one?) rather than GPU_VENDOR, and warn with the recreate remedy on a mismatch. Non-fatal; self-contained + jq-free. Mirrors the Windows twin's Test-HealthyClusterGpuConsistent. +3 cluster.bats cases (167 green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): healthy GPU guard — no temp file, node-image-aware remedy (client#835) Two Bugbot Mediums on _check_healthy_cluster_gpu_consistent: - mktemp failure silently skipped the only path that surfaces this mismatch. Drop the temp file entirely — capture `helm get values` in-memory and grep a here-string. - it always advised "recreate (built before GPU support)" on any node advertising 0, but a GPU-CAPABLE node advertising 0 is a device-plugin/CDI problem recreate won't fix. Inspect the node image: warn recreate only when it's CONFIRMED stock; stay quiet (leave it to the plugin rollout) when capable or unreadable. Mirrors the Windows twin's node-image check. +1 cluster.bats case (168 green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): reconcile GPU keys on the adopt path too (client#835) Bugbot High: _reconcile_adopted_client upgrades with --reuse-values and only healed clientId, so a prior release's env.GPU_REQUESTS/GPU_LIMITS/RUNTIME_CLASS_NAME and its gpu.devicePlugin block survived even after the reuse guard / CDI setup dropped this run to CPU — jobs kept Pending on the live release while the summary said CPU. Force the GPU keys on adopt via --set-string / --set to match this run's decision (the same values the fresh write chooses), mirroring the Windows twin. AMD is left to --reuse-values, except its device-plugin block is reconciled so a stale one can't linger. +2 install-client-helm.bats cases (GPU-wired adopt, CPU adopt). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(installer): make the adopt CPU-fallback test mutation-resistant (client#835) Bugbot: the adopt CPU test only asserted the positive GPU values were absent + the plugin disabled, so deleting the force-empty env.GPU_REQUESTS/GPU_LIMITS/ RUNTIME_CLASS_NAME clears left it green while --reuse-values would keep a prior GPU request. Also assert those force-empty --set-string clears are PRESENT (flag present AND not a GPU value ⇒ forced empty). Mutation-verified: removing the clears now reddens the test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(installer): make the healthy-GPU-guard tests branch-discriminating (client#835) LukasWodka review: two _check_healthy_cluster_gpu_consistent tests asserted only `[ -z "$output" ]`, which is inert — the alloc probe CAPTURES kubectl output into $alloc (never stdout) and the fall-through log doesn't reach $output either, so removing either early-return left the tests green. Assert via record/mock_calls that the skipped probe genuinely didn't run: node-advertises-a-GPU returns before `docker inspect`, and no-GPU-request returns before the kubectl node probe. Mutation-verified: dropping either early-return now reddens the matching test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(chart): bump client chart to 1.9.73 (develop reached 1.9.72) (client#835) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): enumerate all helm statuses in the healthy GPU guard (client#835) Bugbot/#554 house rule: helm list must carry --deployed --failed --pending --uninstalling so a wedged release (which may still request a GPU) is not invisible to _check_healthy_cluster_gpu_consistent. Matches detect_installed_client. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): healthy GPU guard is NVIDIA-specific, no false AMD recreate (client#835) Bugbot: after the backend#2033 merge AMD requests amd.com/gpu on a stock rancher/k3s node (AMD does not use the k3s-cuda image), so _check_healthy_cluster_gpu_consistent — which probes nvidia.com/gpu and treats stock k3s as CPU-only — warned every healthy AMD re-run to recreate a WORKING cluster. Match only an nvidia.com/gpu request; amd and empty no longer trip it. +1 cluster.bats case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): CPU-fallback remedy points to recreate, not a no-op re-run (client#835) Bugbot: after a first-install CDI-gen or image-pull CPU fallback, the remedy said " ix the host and re-run"\ — but the completed CPU client assesses healthy, so the next run fast-paths (assess.sh) and never re-reaches _generate_node_cdi_specs or a Helm rewrite. GPU wiring is fixed at create time, so the honest remedy is to recreate. Both fallbacks now print _recreate_cluster_hint after the host-fix check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…rg builders (backend#2545) (#858) * fix(installer): escape inner quotes in docker-buildx and k3d-create arg builders (backend#2545) #2455 (client#845) fixed CommandLineToArgvW arg-escaping in the shared Invoke-BoundedProcess joiner, but two Start-Process command lines built inline in install-k8s.ps1 kept the naive space-only quoting with the inner `"` unescaped: `docker buildx build` (Build-GpuNodeImage) and `k3d cluster create` (New-K3dCluster). -ArgumentList <one string> reaches the child verbatim like $psi.Arguments, so an arg carrying BOTH a space and a `"` had its inner quotes silently consumed by the OS re-split and merged into the adjacent token. - Delegate both builders to ConvertTo-Win32Arg (the canonical CommandLineToArgvW encoder from #2455), so an arg with a space and a quote survives as one token. - Drop the k3d builder's `@` quote-branch: `@` is not special to CommandLineToArgvW, so a bare `host:node@all` re-splits to the identical single token whether quoted or not -- the change is a no-op there and the fix is escaping the quote the old branch ignored. - Add round-trip tests that push a whitespace+quote arg through each builder's EXACT shipped expression and prove it recovers as one token; regenerate scripts/manifest.sha256. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(installer): pin the backend#2545 decoder to real shell32!CommandLineToArgvW (client#858 review) LukasWodka on #858: the new backend#2545 Describe's Split-Cmdline2545 is the ORACLE the builder round-trip tests trust, but -- unlike the backend#2455 Split-LikeArgvW -- it was only asserted in a comment to "mirror" the real API, never checked against it. If the two from-spec CommandLineToArgvW decoders drift, every encoder test here would validate ConvertTo-Win32Arg against a decoder that no longer matches Windows, and the cross-check on the OTHER copy says nothing about this one. Add the same Windows-gated shell32!CommandLineToArgvW cross-check #845 uses, pointed at Split-Cmdline2545 (distinct namespace TbWin32b so both blocks' Add-Type calls coexist), so this decoder is pinned to the real API too. Keeps the deliberate per-Describe independence (Pester cross-BeforeAll ordering is fragile) while closing the gap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…backend#2621) (#861) * docs(install): write down the release-name convention, and guard it (backend#2621) The installer has always named the release after the namespace — `helm upgrade --install "$TB_NAMESPACE" … --namespace "$TB_NAMESPACE"`, both defaulting to `tracebloc` — and it explains why at the point of decision: the client is identified to the backend by its clientId, "so we don't ask the user to invent one". So the self-service path is consistent by construction. It was written down nowhere. `docs/INSTALL.md`'s only example was `helm install my-tracebloc …`, and `docs/migration-tools/tenant-config.example.env` records the resulting mess as a fact to work around rather than a defect — "The Helm release name (NOT always the namespace name; tenant-a's release is `tracebloc`)". Hand-installed edges therefore diverged, and because HELM CANNOT RENAME A RELEASE, every divergence is permanent: fifteen resource names are prefixed with whatever was typed once, and correcting it means uninstall + reinstall with the downtime and PV re-binding that implies. The concrete case that prompted this: an engineer's given name ends up in `<name>-jobs-manager`, `<name>-auto-upgrade`, `<name>-resource-monitor` on a customer's production cluster, visible to anyone who runs kubectl there. So this documents the convention, including the case the installer does not cover — a multi-tenant cluster, where one namespace per tenant means `release == namespace` still holds — and says plainly what not to do: not a person's name, not a bare environment on a shared cluster, and keep it short because Kubernetes truncates at 63 and the chart appends ~30 of component suffix. AND THE CLAIM IS A MACHINE CHECK, not prose. The new section asserts something about code ("the bundled installer already does this") in the document an operator reads first — which is exactly the shape that decays into advice for behaviour the code stopped having (backend#1729 rule 7). `scripts/tests/release-name-equals-namespace.sh` reads BOTH arguments out of the installer's own invocation and asserts they are the SAME EXPRESSION, whatever that expression is — so renaming the variable keeps it green and passing a different value does not. It holds no copy of the expected name. It also fails if the doc section is deleted, because a guard defending nothing is not a guard. Fails closed: an unreadable installer, a missing invocation, or an unparseable argument are findings, not agreement. Worth recording: the first version of this guard matched a COMMENT mentioning `helm upgrade --install` on line 1025 rather than the invocation on line 2272 — the same "prose satisfies a structural guard" class the file exists to prevent, hit while writing it. It is now anchored to the start of a line, and the comment says why. Three mutations run, baseline restored green after each: installer passes a different release name -> FAIL, prints both values the doc drops the convention section -> FAIL, "protecting nothing" the invocation disappears entirely -> FAIL, refuses to report agreement make drift: all 21 guards green (verified the new one executed). shellcheck clean. NOT INCLUDED, deliberately: `fullnameOverride`. It is filed as tracebloc/backend#2626 with the measurement — 153 `.Release.Name` sites across five classes, of which only two may follow an override, and one of the others feeds `RELEASE_NAME` into `helm status`/`helm rollback` in the auto-upgrade script. A blanket substitution catches that env var on the first pass (mine did) and breaks the auto-upgrade — the exact failure backend#2620 is about. A partial rename is worse than none, so it needs a completeness guard rather than a helper. Ticket: tracebloc/backend#2621 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(2621): the new guard piped into head under pipefail (backend#2621) The repo pipefail early-close check failed on the two parse lines in scripts/tests/release-name-equals-namespace.sh. Under set -euo pipefail an assignment piping into head inherits head SIGPIPE kill once it closes early, so a SUCCESSFUL parse could abort the guard - in a file whose whole point is failing closed on an unparseable installer. Capture-then-slice instead: collect every match with a here-string, then take the first line with parameter expansion. No pipe, so no early-closing reader. Still fails closed - no match leaves the variable empty and the existing -n guards turn that into a finding. Verified at CI severity, not narrower: org-github pipefail-early-close.awk reports 0 findings across every .sh in the tree (was 2), and reintroducing the pipe makes it report the line again. bash -n and shellcheck -S warning -x clean. make check green. The guard still reddens on all four mutations - different release name, changed --namespace, invocation removed, doc claim deleted - and is green restored; every mutation anchor asserted applied. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…ed nothing (backend#2606) (#862) helm-unittest 0.5.2 SILENTLY IGNORES every key under `failedTemplate` except `errorMessage`. Re-measured here before touching anything: `errorPattern: "ZZZ_THIS_STRING_APPEARS_NOWHERE_ZZZ"` on rbac_test.yaml's perDatasetPvcs case ran 19 passed, 19 total. So each of these assertions was exactly equivalent to a bare `failedTemplate: {}` while reading, in review, like a pinned refusal. `--strict` is not a substitute, and that was checked rather than assumed: with `errorPattern` AND an invented `bogusKeyThatDoesNotExist` both set on one assertion, `helm unittest --strict ./client` reported 19 passed, 19 total -- identical to the non-strict run. Two kinds of failure, two remedies: * Template `fail` (2, not the 1 the ticket table listed -- secrets_test.yaml's bootstrapDbReparent case was missed there): converted to `errorMessage` with the template's EXACT full string. Both mutation-proven -- reword the message in the template, 1 failed / 18 passed and 1 failed / 29 passed respectively; restore, 19 passed and 30 passed. * values.schema.json rejection (5): NOT message-assertable at all. The rejection fails at chart load, before any template renders, so `errorMessage` compares against a render error that never happened. These become a bare `failedTemplate: {}` with a comment saying why, so nobody "strengthens" them back into vacuity. Each was probed in the other direction too -- made the values legal, confirmed the assertion reddens -- so a bare assertion is not vacuous in the opposite sense. The guard: scripts/tests/helm-unittest-error-assertions.sh, added to the Makefile's DRIFT_GUARDS, so it runs in `Source-of-truth drift` -- REQUIRED on develop and main. `Helm unit tests` is required on neither, so the guard could only advise from there. It parses the YAML rather than grepping the token, because the token is now named on purpose in several comments warning people off it, and it allowlists `errorMessage` rather than blocklisting `errorPattern`, because a typo (`errorMesage`) fails just as silently. Mutation-proven both ways, plus all three fail-closed paths. 581 tests, 581 passed, before and after. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
LukasWodka
commented
Aug 27, 2026
ContributorAuthor
bugbot run |
…ackend#2253) (#864) * fix(installer): tracebloc upgrade updates a healthy-but-behind CLI (backend#2253) The stop-and-check gate hands a verifiably-healthy machine straight to the home screen ("already set up — no need to run the installer again") and updates nothing. Its CLI floor stops at 0.10.0 (below = mandatory reinstall), but the CLI's own update nudge fires against the latest release — so a CLI at e.g. 0.10.5 with latest 0.10.8 was nagged forever while `tracebloc upgrade` (which re-runs this installer) found the box healthy and changed nothing. Bridge the two definitions without weakening the floor: - assess.sh: a new read-only _assess_cli_behind_latest, gated on TB_UPGRADE_CLI, compares the installed CLI against TB_CLI_LATEST (resolved and passed by the CLI — no network here). classify emits a DISTINCT cli-behind-latest reason, ordered AFTER the floor check so below-floor stays cli-outdated (still a mandatory full reinstall). Inert on every ordinary installer run. - install-k8s.sh main(): on cli-behind-latest, update ONLY the CLI (a small, isolated download via upgrade_cli_only) and exit — no full reconcile. assess stays a read-only classifier; the CLI-install mutation lives in main(). - install.sh: TB_UPGRADE_CLI=1 skips the bootstrap's healthy bailout so the run reaches the gate, WITHOUT forcing a reinstall. The CLI half (setting TB_UPGRADE_CLI / TB_CLI_LATEST) is in tracebloc/cli. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): upgrade_cli_only fails honestly and surfaces cluster drift (backend#2253) Two Bugbot findings on the CLI-only upgrade path: - Failed update no longer reports success. install_tracebloc_cli is non-fatal (written for a client that is already connected), so a failed download would exit 0 and leave the update nag in place while `tracebloc upgrade` looked like it worked. On THIS path the CLI update is the whole job, so verify it: when TB_CLI_LATEST is known and the CLI is still behind it afterward, warn and exit non-zero (telemetry then records failed, not succeeded). - Surface the same k3s-drift (#547/#565) and GPU-consistency (client#835) advisories the healthy hand-off prints — this path also exits before _handle_existing_cluster, so a drifted-but-healthy cluster would otherwise get no signal on upgrade. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
LukasWodka
commented
Aug 27, 2026
ContributorAuthor
bugbot run |
…afe (backend#2625) (#865) * sec(telemetry): release-scope the Collector token Secret, migration-safe (backend#2625) telemetryCollector.tokenSecret.name defaulted to a fixed `tracebloc-telemetry-token` in the SHARED node-agents namespace. Two edges on one cluster collide: two jobs-managers write one Secret, last writer wins, and the loser's Collector authenticates as the wrong tenant — cross-tenant telemetry misattribution. Latent today only because a single Collector is enabled fleet- wide; the collision arrives the moment #1906's prod half enables a second. Option A (migration-safe rename): - Resolve the name in one helper, tracebloc.telemetryTokenSecretName, behind all four consumers (writer env, reader guard + volume, RBAC resourceName), so they can never disagree. Default is release-scoped `<release>-telemetry-token`. - The legacy fixed name is a MIGRATE-ME sentinel, not an override: `helm upgrade --reuse-values` bakes the old default into existing releases' stored values, so only rewriting the sentinel actually migrates them. A genuinely custom name is honoured verbatim. - jobs-manager writes the release-scoped name; the daemonset pre-flight ALSO accepts the legacy name while it exists, so an edge already collecting under the legacy name is not wedged on upgrade (the #2400 deadlock in a new costume). The legacy acceptance and its helper carry, at their declaration, the condition for their own removal. Acceptance: (a) two releases resolve to distinct, release-scoped Secrets — asserted across two renders in telemetry_collector_test.yaml and telemetry-token-agreement.sh; (b) an edge on the legacy name upgrades without the pre-flight tripping — the guard is a lookup-backed `fail`, invisible to `helm template`, so it is exercised live in scripts/tests/telemetry-token-migration.sh (run from the k3d auto-upgrade e2e; self-skips with no cluster); (c) the legacy acceptance declares its own removal condition at the declaration. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(chart): bump version 1.9.74 → 1.9.75 for backend#2625 Chart content changed (telemetry token rename), so the chart-version-guard requires a new Chart.yaml version — a Helm repo only publishes on version change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(telemetry): here-string, not a pipe into grep -q, in the migration guard check The pipefail early-close house-rule flags `printf | grep -q` under errexit+pipefail (the reader closes early and SIGPIPEs the producer). Feed the captured output via a here-string instead — same match, no pipe. (backend#2625) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(telemetry): bound kubectl and make the migration check non-skippable in e2e (backend#2625) Cursor Bugbot (2 findings) on PR #865: - Unbounded `kubectl cluster-info` hangs the k3d job on a wedged API instead of failing fast. Every kubectl call now goes through a `--request-timeout=15s` wrapper. - A skip exited 0, so from e2e-auto-upgrade.sh a skip counted as acceptance (b) passing while the PASS line claimed it was verified. New `--require` flag turns every skip condition into a hard failure; the e2e passes it (a cluster is guaranteed there). Standalone/drift runs still self-skip. Verified live on k3d: three cases green under --require; teardown clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…w (backend#2350) (#867) The auth-proxy E2E shipped RED to staging on client#789 and blocked nothing because it lived in installer-tests.yaml — an expensive, path-filtered workflow whose jobs can never be required status checks (a required check behind a paths filter never reports on a PR outside its paths, so GitHub holds the PR at "waiting for status to be reported" forever; a guard that cannot block is advice, not a gate — backend#1729). Root cause of the red itself was diagnosed and already fixed on develop: the squid probe's curl --retry was inert against curl's negative-DNS cache (#811), and the probe/squid/nginx images are pinned (#813). This change does the remaining enablement so the check can actually gate. Extract the e2e-proxy job into e2e-auth-proxy.yaml, mirroring the move Source-of-truth drift made out of installer-tests' old static job: - pull_request has NO paths filter, so the context always reports and can be required on staging/main without deadlocking a promotion PR that touches no scripts/ file (backmerge, hotfix lane, version-bump promotion). - push keeps a paths filter (pushes aren't gated) and adds staging; the Monday cron is preserved so extracting the job doesn't drop its weekly canary. - job name 'E2E auth-proxy (squid)' is unchanged, so the required-context name and the release train's by-name check reading are preserved. Arming it as a required check on staging + main is a follow-up branch-protection setting, to be flipped after a green-watch window (arm while green). See backend#2350. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
LukasWodka
commented
Aug 27, 2026
ContributorAuthor
REFUSED. This promotion was blocked by 1 HIGH severity finding(s): CLI upgrade skips CA trust wiring, per the severity policy in release-train's README (High stops the line; an unreadable severity blocks too, because unknown is not low; Medium/Low are recorded and ship, at both hops). Blocking threads stay UNRESOLVED by design, so this promotion remains blocked until they are fixed on What the train did with each:
|
…the tag (backend#2638) (#866) _values_pin_mysql_84 reported an 8.4 pin whenever tag=="8.4" && digest=="", but it initialises digest to "" and is fed PARTIAL values — _release_pins_mysql_84 reads `helm get values` WITHOUT --all (chart defaults omitted) and a dev-mode overlay can carry only mysqlClient.tag. In both, the chart-default digest (the amd64-only 5.7 pin) is what actually renders — tracebloc.image makes the digest win over the tag — yet its line is nowhere on STDIN. Treating that missing line as an empty digest skipped the 5.7 arch gate and CrashLooped the amd64-only image on arm64 (adopt / dev-mode). Require the digest key to APPEAR and be empty (sawdigest) before concluding 8.4; an absent digest is "not provably cleared" -> not 8.4 -> the 5.7 arch gate runs (fail closed). The installer's 8.4 heredoc always writes an explicit `digest: ""`, so no genuine opt-in regresses. Deliberately not switched to `helm get values --all`: coalescing can re-default an operator's explicit `digest: ""` back to the chart pin, which would false-refuse a real 8.4 reconcile. Regen scripts/manifest.sha256 for the edited signed sub-script. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
LukasWodka
commented
Aug 27, 2026
ContributorAuthor
REFUSED. This promotion was blocked by 1 HIGH severity finding(s): CLI upgrade skips CA trust wiring, per the severity policy in release-train's README (High stops the line; an unreadable severity blocks too, because unknown is not low; Medium/Low are recorded and ship, at both hops). Blocking threads stay UNRESOLVED by design, so this promotion remains blocked until they are fixed on What the train did with each:
|
…ories (backend#2674) (#870) * test(assess): guard every early-exit path reaches the drift/GPU advisories (backend#2674) assess short-circuits an already-set-up machine before the normal flow reaches _handle_existing_cluster, where the k3s-drift (#547/#565) and GPU-consistency (client#835/#852) advisories run. Every early-exit terminal must run both itself — a rule we kept re-learning one instance at a time (the k3s check, the GPU check, then the cli-behind-latest → upgrade_cli_only path in backend#2253, each patched only after the omission was spotted). New suite scripts/tests/assess-early-exit-drift.bats catches the CLASS: - behavioral: drives the healthy hand-off and upgrade_cli_only, asserts BOTH advisories run (and, for the hand-off, before it); - static enumeration that FAILS CLOSED on a new uncovered terminal: pins the exit-bearing functions in assess.sh (_assess_handoff) and install-cli.sh (upgrade_cli_only), pins _assess_handoff to one call site, and asserts each early-exit decision calls both advisories at the source level; - a fixture proving the enumeration actually detects an unguarded early-exit. Mutation-verified: dropping either advisory fails the behavioral + static tests; adding a new exit-bearing function fails the pin. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(assess): derive the advisory set instead of hardcoding it (backend#2674) Bugbot: the ADVISORIES pair was restated, so a new advisory added to the healthy hand-off but not upgrade_cli_only would pass both the static loop and the behavioral stubs — the same class this suite stops, on the advisory axis. Derive the set from the `declare -F X && X` guard idiom in the reference path (the healthy hand-off) and assert upgrade_cli_only runs the SAME set. Divergence in either direction now fails. Mutation-verified: a 3rd advisory on one path only fails the parity test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(assess): harden the scanner against compound exits + both guard idioms (backend#2674) Two Bugbot findings on the scanner itself — the blind spots that matter most for a class-catcher: - Exit scan matched only a leading `exit`, so a terminal written `foo && exit`, `foo; exit`, or `then exit` slipped the pin. Now matches `exit` as a word in any position — but strips single/double-quoted spans first so an embedded `awk '... exit }'` (as in _assess_cluster_servers_running) is not a false positive, and skips/strips comments. - Advisory derivation saw only the one-liner `declare -F X && X`, so a _check_ advisory added via the `if declare -F X; then X; fi` block (the form install_tracebloc_cli already uses) slipped parity. Now keys on `declare -F _check_*` in either idiom, skipping comments so a commented-out guard cannot pad the set. Removed the now-unused _funcs_calling helper. Fixture extended to every exit spelling + an embedded-awk-exit + a commented-out guard. Mutation-verified: a compound-exit terminal fails the pin; a 3rd advisory via if-then on one path only fails parity. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(assess): count hand-off invocations in any spelling, not just line-leading (backend#2674) Bugbot: the _assess_handoff call-site pin used a line-leading grep, so a second hand-off in the file OWN case-arm style (state) … _assess_handoff ;;), or via &&/then, never incremented the count — the exact inline early-exit this suite exists to catch. Add _count_calls, which counts invocations of a symbol as a word in any position (excluding the definition token and comments/quoted spans, same handling as the exit scan), and use it for the pin. Fixture now plants a case-arm one-liner hand-off and asserts the count. Mutation-verified: a syntactically-valid inline second hand-off fails the pin. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
… download (backend#2679) (#873) * fix(installer): wire CA trust in the CLI-only upgrade path before the download (backend#2679) The explicit `tracebloc upgrade` fast-path (upgrade_cli_only, backend#2253) downloads and cosign-verifies the CLI via install_tracebloc_cli and then EXITS — before main()'s wire_ca_trust (install-k8s.sh) ever runs. Behind a TLS-inspecting proxy with a private/corporate CA, that leaves the download or signature check failing x509 on the very machine where a normal install SUCCEEDS, because the full flow wires CA trust before any tool download (#583). Wire the corporate CA inside upgrade_cli_only before the download — the same idiom this path already uses to re-surface the hand-off's advisories it would otherwise skip by exiting early. wire_ca_trust is idempotent and a no-op when no CA is configured, and the call is declare -F guarded like main()'s own so a stale bootstrap without cluster.sh falls through unchanged. Tests: assert CA trust is wired BEFORE the download on the upgrade path (order is the whole point), and that a missing wire_ca_trust (stale bootstrap) still exits 0. Regenerated scripts/manifest.sha256 for the install-cli.sh change. Closestracebloc/backend#2679 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(installer): drive upgrade_cli_only guard tests under set -e (backend#2679) Bugbot: the stale-bootstrap guard tests drove upgrade_cli_only through bats `run`, which turns errexit OFF — so an unguarded call to a missing wire_ca_trust (or install_tracebloc_cli) was only a command-not-found the function walked past before exit 0. The declare -F guard could be deleted and the tests stayed green, while main() runs this path under `set -e` and a stale bootstrap without cluster.sh would crash `tracebloc upgrade` at 127 before the CLI download. Drive both stale-bootstrap tests in an explicit `set -e` subshell so they exercise the real production seam. Verified by mutation: strip either declare -F guard and the matching test now goes 127 instead of staying green. Fixing the class, not just the flagged instance: the sibling install_tracebloc_cli guard test had the identical latent weakness and is hardened the same way. backend#2679 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
LukasWodka
commented
Aug 27, 2026
ContributorAuthor
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 3bc36a5. Configure here.
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated promotion by the release train (RFC-0008 D14). Head is the train-managed
release-train/to-stagingbranch (a mirror ofdevelop), so it never collides with a human PR. Merged only when the fr-gate is green.Note
High Risk
Changes authentication credential resolution, multi-tenant telemetry Secret naming on upgrade, and the Linux GPU cluster create path — all high-impact for existing edges and installs.
Overview
This promotion bundles chart 1.9.75 with installer, CI, and guard changes that were developed on
develop.Telemetry ingest tokens are now release-scoped (
<release>-telemetry-token) so multiple edges in the shared node-agents namespace do not overwrite one Secret; jobs-manager, the collector DaemonSet, and RBAC sharetracebloc.telemetryTokenSecretName, with legacytracebloc-telemetry-tokenaccepted during migration. Client credentials (clientId/clientPassword) resolve from values, then the live release Secret vialookup, then fail — so upgrades can drop cleartext from Helm release values when the cluster is used for render (GitOps client-side render still needs values).Linux NVIDIA GPU installs wire k3d to the k3s-cuda image, generate native CDI specs in nodes, gate chart GPU requests on
_gpu_wired, set the device plugin’sruntimeClassName: nvidia, and reconcile reused CPU-only clusters without stranding jobs Pending. CronJobs (auto-upgrade,image-refresh) userestartPolicy: Neverso failed Job Pods stay for logs.CI: authenticated proxy E2E moves to
e2e-auth-proxy.yaml(PR runs withoutpaths:so it can be a required check).make driftadds guards for cronjob readability, release-name=namespace, and helm-unittest assertion keys (errorPatternis ignored by helm-unittest 0.5.2).tracebloc upgradecan update only the CLI when the environment is healthy but the CLI is behind latest.Installer fixes include Win32 argument escaping for
docker/k3dspawn, CLIENT_ID from Secret when values omitclientId, and MySQL 8.4 detection requiring an explicitly empty digest in partial values views.Reviewed by Cursor Bugbot for commit 3bc36a5. Bugbot is set up for automated code reviews on this repo. Configure here.