Skip to content

feat(installer): wire the corporate CA into cosign/helm/git, not just curl (#583) - #592

Merged
LukasWodka merged 10 commits into
developfrom
fix/583-wire-ca-proxy
Aug 4, 2026
Merged

feat(installer): wire the corporate CA into cosign/helm/git, not just curl (#583)#592
LukasWodka merged 10 commits into
developfrom
fix/583-wire-ca-proxy

Conversation

@shujaatTracebloc

@shujaatTraceblocshujaatTracebloc commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

What & why

Closes#583 (child 2/4 of #578, network resilience — the single biggest lever for enterprise networks).

A TLS-inspecting corporate proxy re-signs HTTPS with a corporate root CA. Tools that don't trust that root fail x509 — the class behind both field failures (the k3d-checksum failure and the cosign/sigstore verification failure). curl already honored CURL_CA_BUNDLE, and the k3d nodes got the CA at cluster-create (#424), but cosign, helm, and git got nothing. This PR extends the same resolved CA to every host tool that doesn't inherit the system trust store on its own.

Changes

Bootstrap — cosign (install.sh, install.ps1; not manifested — the trust root)

  • Before cosign runs, export SSL_CERT_FILE from TRACEBLOC_CA_BUNDLE / CURL_CA_BUNDLE. cosign's Go HTTPS client reads SSL_CERT_FILE (added to the roots on modern Go, incl. Windows), so keyless verification's calls trust the corporate CA instead of failing x509. No-op when unset.

Main installer — helm / git / curl (cluster.shwire_ca_trust, install-k8s.ps1Set-ToolTrust)

  • Export SSL_CERT_FILE + GIT_SSL_CAINFO (+ CURL_CA_BUNDLE) from the resolved bundle, before preflight's HTTPS probes and any tool download, so helm, git and curl all trust it end-to-end.
  • One plain-language line: Trusting your company's certificate for cosign, helm, git and downloads.
  • Idempotent; no-op when unconfigured; fails fast (before any privileged step) on a set-but-unreadable bundle — reusing the existing _resolve_ca_bundle / Resolve-CaBundle contract.

On the acceptance criteria

Tests

  • bash:wire_ca_trust exports the three vars / no-ops when unset / hard-fails on an unreadable bundle (cluster.bats); the bootstrap integration test runs the real bootstrap with TRACEBLOC_CA_BUNDLE set and a recording mock cosign, asserting cosign saw SSL_CERT_FILE (install-bootstrap.bats).
  • PowerShell:Set-ToolTrust exports SSL_CERT_FILE/GIT_SSL_CAINFO / no-ops when unset (install-k8s.Tests.ps1); the bootstrap sets SSL_CERT_FILE from the CA env (install.Tests.ps1).
  • Full Pester 436/0/9; cluster.bats 78/0; manifest --check, check-facts, check-style clean. (cluster.sh / install-k8s.* are manifested — manifest.sha256 regenerated; the bootstrap scripts are the trust root and stay unmanifested.)

Notes


Note

Medium Risk
Changes installer TLS trust wiring across bootstrap and main install paths on all platforms; behavior is platform-gated and fail-fast, but mistakes could still affect cosign/git/helm behind corporate proxies.

Overview
Corporate CA trust is applied earlier and more honestly across the installer stack (#583), so TLS-inspecting proxies fail at setup with clear errors instead of mid-download x509 failures.

On Linux, wire_ca_trust (bash) sets SSL_CERT_FILE and GIT_SSL_CAINFO from TRACEBLOC_CA_BUNDLE / CURL_CA_BUNDLE only when those vars are unset; it does not overwrite CURL_CA_BUNDLE. The main install-k8s flow calls this before preflight and tool downloads. Bootstrap install.sh adds _bootstrap_wire_ca so cosign sees SSL_CERT_FILE on Linux before manifest verification.

On macOS, the same helpers validate the bundle path but export nothing—Go tools use Keychain, and exporting could shrink curl trust—users get Keychain/offline hints instead.

On Windows, Set-ToolTrust sets onlyGIT_SSL_CAINFO (not SSL_CERT_FILE); install.ps1 validates CA readability before cosign but likewise does not wire PEM into Go tools, with hints to use the certificate store or offline installer.

User messaging distinguishes what was wired vs pre-set bundles kept (replace-not-augment), avoiding false “Trusting…” success when nothing changed. Tests cover Linux/macOS bootstrap behavior, wire_ca_trust edge cases, and PowerShell Set-ToolTrust.

Reviewed by Cursor Bugbot for commit 1b40cbe. Bugbot is set up for automated code reviews on this repo. Configure here.

… curl (#583)
Child 2/4 of #578 — the single biggest lever for TLS-inspecting corporate networks.
A break-and-inspect proxy re-signs HTTPS with a corporate root CA; tools that don't
trust that root fail x509. curl already honored CURL_CA_BUNDLE and the k3d NODES got
the CA at cluster-create (#424), but cosign, helm and git got nothing — the class
behind both field failures (the k3d-checksum and the cosign/sigstore x509 failures).
Extend the SAME resolved CA to every host tool that doesn't inherit the system store:
- Bootstrap (install.sh / install.ps1): export SSL_CERT_FILE from TRACEBLOC_CA_BUNDLE
/ CURL_CA_BUNDLE before cosign runs, so keyless verification's HTTPS calls trust the
corporate CA (cosign's Go client reads SSL_CERT_FILE). Not manifested (trust root).
- Main installer: wire_ca_trust (bash) / Set-ToolTrust (PS) export SSL_CERT_FILE +
GIT_SSL_CAINFO (+ CURL_CA_BUNDLE) from the resolved bundle, run BEFORE preflight's
probes and any download, so helm, git and curl all trust it. Plain-language line:
"Trusting your company's certificate for cosign, helm, git and downloads." No-op
when unconfigured; fails fast on a set-but-unreadable bundle. When no CA is provided,
tools fall back to the system store (which enterprise IT usually populates) — no user
knowledge of CAs required in that common case.
Tests: bats (wire_ca_trust exports/no-op/hard-fail; bootstrap cosign sees SSL_CERT_FILE
via a recording mock) + Pester (Set-ToolTrust exports/no-op; bootstrap sets SSL_CERT_FILE).
cluster.sh / install-k8s.* are manifested; manifest regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shujaatTraceblocshujaatTracebloc self-assigned this Aug 4, 2026
@shujaatTracebloc
shujaatTracebloc marked this pull request as ready for review August 4, 2026 14:33
Comment threadscripts/install.sh Outdated
…ate CA (#583)
Bugbot (Medium): _bootstrap_wire_ca / wire_ca_trust / Set-ToolTrust exported
CURL_CA_BUNDLE derived from the resolved bundle. But CURL_CA_BUNDLE is replace-not-
augment, and TRACEBLOC_CA_BUNDLE is typically a corp-root-ONLY PEM (its documented
k3d-node use), so re-exporting it could REPLACE curl's working trust with a bundle
missing the public roots — breaking the manifest/sig fetches that were succeeding via
the system store. The PowerShell bootstrap already set only SSL_CERT_FILE.
curl already honors the user's own CURL_CA_BUNDLE natively, so we never re-export it.
We only wire the tools that had NO corporate trust before: cosign/helm/Go
(SSL_CERT_FILE) and git (GIT_SSL_CAINFO). This also makes the bash and PS bootstraps
symmetric (both set only SSL_CERT_FILE).
Adds a regression test asserting wire_ca_trust leaves a pre-set CURL_CA_BUNDLE intact
while still exporting SSL_CERT_FILE for the other tools.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadscripts/install.ps1 Outdated
Comment threadscripts/install.sh Outdated
…mac + fail fast (#583)
Two Bugbot follow-ups:
- [High] SSL_CERT_FILE is inert for cosign/helm on Windows AND macOS. cosign/helm are
Go; Go reads SSL_CERT_FILE only on Linux — on Windows it uses the certificate store
and on macOS the Keychain, ignoring the env var. My comment wrongly claimed modern
Go honors it on Windows. Corrected:
* Linux: keep SSL_CERT_FILE (effective) + GIT_SSL_CAINFO; announce cosign/helm/git.
* macOS (wire_ca_trust): still set the vars but announce only git + downloads, and
hint that cosign/helm read the Keychain (add the CA there, or use the offline path).
* Windows (Set-ToolTrust): set GIT_SSL_CAINFO (Git-for-Windows is OpenSSL-backed);
do NOT set SSL_CERT_FILE (inert/misleading); hint cosign/helm read the cert store.
* Windows bootstrap (install.ps1): drop the inert SSL_CERT_FILE set entirely.
The robust cross-platform cosign fix for a PEM-only CA is the offline bundle (#584).
- [Med] A set-but-unreadable CA bundle now fails fast with a clear "can't be read"
message in both bootstraps, instead of silently no-opping and surfacing later as a
generic cosign authenticity error.
Tests updated for the platform-accurate behavior: Linux vs macOS announce, Windows
sets only GIT_SSL_CAINFO (not SSL_CERT_FILE) + store hint, and the bootstrap fail-fast
on a bad CA path (bash + PS).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadscripts/lib/cluster.sh Outdated
…O either (#583)
Bugbot (Medium): wire_ca_trust / Set-ToolTrust set GIT_SSL_CAINFO unconditionally from
the resolved (corp-root-only) bundle. GIT_SSL_CAINFO is replace-not-augment (same
OpenSSL contract as CURL_CA_BUNDLE), so a fuller pre-set git CA bundle got overwritten
and host git HTTPS could x509-fail on non-intercepted endpoints. The same applies to
SSL_CERT_FILE.
Apply the consistent rule everywhere we wire trust: only set a trust var the user
hasn't already set — never override their existing bundle. Covers SSL_CERT_FILE (bash
bootstrap + wire_ca_trust) and GIT_SSL_CAINFO (wire_ca_trust + Set-ToolTrust); curl's
CURL_CA_BUNDLE was already left untouched. Adds regression tests (bash + Pester) that a
pre-set SSL_CERT_FILE / GIT_SSL_CAINFO survives while an unset one is still wired.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadscripts/install-k8s.ps1 Outdated
Comment threadscripts/install.ps1
shujaatTraceblocand others added 2 commits August 4, 2026 17:30
… on Windows (#583)
Two Bugbot follow-ups:
- [Med] Set-ToolTrust printed "…and downloads" trust the corporate CA, but on Windows
downloads use the certificate store (Invoke-WebRequest/Schannel) which this path never
configures — only GIT_SSL_CAINFO. Green message, still-failing fetch. The announce now
names only what's actually wired (git), and the store hint covers cosign, helm AND the
installer's downloads. Same over-claim dropped on Linux/macOS: curl "downloads" trust
the user's own CURL_CA_BUNDLE (which we deliberately don't touch), so Linux announces
"cosign, helm and git" and macOS "git" only.
- [Med] The Windows bootstrap CA fail-fast only tested existence (Test-Path -PathType
Leaf); a present-but-unreadable file slipped through to a generic cosign error. It now
also opens the file (mirrors bash -r and Resolve-CaBundle) and fails fast with a clear
"can't be read" message.
Tests updated: Linux/macOS announce wording (no "downloads"), and the Windows success
line names only git while the store hint covers downloads.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadscripts/install.sh
Comment threadscripts/lib/cluster.sh Outdated
The 20 assertions this PR adds to cluster.bats and install-bootstrap.bats
were written multi-assertion without `|| return 1`, so under bats only the
last command decided each test -- a regression in the cosign/helm/git CA
wiring would have passed silently. Same hardening #527 applies suite-wide;
appending it here keeps this PR green under #527's bats-hygiene gate
whichever lands first. Both suites pass with enforcement on. The Pester
additions need nothing: Should throws, so every assertion already enforces.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor

Taking this over from @shujaatTracebloc at Lukas's request (assignee updated per the handover rule — author unchanged).

Review verdict on the CA wiring itself: sound. All six earlier Bugbot findings are fixed at head (only-if-unset augment-safe exports, no CURL_CA_BUNDLE clobber, honest per-platform announce — Go ignores SSL_CERT_FILE on Windows/macOS so it isn't claimed there, fail-fast on unreadable bundles incl. the PS OpenRead probe). manifest.sha256 recomputed clean. No #591-class encoding risk — the PS side never reads file contents (zero-byte readability probe only) and bash only tests -f/-r.

One gap, now fixed in the pushed commit: the 20 new bats assertions were written without || return 1, so under bats they were advisory — a CA-wiring regression would have passed silently, and #527's incoming bats-hygiene gate would fail on exactly these lines. Hardened all 20 (cluster.bats ×15, install-bootstrap.bats ×5); both suites pass with enforcement on. The Pester additions are fine as-is (Should throws).

bugbot run

Comment threadscripts/lib/cluster.sh Outdated
… ×2)
Two Darwin holes, same root: exporting trust vars the platform ignores.
- SSL_CERT_FILE: Go reads the Keychain on macOS, so the export helped neither
cosign nor helm — while OpenSSL-backed curl DOES honor it, replace-not-
augment, so a corp-root-only bundle shrank download trust for zero gain.
Dropped from wire_ca_trust and platform-gated in _bootstrap_wire_ca
(readability fail-fast still runs everywhere).
- GIT_SSL_CAINFO: Apple's system git (SecureTransport) ignores it, and the
clone that matters most — Homebrew's own bootstrap — runs system git. The
"Trusting your company's certificate for git" claim was false on Darwin.
Darwin now exports neither var and prints one honest hint: add the CA to the
login Keychain so git, cosign and helm trust it. Same decision, same reason
as Windows (store-based trust; no inert claims). Tests: Darwin announce
updated, Darwin exports-nothing pinned at both layers, and the Linux
bootstrap test now stubs uname so it doesn't flip on a macOS dev machine.
Manifest regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor

Fixed both Bugbot findings — same root, same resolution as the PR's own Windows decision: on macOS, wire nothing.SSL_CERT_FILE helped neither cosign nor helm there (Go reads the Keychain) while shrinking OpenSSL-curl's download trust to the corp root; and the git success claim was false — Apple's SecureTransport git ignores GIT_SSL_CAINFO, including during Homebrew's own bootstrap clone. Darwin now exports neither var and prints one honest Keychain hint covering git/cosign/helm. Readability fail-fast unchanged on every platform. Tests pinned at both layers (lib + bootstrap, with a stubbed uname so they don't flip by host OS); manifest regenerated; suites green.

bugbot run

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 33d6744. Configure here.

Comment threadscripts/lib/cluster.sh Outdated
wire_ca_trust and Set-ToolTrust printed the green "Trusting your company's
certificate…" even when every only-if-unset guard skipped its export --
claiming wiring that did not happen, and masking a pre-set bundle that may
still lack the corporate CA. Both now track wired vs kept per variable: the
success names only what was actually exported, and anything kept gets an
explicit "make sure that bundle includes your company's CA" hint instead.
Pinned on both layers: both-kept claims nothing, partial pre-set claims only
the wired half (bats), skipped export claims nothing (Pester). Manifest
regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor

Fixed: both wire_ca_trust and Set-ToolTrust now track wired-vs-kept per variable — the green success names only what was actually exported, and a kept pre-set bundle gets an explicit "make sure that bundle includes your company's CA" hint instead of a false claim. Pinned at both layers: both-kept → no claim + hint (bats + Pester), partial pre-set → success names only the wired half (bats). Manifest regenerated; cluster.bats 85/85, Pester Set-ToolTrust 4/4.

bugbot run

shujaatTracebloc pushed a commit that referenced this pull request Aug 5, 2026
…eqs hang (#593)
* fix(ci): bound the two unbounded network waits behind the ubuntu Prereqs hang
Three times on 2026-08-04 (#525, #592) the "Prereqs — ubuntu:*" matrix jobs
died at the 20-minute job timeout with nothing in the log but "Installing
Docker…", and once more failed in 20 seconds with a registry-1.docker.io
timeout (exit 125). Two unbounded waits, one per layer:
- Workflow: `docker run` pulls the distro image implicitly with no timeout,
so Hub connectivity trouble either failed fast (exit 125) or stalled the
whole job. Both container-matrix jobs (distro-prereqs, path-persist) now
pre-pull with three bounded attempts (timeout 300 + backoff) and an honest
"runner-to-registry connectivity, not this PR" error.
- setup-linux.sh: the get.docker.com convenience script's internal
apt/download.docker.com fetches carry no timeout, so a stalled connection
hung silently behind the spinner. The run is now bounded at 10 minutes
(healthy installs take 1-3) and fails with a clear stalled-download message
telling the operator to re-run; the fetch of the script itself already had
retry + curl_secure timeouts. Same shape as the existing dpkg-lock and
kubectl-fetch bounds.
New bats test pins the timeout bound on the get.docker.com branch (hardened
with || return 1 for the incoming #527 hygiene gate). Manifest regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ci): distinguish a stall from a real failure; keep the pull budget small (Bugbot ×2)
- setup-linux.sh: `if ! spin_cmd …; then error "stalled 10 minutes"` fired on
ANY failure, mislabelling a fast real apt/script error as a stall — and it
bypassed the existing spin_cmd_bounded helper, which returns 124 only on the
deadline and tails the log on every failure. Switched to it: rc 124 gets the
stalled-download message, any other rc gets an honest install-failed message
pointing at the log tail. Harness gains a default spin_cmd_bounded mock; the
bats test now pins the helper + its 600s bound.
- installer-tests.yaml: three timeout-300 attempts + backoff could eat ~16 of
the job's 20 minutes, so a late-succeeding pull just moved the death from
the pull to the install. Bounds resized (3 × timeout 90, 10/20s backoff,
~5.5 min worst case) so the job keeps most of its budget; a healthy pull
takes seconds.
Manifest regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): prepare-host gets prepare-host re-run advice (Bugbot)
The new get.docker.com stall/failure errors always said "re-run the
installer" — but with TB_PREPARE_HOST_MODE set that points an admin at a
full provision as themselves, the exact outcome prepare-host exists to
prevent. Pick the re-run verb by mode, matching the daemon-check errors
later in the same function. Manifest regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Network resilience (2/4): auto-wire the corporate CA + proxy into every tool (cosign/docker/k3d/helm)

2 participants

@shujaatTracebloc@LukasWodka