You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Closes#584 (child 3/4 of #578, network resilience). This is the item that actually fixes the field Windows install failure on a TLS-inspecting corporate network.
Keyless cosign verification needs sigstore's transparency log (Rekor) at verify time, and our short-lived keyless cert is expired by install time — so a network that blocks or TLS-inspects sigstore fails verification even for a valid signature. #583's CA wiring doesn't cover this on Windows/macOS (Go ignores SSL_CERT_FILE there), and skipping the tlog was proven not to verify the expired cert. The fix is an offline Sigstore bundle: it carries the Rekor inclusion proof (SET), so the cert's validity at signing is established with no live call.
Changes
Signing — release-helm-chart.yaml
cosign sign-blob now also emits manifest.sha256.bundle and publishes it as a release asset, alongside the existing .sig/.cert.
Verify — install.sh + install.ps1
Prefer cosign verify-blob --bundle <bundle> --certificate-identity-regexp … --certificate-oidc-issuer … --offline — a full check (signature + cert identity + tlog inclusion) with no live Rekor.
Fallback: if the bundle 404s (older release) or doesn't verify, fall through to the existing online .sig/.cert keyless path. That path does the same full keyless verification, just needing live Rekor — so it's a fallback, never a security downgrade.
bats: bootstrap prefers --bundle --offline when a bundle is published (and does not touch the sig/cert path); falls back to sig/cert when no bundle exists (older release); cosign-failure still fails closed (unchanged).
Pester:Confirm-ManifestSignature verifies --bundle --offline first with a sig/cert fallback; Invoke-CosignVerifyBlob is fail-closed (sentinel + stderr suppression).
Full Pester 449/0/9; guards + manifest --check clean. install.sh/install.ps1 are the bootstrap trust root (not manifested), so no manifest change.
⚠️ Sequencing (cross-repo-ish, but all in this repo)
The installer prefers the bundle but a new release must be cut after merge so the manifest.sha256.bundle asset actually exists. Until then — and for all existing releases — the fallback (online sig/cert) path runs, so nothing regresses. Once a release ships the bundle, sigstore-blocked networks (incl. the field Windows case) verify fully offline with no env var or user action.
Notes
The --insecure-ignore-tlog shortcut from the earlier cosign thread was proven not to verify our keyless cert — this bundle approach is the correct version.
Medium Risk
Changes the bootstrap trust root and release signing assets; behavior is backward-compatible via fallback, but a bad bundle rollout could affect installs until the next release ships bundles correctly.
Overview Adds offline manifest verification so installs succeed on networks that block or TLS-inspect Sigstore/Rekor, and so expired short-lived keyless certs can still be validated using the bundle’s embedded signing-time proof.
The release workflow now emits and publishes manifest.sha256.bundle alongside the existing .sig/.cert when signing manifest.sha256.
install.sh and install.ps1 try cosign verify-blob --bundle … --offline first (same identity pins as before). If the bundle is missing (older releases) or verification fails, they fall back to the existing online .sig/.cert path—same keyless checks, not a weaker mode. On Windows, Invoke-CosignVerifyBlob centralizes fail-closed $LASTEXITCODE seeding and stderr suppression for both paths.
Tests (bats + Pester) cover bundle-first success, sig/cert fallback, older releases without a bundle, and fail-closed when both paths fail.
Reviewed by Cursor Bugbot for commit 5669847. Bugbot is set up for automated code reviews on this repo. Configure here.
…Rekor (#584)
Child 3/4 of #578. Keyless cosign verification needs sigstore's Rekor at verify time,
and our short-lived keyless cert is expired by install time — so a network that blocks
or TLS-inspects sigstore fails verification even for a valid signature (the class behind
the field Windows failure on a TLS-inspecting corporate network that #583's CA wiring
does NOT cover, since Go ignores SSL_CERT_FILE on Windows/macOS). Skipping the tlog was
proven not to verify the expired cert; the offline bundle carries the Rekor inclusion
proof (SET) so the cert's validity at signing can be established without any live call.
- Signing (release-helm-chart.yaml): cosign sign-blob now also emits
manifest.sha256.bundle and publishes it as a release asset, alongside the .sig/.cert.
- Verify (install.sh + install.ps1): prefer `verify-blob --bundle <bundle> --offline`
(full check — signature + cert identity + tlog inclusion — with NO live Rekor). Falls
through to the existing online .sig/.cert keyless path for releases cut before the
bundle existed, or if the bundle doesn't verify — the SAME full check, just needing
live Rekor, so it's a fallback, never a downgrade.
- PS: extracted Invoke-CosignVerifyBlob so the fail-closed sentinel (nonzero
$LASTEXITCODE seed) + stderr suppression (#576) back BOTH paths from one place.
Tests: bootstrap prefers --bundle --offline when a bundle is published and does NOT hit
the sig/cert path; falls back to sig/cert when no bundle (older release); fail-closed on
a bad signature unchanged. Pester + bats green.
SEQUENCING: needs a NEW release cut after merge so the bundle asset exists; until then
the fallback (online) path runs. Existing releases keep working via the fallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
Nice, careful PR — offline bundle is the right fix and the fallback is genuinely a fallback, not a downgrade. Approving.
One gap worth a follow-up (non-blocking): the bundle-verifies-then-fails path isn't tested — see inline. Minor nit: on Windows, folding the try/catch into Invoke-CosignVerifyBlob means a cosign that can't launch now surfaces the generic "couldn't confirm authentic" message instead of the old "couldn't run the verification step" — same fail-closed, just a slightly misleading message for a broken binary.
…ail-closed (reviewer)
saadqbal: the bundle-present-but-verify-fails -> sig/cert fallback branch was never
exercised (both bundle tests forced cosign exit 0; the fail-closed test had no bundle),
so a regression there would stay green. Add:
- bats: a fallback case (bundle published, cosign REJECTS the --bundle verify but ACCEPTS
sig/cert -> the sig/cert path runs and the install proceeds) and a both-fail case
(bundle present, every cosign verify fails -> fail closed, privileged step never runs).
- Pester: behavioural (not source-text) equivalents driving Confirm-ManifestSignature —
bundle-fails falls through to sig/cert (2 verify calls, no throw); both-fail throws the
authenticity error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
e2eRun the full E2E last-mile journey (create_cluster → CLI → cluster info → data validate)
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Closes#584 (child 3/4 of #578, network resilience). This is the item that actually fixes the field Windows install failure on a TLS-inspecting corporate network.
Keyless cosign verification needs sigstore's transparency log (Rekor) at verify time, and our short-lived keyless cert is expired by install time — so a network that blocks or TLS-inspects sigstore fails verification even for a valid signature. #583's CA wiring doesn't cover this on Windows/macOS (Go ignores
SSL_CERT_FILEthere), and skipping the tlog was proven not to verify the expired cert. The fix is an offline Sigstore bundle: it carries the Rekor inclusion proof (SET), so the cert's validity at signing is established with no live call.Changes
Signing —
release-helm-chart.yamlcosign sign-blobnow also emitsmanifest.sha256.bundleand publishes it as a release asset, alongside the existing.sig/.cert.Verify —
install.sh+install.ps1cosign verify-blob --bundle <bundle> --certificate-identity-regexp … --certificate-oidc-issuer … --offline— a full check (signature + cert identity + tlog inclusion) with no live Rekor..sig/.certkeyless path. That path does the same full keyless verification, just needing live Rekor — so it's a fallback, never a security downgrade.Invoke-CosignVerifyBlobso the fail-closed sentinel (nonzero$LASTEXITCODEseed) + stderr suppression (Installer must never expose tracebloc internals or user PII in logs/output #576) back both paths from one place.Tests
--bundle --offlinewhen a bundle is published (and does not touch the sig/cert path); falls back to sig/cert when no bundle exists (older release); cosign-failure still fails closed (unchanged).Confirm-ManifestSignatureverifies--bundle --offlinefirst with a sig/cert fallback;Invoke-CosignVerifyBlobis fail-closed (sentinel + stderr suppression).--checkclean.install.sh/install.ps1are the bootstrap trust root (not manifested), so no manifest change.The installer prefers the bundle but a new release must be cut after merge so the
manifest.sha256.bundleasset actually exists. Until then — and for all existing releases — the fallback (online sig/cert) path runs, so nothing regresses. Once a release ships the bundle, sigstore-blocked networks (incl. the field Windows case) verify fully offline with no env var or user action.Notes
--insecure-ignore-tlogshortcut from the earlier cosign thread was proven not to verify our keyless cert — this bundle approach is the correct version.Note
Medium Risk
Changes the bootstrap trust root and release signing assets; behavior is backward-compatible via fallback, but a bad bundle rollout could affect installs until the next release ships bundles correctly.
Overview
Adds offline manifest verification so installs succeed on networks that block or TLS-inspect Sigstore/Rekor, and so expired short-lived keyless certs can still be validated using the bundle’s embedded signing-time proof.
The release workflow now emits and publishes
manifest.sha256.bundlealongside the existing.sig/.certwhen signingmanifest.sha256.install.shandinstall.ps1trycosign verify-blob --bundle … --offlinefirst (same identity pins as before). If the bundle is missing (older releases) or verification fails, they fall back to the existing online.sig/.certpath—same keyless checks, not a weaker mode. On Windows,Invoke-CosignVerifyBlobcentralizes fail-closed$LASTEXITCODEseeding and stderr suppression for both paths.Tests (bats + Pester) cover bundle-first success, sig/cert fallback, older releases without a bundle, and fail-closed when both paths fail.
Reviewed by Cursor Bugbot for commit 5669847. Bugbot is set up for automated code reviews on this repo. Configure here.