Uh oh!
There was an error while loading. Please reload this page.
test(e2e): full seal suite vs the dev backend (backend#1184 fast-follow) - #598
Conversation
… fast-follow New e2e-full-seal.sh installs the working-tree chart on real k3d as the dedicated dev e2e-test-agent (real credentials, CLIENT_ENV=dev), waits for every release PVC to Bind and jobs-manager to hold a real backend session, then runs helm test UNFILTERED — egress-enforcement + backend-reachability + storage-assertions in one release, hook-presence-guarded so a regated check can't vanish silently. New helm-ci job full-seal-e2e runs it on push/dispatch only and skips green with a notice until the e2e-test-agent secrets (TB_E2E_CLIENT_ID / TB_E2E_CLIENT_PASSWORD) are provisioned. The egress positive control moves verbatim into e2e-common.sh (one copy, shared with e2e-seal-check.sh). SEAL-CHECK.md gains the CI-coverage map and drops three follow-ups that shipped since (#541 live probe, cli#393 verdict, cli#449 matrix). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka
commented
Aug 5, 2026
bugbot run |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
- credentials travel in a mode-0600 temp values file, never on argv (process-list exposure on a shared runner + helm --set comma/brace mangling); removed on every exit path — installer parity - the PVC wait does one guarded fetch per iteration, so a transient kubectl failure retries until the deadline instead of aborting under set -euo pipefail - full-seal-e2e job timeout 30m -> 45m: the script stacks a 300s PVC wait, two 300s rollouts and a 600s unfiltered helm test on create_cluster's 15m bound — GHA must not kill a slow-but-healthy run Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka
commented
Aug 5, 2026
All three findings addressed: creds now travel in a 0600 temp values file (never argv — and immune to helm --set's comma/brace mangling), the PVC wait retries through kubectl blips instead of aborting under bugbot run |
Uh oh!
There was an error while loading. Please reload this page.
…ull-seal install (Bugbot) The chart's PVC default is ReadWriteMany; rancher.io/local-path never provisions it, so every claim sat Pending and both the Bound pre-wait and storage-assertions were guaranteed to fail once the job activates. The installer writes pvcAccessMode: ReadWriteOnce for this exact storage path — the harness now installs with the same value. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka
commented
Aug 5, 2026
Second-round finding fixed: the install now pins bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d639aca. Configure here.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.

What (the backend#1184 deferred fast-follow)
backend#1184 closed accepting CI-substrate coverage as v1-sufficient, with one recorded residual: "the FULL suite (incl. backend-reachability + bound-PVC storage-assertions) on the dev-env e2e-test-agent with real credentials." This is that harness.
scripts/tests/e2e-full-seal.sh(new)Real k3d cluster → install the working-tree chart as the dedicated dev
e2e-test-agentclient (real creds,CLIENT_ENV=dev, lockdown engaged, same base profile as the sibling) → crisp pre-asserts (every release PVC Bound within 300s — separates "slow to bind" from "assertions wrong"; mysql + jobs-manager rolled out, i.e. a real backend session) → shared egress positive control → hook-presence guard for all three checks (an unfilteredhelm test"passes" a release whose hooks silently stopped rendering — same trap the sibling guards for its--filter) →helm testunfiltered:egress-enforcement+backend-reachability+storage-assertionsin one release. On failure, every seal-check pod log dumps via the §8.2 enumeration label.full-seal-e2ejob (helm-ci)workflow_dispatchonly, never PRs — fork PRs can't read secrets, and a real dev-backend login per PR is platform churn for no extra signal (the secret-free enforcement probe keeps covering PRs).::noticeuntil provisioned — the job is live the moment the secrets exist, no second PR needed.workflow_dispatchadded to helm-ci for on-demand runs (e.g. right after provisioning).Refactor + docs
e2e-seal-check.shintolib/e2e-common.sh(provenance + test(seal-check): run the egress-enforcement probe live in k3d CI (#1184) #541 review rationale documented at the function) — one copy, two callers, no drift.SEAL-CHECK.md: new CI-coverage map (what runs where, activation contract, "never a person's shared dev identity — the backend#1180 token-churn failure class"), and the follow-ups list drops three items that shipped since it was written (test(seal-check): run the egress-enforcement probe live in k3d CI (#1184) #541 live probe, cli#393 CLI verdict, cli#449 §8.3 matrix).installer-tests.yamlshellcheck gates now include the new script; helm-ci path filters gain the new script +lib/e2e-common.sh(which was already load-bearing for the seal job but absent from the filter).Provision a dedicated dev-platform client (
e2e-test-agent) and add two repo Actions secrets:TB_E2E_CLIENT_ID,TB_E2E_CLIENT_PASSWORD. Until then the job skips green by design. (Public repo: fork PRs can't read the secrets; the job never runs on PRs anyway.)Test plan
bash -n+shellcheck --severity=errorgreen on all three touched scripts (both CI shellcheck lists updated).scripts/tests/isn't manifest-covered); style guard clean.seal-check-e2e) runs in THIS PR's CI — proving the factored positive control live on k3d.Epic: tracebloc/backend#1151 (workstream G residual) · closes the backend#1184 fast-follow · RFC-CLI-0003 D12 §8.2
🤖 Generated with Claude Code
Note
Low Risk
Changes are CI/e2e harness and documentation only; no chart runtime or production install path. Real credentials stay in Actions secrets and a dedicated dev test client, with PRs excluded from the credentialed job.
Overview
Adds
e2e-full-seal.shand a helm-cifull-seal-e2ejob to run the complete chart seal suite (egress-enforcement,backend-reachability,storage-assertions) on k3d against the dev backend with dedicatede2e-test-agentcredentials—not just the secret-free egress probe PRs already get.The new script installs the working-tree chart with real creds in a mode-0600 values file, waits for PVCs to bind and for
jobs-managerrollout, verifies all three test hooks exist (guards against silent emptyhelm test), then runs unfilteredhelm testwith failure log dumps via the seal-check label.CI wiring:
workflow_dispatchon helm-ci; path filters include the new script ande2e-common.sh; installer shellcheck lists updated. The job runs only on push/dispatch (not PRs), uses 45m timeout and concurrency so one dev-agent session runs at a time, and skips green with a notice untilTB_E2E_CLIENT_ID/TB_E2E_CLIENT_PASSWORDsecrets exist.Refactor: egress positive-control logic moves from
e2e-seal-check.shintoe2e_egress_positive_controline2e-common.shfor both seal e2e scripts.Docs:
SEAL-CHECK.mdgains a CI coverage map and trims follow-ups that shipped elsewhere.Reviewed by Cursor Bugbot for commit d639aca. Bugbot is set up for automated code reviews on this repo. Configure here.