Uh oh!
There was an error while loading. Please reload this page.
docs(seal-check): record k3d/k3s NetworkPolicy substrate verification (backend#1184) - #504
Conversation
… (backend#1184) RFC-0003 §8.4 says 'do not assume k3d enforces NetworkPolicy'. Ran the substrate check on a throwaway k3d v5.8.3 / k3s v1.33.6+k3s1 cluster: a standalone deny-egress NetworkPolicy on a probe pod took a curl to 1.1.1.1:443 reachable -> BLOCKED -> reachable-again-after-removal (HTTP 301 -> connect failure -> HTTP 301). The block is attributable to the policy, so k3s's embedded kube-router controller does enforce egress NetworkPolicy on this k3d version — the substrate doubt is resolved. Records the result in the §8.4 runbook + the follow-ups list; the §8.3 k3d cell now reads 'substrate verified; full-probe run pending'. Recording the full-chart egress-enforcement probe run against a deployed release stays open (still backend#1184). Docs-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka
commented
Jul 30, 2026
👋 Heads-up — Code review queue is at 33 / 30 Above the WIP limit. The team convention is to review existing PRs before opening new work. Open PRs currently in Code review (oldest first):
Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.) |
saqlainsyed007
left a comment
There was a problem hiding this comment.
Docs-only review of docs/SEAL-CHECK.md. The experiment itself is sound and honestly scoped (curl to an IP so no DNS is needed; reachable→blocked→reachable proves attribution). But the change asserts an edit to the §8.3 matrix cell that it never actually makes, leaving the document self-contradictory. Details inline.
🤖 Generated with Claude Code
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…ain #504) - Update the §8.3 k3d matrix cell (was 'verification run pending; do not assume') to 'Substrate verified; full-probe run pending', matching the prose that cross-references it — the concrete edit the PR promised but never made (Saqlain #i). - Status note now LEADS with 'still UNSEALED for the egress guarantee on k3d until the full-chart probe is recorded', honoring this doc's unsealed-never-silently-sealed philosophy — a skimmer can no longer read the bold VERIFIED as 'sealed' (Saqlain #ii). - State the run evidence (versions + reachable->blocked->reachable) ONCE in the status note; the post-runbook paragraph, the follow-ups bullet, and the matrix cell now reference it instead of restating — which is how the cell drifted out of sync in the first place (Saqlain #iii). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka
commented
Jul 31, 2026
Thanks — all three addressed in
|
saqlainsyed007
left a comment
There was a problem hiding this comment.
All three review findings addressed in b30bdf9:
- §8.3 k3d matrix cell now reads "substrate verified; full-probe run pending" — matches the prose that references it; self-contradiction resolved.
- Status block now leads with the explicit "still UNSEALED for the egress guarantee on k3d" framing, restoring the doc's "never silently sealed" philosophy.
- Evidence now lives in a single record (the §8.4 Status note); the post-runbook paragraph and follow-ups reference it instead of restating.
Clean, root-cause fixes.
🤖 Generated with Claude Code
Uh oh!
There was an error while loading. Please reload this page.
LukasWodka
commented
Aug 2, 2026
/fr-pass |
…184) (#541) * test(seal-check): run the egress-enforcement probe live in k3d CI (#1184) RFC-0003 D12: the chart's enforcement probes ship as `helm.sh/hook: test` Jobs, but `helm test` ran nowhere in CI — SEAL-CHECK §8.4 recorded the k3s NetworkPolicy substrate as verified (#504) while the full-chart egress probe run stayed "pending". This closes that gap. - scripts/tests/e2e-seal-check.sh: install the local chart on a real k3d cluster with public images + the egress lockdown engaged (allowExternalHttps=false), then `helm test --filter` the egress-enforcement seal-check. Requires BOTH a zero exit AND the probe's `OK egress lockdown verified` marker in the logs — guarding the helm-test-`--filter`-matches-nothing silent-pass trap. - .github/workflows/helm-ci.yaml: a `seal-check-e2e` job mirroring upgrade-e2e (stock ubuntu runner, zero secrets — public curl vs 1.1.1.1). - client/tests/egress_enforcement_check_test.yaml: pin the probe Job's metadata.name so the e2e --filter can never silently drift off it. Local: shellcheck clean · helm template renders the probe Job · helm-unittest 27 suites / 320 tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): install k3d/helm/kubectl before create_cluster (exit 127) The sourced libs define install_kubectl/install_k3d/install_helm but do not call them; create_cluster + helm need the binaries on PATH first, and a stock runner has none preinstalled. Mirror e2e-auto-upgrade.sh's prerequisite block + the post-create node-ready wait. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): drive helm test off exit code, not --logs (Job hook) The egress-enforcement probe is a Job-type test hook with hook-delete-policy hook-succeeded. `helm test --logs` looks up the pod by the Job's bare name, but a Job's pod has a generated suffix ("pods not found"), and Helm deletes the Job on success anyway — so --logs errored even though the probe passed. Drive off the exit code instead (the probe exits 0 only when egress is verified blocked), guard the --filter-silent-pass by asserting the hook is in `helm get hooks` first, and dump the pod log via kubectl only on failure (the Job persists when it fails). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): address Bugbot review (autostart, paths, shellcheck) Three Bugbot findings on the new e2e seal-check: - Medium: set TRACEBLOC_NO_AUTOSTART=1 before create_cluster (like the sibling e2e-*.sh) so it never mutates the host's Docker restart policy / runs `systemctl enable docker`. - Medium: add scripts/tests/e2e-seal-check.sh to helm-ci.yaml on.push/ on.pull_request paths so script-only edits re-trigger the k3d job. - Low: enumerate the script in the installer-tests ShellCheck gate (both the error and warning passes), matching the other e2e entrypoints. The High finding ("Job logs never reach marker check") was already resolved in the prior commit — the script drives off `helm test`'s exit code and no longer greps --logs output. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): avoid SIGPIPE/pipefail false-fail in hook guard (Bugbot) `helm get hooks | grep -q` lets grep close the pipe on first match, SIGPIPE-ing helm mid-write; under set -o pipefail that false-fails the guard even when the hook exists. Capture helm output to a var, then grep a here-string — no pipe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(seal-check): bump job timeout 20m->30m (Bugbot) The script's own bounds (create_cluster up to 15m + helm test 360s + tool install + helm install) can exceed a 20m GHA cap on a slow cluster bring-up, false-failing even while each component is inside its own timeout. Match the sibling k3d job (upgrade-e2e = 30m). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): positive control + probe timeout + lib paths + NS (Saqlain) - Positive control (blocking): before trusting a BLOCKED probe, prove the cluster can REACH the host from a non-policied pod (in `default`, ungoverned by the namespace-scoped training-egress policy). Without it, egress failing for an unrelated reason (runner firewall / target outage / rate-limit) would make the probe print OK and pass green while the NetworkPolicy did nothing. A reachable positive + a blocked training pod = the block is attributable to the policy. Positive-control failure now fails the seal-check as inconclusive. - Bump enforcementProbeTimeoutSeconds 60s->240s (blocking): on a cold GHA runner k3s can take >60s to program the pod iptables while the chart installs; the probe is single-shot (backoffLimit 0), so 60s false-fails. 240s is well inside the 360s helm-test budget. - Add scripts/lib/** to helm-ci paths (blocking): the script sources scripts/lib/{common,setup-linux,cluster,preflight}.sh, so a lib-only edit must re-trigger seal-check-e2e + upgrade-e2e (both depend on it). - Derive NS from CLUSTER_NAME so a CLUSTER_NAME override isolates a run under one name instead of desyncing cluster vs release/namespace. shellcheck clean; workflow parses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): --tlsv1.2 (house rule) + SA wait + required Lint list (Bugbot) - Add --tlsv1.2 to the positive-control curl — the curl-tls house rule (the required quality/house-rules gate) rejects a curl that could negotiate a downgraded TLS version. - Wait for the default ServiceAccount before the positive-control kubectl run: a fast runner can schedule the pod before the SA exists, aborting under set -e with 'serviceaccount default not found' before the attribution message. - Add e2e-seal-check.sh to the standard-checks.yml Lint shellcheck list (the REQUIRED branch-protection gate) — installer-tests had it, the required Lint did not, so a shellcheck regression in the new script could miss the gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Summary
Records the k3d/k3s NetworkPolicy substrate verification that RFC-0003 §8.4 explicitly demands ("do not assume k3d enforces NetworkPolicy"). Docs-only — one file.
What was run
On a throwaway
k3d v5.8.3cluster (k3s v1.33.6+k3s1), a standalone deny-egressNetworkPolicy(podSelector on a probe pod,policyTypes: [Egress], emptyegress:) was applied and acurlfrom the pod to1.1.1.1:443was measured across three states:The reachable → blocked → reachable transition makes the block attributable to the policy, not a fluke or an image-pull artifact. So k3s's embedded (kube-router) controller does enforce egress NetworkPolicy on this k3d version — flannel handles pod networking, but the netpol controller is active and enforcing.
Honest scope (deliberately precise)
This verifies the substrate primitive — the specific doubt §8.4 raises. It is not the full-chart
egress-enforcementprobe run against a deployed release; that end-to-end run stays open under backend#1184, and I've kept the §8.3 k3d cell reading "substrate verified; full-probe run pending" rather than overclaiming "sealed".Why it matters
The chart's
egress-enforcementseal-check relies on the CNI actually dropping egress. That reliance was documented-but-unverified ("expected is not verified"). It's now verified for the k3d/k3s substrate, so the local-dev "nothing gets out" guarantee rests on measured behavior, not an assumption.Epic: backend#1151 · Ticket: backend#1184 (D12)
🤖 Generated with Claude Code
Note
Low Risk
Documentation-only change to seal-check status and runbook text; no runtime, chart, or security behavior changes.
Overview
docs/SEAL-CHECK.mdnow records a completed k3s egress NetworkPolicy substrate check on k3d v5.8.3 / k3s v1.33.6+k3s1 (2026-07-30): deny-egress policy produced reachable → blocked → reachable again oncurlto1.1.1.1:443, with that evidence centralized in the §8.4 Status note.The §8.3 k3d matrix cell and follow-ups are updated to substrate verified; full-chart
egress-enforcementprobe still pending, so k3d remains unsealed for the egress guarantee until the deployed-release runbook (steps 1–4) is recorded—without overclaiming sealed status.Reviewed by Cursor Bugbot for commit b30bdf9. Bugbot is set up for automated code reviews on this repo. Configure here.