Uh oh!
There was an error while loading. Please reload this page.
Sync develop → main for v1.8.1 chart release (egress-enforcement-check standard-mode CNI fix) - #278
Merged
Merged
Conversation
…CNI reconciliation window (#276) * fix(egress-enforcement-check): retry probe to tolerate standard-mode CNI reconciliation window The SECURITY §8.2 egress-lockdown enforcement helm test false-failed on clusters whose CNI enforces egress NetworkPolicy in "standard" mode — notably AWS VPC CNI with NETWORK_POLICY_ENFORCING_MODE=standard, which both the dev and prod fleets use. Standard mode allows a brand-new pod all traffic until its per-pod policy reconciles (a few-second startup window); the probe curled the canary host instantly on startup, won that race, connected, and reported "EGRESS LOCKDOWN NOT ENFORCED" — even though the lockdown is enforced for real training pods, which start far slower than the window. Retry the probe until egress is observed blocked, bounded by a new networkPolicy.training.enforcementProbeTimeoutSeconds (default 60s): PASS as soon as the connect is blocked, FAIL only if the host stays reachable for the whole timeout. Correct on standard mode, strict mode, and genuinely non-enforcing clusters alike. Also refresh the stale values.yaml CNI guidance (AWS VPC CNI does enforce with enableNetworkPolicy=true) and extend helm-unittest (full suite 267 pass). Verified on dev (EKS, tb-client-dev-templates): a training-labelled pod that sleeps 30s then probes is blocked on 1.1.1.1:443 and example.com:443, and reaches the backend via the squid gateway. Refs #275, tracebloc/client-runtime#104, tracebloc/client-runtime#102. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(egress-enforcement-check): NOT-ENFORCED verdict outranks a late DNS failure Bugbot (PR #276): the post-loop inconclusive branch keyed off only the last curl exit code. If an earlier attempt observed the host reachable (rc=0) but a later attempt failed DNS (rc=6), the hook reported "inconclusive / no TCP attempted" instead of "NOT ENFORCED" — a misdiagnosis that could send an operator chasing DNS when the CNI simply isn't enforcing egress. Latch a `reached` flag whenever a probe establishes a TCP connection (any curl exit other than 6) and branch on it after the loop: a connect seen on ANY attempt yields the definitive NOT-ENFORCED verdict; INCONCLUSIVE is reserved for the case where no TCP connection ever succeeded. helm-unittest updated. Refs #275. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Readies the 1.8.1 release, which ships the egress-enforcement-check standard-mode CNI fix (#276, closes#275): the §8.2 helm-test gate now retries through the per-pod NetworkPolicy reconciliation window instead of racing it, so it no longer false-fails on AWS VPC CNI standard mode (the dev and prod fleets both use it). Validated end-to-end on dev — the patched `helm test` goes green where the previous instant-probe test failed. Refs #275. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
aptracebloc
approved these changes
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Syncs
develop → mainfor the v1.8.1 chart release.What 1.8.1 ships
developis exactly two commits ahead ofmain:1.8.0 → 1.8.1(version + appVersion).egress-enforcement-checkstandard-mode CNI fix (the substantive change).The §8.2 egress-lockdown
helm testgate was false-failing on CNIs that enforce egress NetworkPolicy instandardmode (notably AWS VPC CNI withNETWORK_POLICY_ENFORCING_MODE=standard— both the dev and prod fleets). The probe ran instantly on pod startup, inside the per-pod policy reconciliation window, connected, and reportedEGRESS LOCKDOWN NOT ENFORCED. #276 makes the probe retry until egress is observed blocked (bounded byenforcementProbeTimeoutSeconds, default 60s), so it measures steady-state enforcement. This is the gate used to verify each fleet during the client-runtime#102 egress-lockdown rollout, so it must ship before the per-fleet flips can rely onhelm test.Closes#275.
Validation
standardmode): the patchedhelm testgoes green where the previous instant-probe failed; a live egress-restriction demo confirmed a training pod's direct external calls are all blocked and only the allowlisted backend is reachable via the squid gateway. Dev rolled back to its original state afterward.After merge
Publish the v1.8.1 GitHub release (tag on
main) → triggersrelease-helm-chart.yamlto package + publish to the helm repo. Then the client-runtime#102 per-fleet rollout can run with a trustworthy gate.🤖 Generated with Claude Code
Note
Low Risk
Changes only the helm test hook script and chart metadata/docs; no runtime training workload or NetworkPolicy rules are modified.
Overview
Releases client chart v1.8.1 and fixes false failures in the §8.2
helm testegress-enforcement-check on CNIs that enforce egress only after a short per-pod reconcile (e.g. AWS VPC CNINETWORK_POLICY_ENFORCING_MODE=standard).The probe retries
curlto the canary host until TCP egress is blocked (pass) ornetworkPolicy.training.enforcementProbeTimeoutSeconds(default 60s) elapses while still reachable (fail). Areachedlatch records any successful TCP connect so a late DNS failure cannot downgrade a definitive “not enforced” result to inconclusive.values.yamldocuments the new knob and updated EKS VPC CNI guidance; helm-unittest asserts the loop, timeout wiring, and default.Reviewed by Cursor Bugbot for commit 2f22398. Bugbot is set up for automated code reviews on this repo. Configure here.