Skip to content

fix(#459): isolate e2e path 2 from path 1's --reuse-values replay contamination - #529

Merged
shujaatTracebloc merged 2 commits into
developfrom
fix/459-reuse-values-replay
Aug 3, 2026
Merged

fix(#459): isolate e2e path 2 from path 1's --reuse-values replay contamination#529
shujaatTracebloc merged 2 commits into
developfrom
fix/459-reuse-values-replay

Conversation

@shujaatTracebloc

@shujaatTraceblocshujaatTracebloc commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

#459 — chart-default pin updates never reach an edge once hand-upgraded with --reuse-values

helm upgrade --reuse-values records the previous release's computed values (chart defaults + overrides, materialized) as the new release's user-supplied values. From then on, every --reset-then-reuse-values upgrade — exactly what the fleet auto-upgrade CronJob runs — replays those frozen defaults as if an operator set them, winning over the new chart's defaults. So an edge ever hand-upgraded with --reuse-valuessilently stops receiving chart-default updates — most importantly images.ingestor.prodDigest.

Where it bit: the e2e tripwire

e2e-auto-upgrade.sh modelled this on one release: path 1 (--reuse-values) contaminated the recorded values, then path 2 (the fleet auto-upgrade) ran on that same contaminated release. Path 2's pin assertion passed only because the baseline pin and the working-tree pin coincided — the ERA NOTE tripwire the issue documents: the first prodDigest bump would trip it.

Fix (issue decision 1 — isolate path 2 from path 1)

Between the paths, reset the recorded values to just the genuine install-time overrides:

helm upgrade "$NS" "$CHART_DIR" --reset-values \
--set clientId=… --set clientPassword=… --set storageClass.provisioner=…

so paths 2–4 assert clean-edge auto-upgrade behavior — the fleet's real contract on an edge no one hand-upgraded. The working-tree pin now lands on path 2 regardless of the baseline era; no false trip on a future bump.

Added an assertion using the issue's own contamination fingerprinthelm get valueswithout--all must not carry the chart-default prodDigest key after the reset — and rewrote the ERA NOTE to document the resolution.

A contaminated real edge (one an operator hand-upgraded with --reuse-values) stays a separate fleet-audit concern; its remediation is exactly this reset (--reset-values + re-apply intended overrides), noted in-code.

Acceptance criteria

Gates

  • shellcheck --severity=error, check-style, bash -n clean.
  • Note: e2e-auto-upgrade.sh runs in the label-gated e2e job (real k3d cluster + published chart), so it isn't exercised by default PR CI — the change is shellcheck-clean and follows the script's existing idioms/assert-from-live-cluster pattern.

Closes#459


Note

Low Risk
Test-only change to e2e-auto-upgrade.sh; no production Helm or chart behavior is modified.

Overview
The fleet auto-upgrade e2e gate used one Helm release for both manual --reuse-values (path 1) and fleet --reset-then-reuse-values (path 2), so path 1’s contamination of recorded values could make path 2’s prod pin assertion pass only when baseline and working-tree pins matched.

Between path 1 and path 2, the script now runs helm upgrade against the published baseline chart ($PREV) with --reset-values plus the three real install overrides (clientId, clientPassword, storageClass.provisioner), so new chart defaults are not pre-applied from the local tree. It fails if images.ingestor.prodDigest still appears in user-supplied values (helm get values without --all). Paths 2–4 then exercise clean-edge auto-upgrade behavior; comments replace the old ERA NOTE tripwire with #459 resolution notes.

Reviewed by Cursor Bugbot for commit 5b6ee21. Bugbot is set up for automated code reviews on this repo. Configure here.

…tamination
`helm upgrade --reuse-values` records the previous release's COMPUTED values (chart
defaults + overrides) as the new release's user-supplied values. Every later
--reset-then-reuse-values (the fleet auto-upgrade) then replays those frozen defaults as
if an operator set them, so an edge ever hand-upgraded with --reuse-values silently stops
receiving chart-default updates — most importantly images.ingestor.prodDigest.
e2e-auto-upgrade.sh modelled this on ONE release: path 1 (--reuse-values) contaminated
the recorded values, then path 2 (the fleet auto-upgrade) ran on that contaminated
release. Path 2's pin assertion passed only because the baseline pin and the working-tree
pin coincided — the ERA NOTE tripwire: the first prodDigest bump would trip it.
Resolution (issue decision 1 — isolate path 2 from path 1): between the paths, reset the
release's recorded values to just the genuine install-time overrides
(helm upgrade --reset-values --set clientId/clientPassword/storageClass), so paths 2-4
assert CLEAN-edge auto-upgrade behavior — the fleet's real contract on an edge no one
hand-upgraded. Added an assertion using the issue's own contamination fingerprint
(`helm get values` WITHOUT --all must not carry the chart-default prodDigest key), and
rewrote the ERA NOTE to document the resolution. A contaminated REAL edge stays a separate
fleet-audit concern; its remediation is exactly this reset.
shellcheck + check-style clean; bash -n ok. (Runs in the label-gated e2e job — needs a
real k3d cluster + the published chart, so it's not exercised by default PR CI.)
Closes#459
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shujaatTraceblocshujaatTracebloc self-assigned this Jul 31, 2026
@shujaatTracebloc
shujaatTracebloc marked this pull request as ready for review August 3, 2026 06:56

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 301c1f8. Configure here.

Comment threadscripts/tests/e2e-auto-upgrade.sh
… a real test (Bugbot)
The isolation reset targeted $CHART_DIR (the local working-tree chart), so it pre-applied
the local chart's new defaults (working-tree prod pin, egress gateway) BEFORE path 2. Path
2 then became a same-version no-op whose live prodDigest / "new defaults flowed" assertions
already held from the isolation step — a --reset-then-reuse-values → --reuse-values
regression would slip through (computed values would still carry the local pin). Reset to
the PUBLISHED chart ($PREV) instead, so path 2 is a genuine published→local upgrade that
MUST pull the new defaults for its assertions to pass.
shellcheck + check-style + bash -n clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shujaatTracebloc
shujaatTracebloc merged commit 5a6728a into developAug 3, 2026
45 checks passed
@shujaatTracebloc
shujaatTracebloc deleted the fix/459-reuse-values-replay branch August 3, 2026 07:11
@LukasWodka

Copy link
Copy Markdown
Contributor

/fr-pass

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@shujaatTracebloc@LukasWodka@saadqbal