Skip to content

feat(#585): re-home all images onto a private mirror via global.imageRegistry - #604

Merged
shujaatTracebloc merged 5 commits into
developfrom
fix/585-mirror-and-airgap
Aug 5, 2026
Merged

feat(#585): re-home all images onto a private mirror via global.imageRegistry#604
shujaatTracebloc merged 5 commits into
developfrom
fix/585-mirror-and-airgap

Conversation

@shujaatTracebloc

@shujaatTraceblocshujaatTracebloc commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

What & why

Restricted-network / air-gapped installs can now point every image the chart
pulls at a private registry mirror with a single knob, following the standard
global.imageRegistry convention. This is the mechanism for both the
"reachable private mirror" and the "fully air-gapped internal registry" cases
the preflight check (#601) directs operators toward.

Contributes to #585.

Chart

  • global.imageRegistry re-homes all images: tracebloc/*, the spawned
    ingestor (INGESTOR_IMAGE_REPOSITORY), the training-job image host
    (JOB_IMAGE_HOST), and the alpine/*, ubuntu/squid, busybox, curl
    helper images. New tracebloc.mirrorPrefix helper for the registry-less
    utility images.
  • Precedence: an explicit images.ingestor.repository still wins; an
    explicit per-image registry is overridden by the global mirror (Bitnami
    semantics).
  • Byte-identical default: empty/unset renders exactly as before
    (docker.io / ghcr.io, alpine/* unprefixed). An empty
    global.imageRegistry is hardened so JOB_IMAGE_HOST never degrades to /.
  • Documented global.imageRegistry in values.yaml.

Installer (one knob)

  • TRACEBLOC_IMAGE_REGISTRY writes global.imageRegistry into the generated
    values.
  • TRACEBLOC_REGISTRY_USERNAME / TRACEBLOC_REGISTRY_PASSWORD (optional
    TRACEBLOC_REGISTRY_SERVER, TRACEBLOC_REGISTRY_EMAIL) mint the chart's
    imagePullSecret; the pull-secret server is derived as https://<host>
    (the chart schema requires a URI, while global.imageRegistry is a bare host).
  • Bash _image_mirror_yaml and PowerShell Get-ImageMirrorYaml are at parity
    and were verified end-to-end through helm template (images re-homed, pull
    secret rendered, YAML single-quote escaping correct).

Docs

  • Rewrote the "Blocked container registry (mirror / air-gapped)" section in
    docs/INSTALL.md. The previous per-image images.<name>.registry override it
    documented was a no-op against the current chart; the section now shows the
    working global.imageRegistry knob (installer env var + plain-Helm forms) and
    a command to enumerate exactly which images to mirror.

Tests

  • helm-unittest: new global_image_registry_test.yaml (15 cases — mirror
    re-homing per template, byte-identical defaults, precedence guards).
  • bats: 7 cases for _image_mirror_yaml.
  • Pester: 7 cases for Get-ImageMirrorYaml.
  • Full suites green locally (helm-unittest 340 pass, Pester 427 pass); no new
    failures (the 4 pre-existing helm-unittest schema-negative failures are
    unrelated and also fail on develop).

🤖 Generated with Claude Code


Note

Medium Risk
Changes every image reference and runtime-spawned job pull paths; misconfiguration can break pulls cluster-wide, though unset mirror preserves prior defaults.

Overview
Adds global.imageRegistry (Bitnami-style) so restricted-network and air-gapped installs can point every chart image at one private mirror with a single value. Chart 1.9.14.

Chart: New tracebloc.mirrorPrefix for registry-less utility images (alpine/*). tracebloc.image call sites and CronJob image lines now use global.imageRegistry where applicable. jobs-manager gets dynamic INGESTOR_IMAGE_REPOSITORY (mirror re-homes default ghcr.io/tracebloc/ingestor unless images.ingestor.repository is set) and JOB_IMAGE_HOST (was hardcoded docker.io/). Egress squid registry precedence: global mirror wins; empty global.imageRegistry still falls through to per-image registry via | default.

Installers:TRACEBLOC_IMAGE_REGISTRY (and optional registry creds) emit mirror YAML in bash _image_mirror_yaml and PowerShell Get-ImageMirrorYaml, wired into generated values.

Docs:INSTALL.md mirror section documents the single knob and drops the old per-image override pattern that did not match the chart.

Tests: Helm unittest suite global_image_registry_test.yaml, plus bats/Pester for mirror YAML helpers.

Reviewed by Cursor Bugbot for commit d0a8035. Bugbot is set up for automated code reviews on this repo. Configure here.

…Registry
Restricted-network / air-gapped installs can now point every image the chart
pulls at a private registry mirror with a single knob.
- Chart: global.imageRegistry (Bitnami convention) re-homes tracebloc/*, the
spawned ingestor + training-job images (JOB_IMAGE_HOST), and the alpine/*,
ubuntu/squid, busybox and curl helper images. Empty/unset renders
byte-identical to before (docker.io / ghcr.io). An explicit
images.ingestor.repository still wins; an explicit per-image registry is
overridden by the global mirror. New tracebloc.mirrorPrefix helper for the
registry-less utility images.
- Installer one knob: TRACEBLOC_IMAGE_REGISTRY writes global.imageRegistry into
the generated values; TRACEBLOC_REGISTRY_USERNAME / TRACEBLOC_REGISTRY_PASSWORD
also mint the imagePullSecret (server derived as https://<host>). Bash
(_image_mirror_yaml) and PowerShell (Get-ImageMirrorYaml) parity.
- docs/INSTALL.md: rewrote the blocked-registry / air-gap section (the old
per-image images.<name>.registry override was a no-op against this chart).
- Tests: helm-unittest suite (15), bats (7), Pester (7).
Contributes to #585.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shujaatTraceblocshujaatTracebloc self-assigned this Aug 5, 2026
@shujaatTracebloc
shujaatTracebloc marked this pull request as ready for review August 5, 2026 09:04
shujaatTraceblocand others added 2 commits August 5, 2026 11:04
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadscripts/lib/install-client-helm.sh
… creds-only (Bugbot)
The chart schema requires dockerRegistry.server whenever create is true. When
registry credentials were set without a mirror or an explicit server, the
generated values omitted server and helm install failed with a schema error
instead of authenticating to Docker Hub. Default the server to
https://index.docker.io/v1/ in that case (bash + PowerShell), and always emit
it. Adds bats + Pester coverage; regenerates manifest.sha256.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1e090ef. Configure here.

Comment threadclient/templates/egress-proxy-deployment.yaml Outdated
…-image registry (Bugbot)
dig treats the chart-default empty global.imageRegistry as present, so the squid
image never fell back to egressProxy.image.registry — an explicit per-image
registry was silently dropped to docker.io on fresh installs and
reset-then-reuse upgrades. Resolve the mirror with a "" fallback and | default
through to the per-image registry (then docker.io); the global mirror still wins
when set. Adds a helm-unittest regression case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@saadqbal

Copy link
Copy Markdown
Contributor

Please make sure any change is backward compatible.

@shujaatTracebloc
shujaatTracebloc merged commit 5aedd2d into developAug 5, 2026
71 of 72 checks passed
@shujaatTracebloc
shujaatTracebloc deleted the fix/585-mirror-and-airgap branch August 5, 2026 10:03
LukasWodka added a commit that referenced this pull request Aug 5, 2026
git hoisted the shared closing brace out of the conflict region during the
rebase onto #604; the file then died at parse (1 of 88 tests ran).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka added a commit that referenced this pull request Aug 5, 2026
Semantic rebase conflict with #604: every other image include gained the
mirror dig while the guard (written pre-#604, merged clean textually)
kept a hardcoded docker.io — on mirrored/air-gapped edges the always-on
guard alone would ImagePullBackOff and block mysql on exactly the fleets
#604 serves. Same dig expression now + a mirror re-home pin test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasWodka added a commit that referenced this pull request Aug 6, 2026
…r format guard (backend#723 PR-2) (#597)
* feat(mysql): A2 engine split — 8.4 opt-in for fresh installs + datadir format guard (backend#723 PR-2)
Chart: mysql-format-guard init container fails fast (with an actionable
message) when the engine major and datadir format disagree — 8.4 over a
5.7-format datadir and 5.7 over an 8.x one are both refused before mysqld
CrashLoops; the 8.0 transit hop and custom digest pins stand down.
tracebloc.mysqlEngineMajor derives the expected engine (digest-wins,
mirroring tracebloc.image); the 5.7 digest literal is CI-pinned to the
values default. Default render changes by exactly the guard.
Installer (A2, decision 2026-08-05): _resolve_mysql_engine picks the engine
for the generated values — explicit TB_MYSQL_ENGINE wins; a previous 8.4
opt-in is sticky; any existing release or real datadir content pins 5.7;
only a fresh arm64 install auto-selects 8.4 (native multi-arch instead of
amd64 emulation). amd64 fresh installs stay 5.7 for now (soak first).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(installer): fail CLOSED on an unlistable mysql datadir in the engine resolver (Bugbot)
The auto branch treated a failed ls -A as an empty datadir — on arm64,
--reuse-data commonly leaves a uid-999 mysql dir the host user cannot
list, so the resolver opted the reuse into 8.4 and the format guard then
(correctly) refused the 5.7 datadir: the reuse path never came up. An
unlistable dir now counts as content (mirrors _leftover_data_dirs' fail-
closed stance for the same ownership case), with a chmod-000 regression
test. Rebased over #593/#527/#525 (manifest regenerated; my bats
negations now carry the #527 '|| return 1' enforcement idiom).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(bats): harden the 11 new engine-resolver assertions per the #527 hygiene guard
bats-hygiene's scanner requires every standalone bracket assertion in an
@test body to end in '|| return 1'; the resolver tests added on this
branch predated rebasing onto that guard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(tests): restore the brace the #604 merge seam ate (last mirror test)
git hoisted the shared closing brace out of the conflict region during the
rebase onto #604; the file then died at parse (1 of 88 tests ran).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(chart): the format guard honors global.imageRegistry (Bugbot)
Semantic rebase conflict with #604: every other image include gained the
mirror dig while the guard (written pre-#604, merged clean textually)
kept a hardcoded docker.io — on mirrored/air-gapped edges the always-on
guard alone would ImagePullBackOff and block mysql on exactly the fleets
#604 serves. Same dig expression now + a mirror re-home pin test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(chart): bump 1.9.20 -> 1.9.21, as this PR changes chart content
My earlier merge resolution took develop's 1.9.20 verbatim, reasoning from the
release train's rule: v1.9.20 is untagged, and the train's version_preflight
only refuses when the version is ALREADY released, so one bump covers a whole
release cycle. That reasoning is correct for the train and wrong for this repo.
client/scripts/chart-version-guard.sh enforces a stricter rule for a
repo-specific reason: chart content reaches installs only via a NEW chart
version, because a Helm repo publishes on version change. An unbumped
template/values edit therefore either reaches nobody or overwrites an
already-published version. Both have happened here - the perIngestionTables
block shipped dark in PR #472, and ingestor-0.2.0.tgz was overwritten 5x
between 2026-05-20 and 2026-07-29.
This PR changes client/templates/** and client/values.yaml, so it needs its
own version rather than riding develop's. v1.9.21 is untagged.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@shujaatTracebloc@saadqbal@LukasWodka