Skip to content

sec(ci): actor-gate the k3s-cuda publish dispatch (backend#1857) - #696

Merged
LukasWodka merged 2 commits into
developfrom
sec/1857-gate-k3s-cuda-dispatch
Aug 13, 2026
Merged

sec(ci): actor-gate the k3s-cuda publish dispatch (backend#1857)#696
LukasWodka merged 2 commits into
developfrom
sec/1857-gate-k3s-cuda-dispatch

Conversation

@LukasWodka

@LukasWodkaLukasWodka commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Part of tracebloc/backend#1857.

The breach

build-k3s-cuda.yaml is dispatch-only and pushes ghcr.io/tracebloc/k3s-cuda. Nothing authorized the dispatcher. That breaches the rule decided on backend#1422 (Lukas, 2026-08-05) and restated at the top of release-train's PUBLISH-PATHS.md:

no un-gated workflow_dispatch that publishes.

Why it was missed: it was added one day after the 2026-08-05 inventory was measured, and the client repo had no rows in PUBLISH-PATHS.md at all — the org's most customer-visible publish path. It was found by release-train's new publish-inventory-check.sh (tracebloc/release-train#69), which compares the inventory against every workflow in the fleet instead of trusting that it was kept current.

The gate

Copied, not reinvented, from the fleet's canonical block — data-ingestors/release-image.yml, backend/docker-build.yml, client-runtime/publish-images.yml all carry it verbatim. Three properties, each load-bearing:

propertywhy
a failing step, not a skipped jobred and auditable; a skipped job reports green and hides the denial (backend#1424)
bothgithub.actorandgithub.triggering_actoractor stays the original dispatcher on a re-run while triggering_actor is whoever clicked it — checking one lets a non-allowlisted user replay an allowlisted dispatch (backend#1536)
gated on inputs.pushbuild-only validation stays open to everyone. The rule is about publishing, not building

Verified, not assumed

Ran every path against the exact shell the step executes:

actor=LukasWodka trigger=LukasWodka -> ALLOWED
actor=saadqbal trigger=saadqbal -> ALLOWED
actor=waqaskhanroghani trigger=waqaskhanroghani -> DENIED
actor=LukasWodka trigger=waqaskhanroghani -> DENIED <- the replay case
actor=waqaskhanroghani trigger=LukasWodka -> DENIED
actor=<empty> trigger=<empty> -> DENIED <- fail-closed

The fourth row is the one that matters: an allowlisted dispatch re-run by anyone else is refused. That is the failure mode a single-identity check silently permits.

make check: green actionlint: clean scripts/manifest.sha256: up to date

A bigger gap this does NOT close

This workflow has no ref restriction. Its only if: was inputs.push, so a dispatch from any branch could publish — with the image tag assembled from two user-supplied inputs (k3s_tag, cuda_tag), and that constructed tag (v1.29.4-k3s1-cuda-12.4.1-base-ubuntu22.04) is the one the GPU installer pins.

This PR reduces that from any writer to two admins, which is a real reduction but not a fix. Confining publishes to a reviewed ref — as client-runtime/build-mysql-client.yml does with if: github.ref == 'refs/heads/develop' — is the natural companion. I have deliberately not done it here: choosing which ref is a policy decision rather than a copy of an existing pattern, and it would change who can legitimately use this workflow. Happy to add it in this PR if you name the ref.

Related correction

The sweep that produced backend#1857 named two un-gated dispatch publishers. On inspection the second one — client-runtime/build-mysql-client.yml — is already contained by construction: if: github.ref == 'refs/heads/develop' means a dispatch can only republish what develop already is, which is precisely the reasoning PUBLISH-PATHS.md already accepts as compliant for data-ingestors/publish-images.yml. It needs an inventory row, not a gate. Correcting that on backend#1857 and in release-train#69 rather than gating a workflow that does not need it.


Note

Medium Risk
Touches customer-pulled container publish controls; misconfiguration could block legitimate releases or still leave tag-mutability gaps called out in backend#1867.

Overview
Adds a fail-closed publish authorization step to the manual build-k3s-cuda workflow so pushing ghcr.io/tracebloc/k3s-cuda is no longer available to any repo writer with dispatch access.

When inputs.push is true, the new step runs before checkout/GHCR login and denies the job unless bothgithub.actor and github.triggering_actor are on the allowlist (LukasWodka, saadqbal), blocking replay of an allowlisted run by someone else on re-run. It also refuses publish unless the workflow runs on staging or main, since GPU installs pull that mutable tag at cluster-create.

Build-only dispatches (push=false) are unchanged—no actor/ref gate on validation builds.

Reviewed by Cursor Bugbot for commit e865cdc. Bugbot is set up for automated code reviews on this repo. Configure here.

build-k3s-cuda.yaml is dispatch-only and pushes
ghcr.io/tracebloc/k3s-cuda, and until now nothing authorized the
dispatcher. That breaches the decided rule (backend#1422, 2026-08-05):
no un-gated workflow_dispatch that publishes.
It was missed because it was added one day AFTER the 2026-08-05 inventory
was measured, and the client repo had no rows in PUBLISH-PATHS.md at all
-- the org's most customer-visible publish path. Found by
release-train's publish-inventory-check.sh, which now compares the
inventory against every workflow in the fleet.
The gate is the fleet's canonical block, copied not reinvented from
data-ingestors/release-image.yml, backend/docker-build.yml and
client-runtime/publish-images.yml:
- a failing STEP, not a skipped job: red and auditable rather than a
green run that hides the denial (backend#1424)
- BOTH github.actor and github.triggering_actor must pass, because
actor stays the ORIGINAL dispatcher on a re-run while
triggering_actor is whoever clicked it -- checking one lets a
non-allowlisted user replay an allowlisted dispatch (backend#1536)
- gated on `inputs.push`, so build-only validation stays open to
everyone; the rule is about publishing, not about building
Exercised every path against the exact shell the step runs:
actor=LukasWodka trigger=LukasWodka ALLOWED
actor=saadqbal trigger=saadqbal ALLOWED
actor=<other> trigger=<other> DENIED
actor=LukasWodka trigger=<other> DENIED <- the replay
actor=<other> trigger=LukasWodka DENIED
actor=<empty> trigger=<empty> DENIED
NOT fixed here, and worth a separate decision: this workflow has NO ref
restriction. Its only `if:` was `inputs.push`, so a dispatch from ANY
branch could publish, with the image tag built from two user-supplied
inputs -- and that constructed tag is what the GPU installer pins. The
allowlist reduces this from "any writer" to "two admins"; confining
publishes to a reviewed ref (as client-runtime/build-mysql-client.yml
does with `github.ref == 'refs/heads/develop'`) is the natural companion
and is a policy call, not a copy of an existing pattern.
make check: green. actionlint: clean. manifest.sha256: up to date.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LukasWodkaLukasWodka self-assigned this Aug 13, 2026
The actor allowlist bounded WHO could publish. This bounds WHAT can be
published, which is the control that actually matters here.
install-k8s.ps1:672 defaults $K3S_CUDA_IMAGE to
ghcr.io/tracebloc/k3s-cuda:$K8S_VERSION-cuda-$CUDA_BASE_TAG, and its own
comment states: "The installer PULLS this image automatically at
cluster-create -- the user never builds or pulls anything by hand." So
this tag is fetched onto customer machines. Before this change the only
`if:` on the job was `inputs.push`, so an allowlisted dispatcher could
publish it from ANY branch -- putting unreviewed content behind a
customer-pulled tag.
staging AND main, not develop:
* both are train-owned and protected. develop is the least-reviewed
integration branch and has no business behind a customer-pulled tag.
* staging is included deliberately, to avoid an ordering trap rather
than to be lenient. The tag encodes K8S_VERSION + CUDA_TAG, and
check-facts.sh pins those across install-k8s.ps1, the Dockerfile,
build.sh and this workflow -- so a bump lands in all four at once.
Publishing from staging lets the image for a new tag exist BEFORE
main's installer starts asking for it. main-only would leave a window
where prod installs pull a tag nobody has pushed yet.
Build-only validation is untouched: push=false runs from any branch, by
anyone, and the error message says so. The rule is about publishing.
Both checks live in ONE step under one `inputs.push` condition, so the
gate cannot drift out of sync with the thing it guards.
Full matrix exercised against the exact shell:
LukasWodka / LukasWodka / refs/heads/main ALLOWED
LukasWodka / LukasWodka / refs/heads/staging ALLOWED
saadqbal / saadqbal / refs/heads/staging ALLOWED
LukasWodka / LukasWodka / refs/heads/develop DENIED
LukasWodka / LukasWodka / refs/heads/feat/anything DENIED
LukasWodka / LukasWodka / refs/pull/1/merge DENIED
<other> / <other> / refs/heads/main DENIED
LukasWodka / <other> / refs/heads/main DENIED <- replay
NOT closed by this, and filed as backend#1867: the tag stays mutable and
the installer pins it by TAG, not digest. Two admins on a protected branch
is a process control standing in for a technical guarantee. The org has
solved this twice already -- prodDigest for the ingestor, and
RFC-BACKEND-1246 for training images -- and k3s-cuda is the remaining
customer-pulled image that is neither digest-pinned nor immutable-tagged.
make check: green. actionlint: clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
ContributorAuthor

Added the ref restriction — I picked staging + main rather than handing the choice back. Here is the reasoning, so it is easy to overrule.

Why a ref restriction is not optional.install-k8s.ps1:672 defaults $K3S_CUDA_IMAGE to ghcr.io/tracebloc/k3s-cuda:$K8S_VERSION-cuda-$CUDA_BASE_TAG, and the comment three lines above it says: "The installer PULLS this image automatically at cluster-create — the user never builds or pulls anything by hand." So this tag lands on customer machines. Build-GpuNodeImage is a local fallback, not the primary path. An actor allowlist alone still let two admins publish it from any branch.

Why staging and main, and not develop:

  • Both are train-owned and protected. develop is the least-reviewed integration branch and has no business behind a customer-pulled tag.
  • staging is included to avoid an ordering trap, not to be lenient. The tag encodes K8S_VERSION + CUDA_TAG, and check-facts.sh pins those across install-k8s.ps1, the Dockerfile, build.sh and this workflow — so a bump lands in all four together. Publishing from staging lets the image for a new tag exist before main's installer starts asking for it. Restricting to main only would leave a window where prod installs pull a tag nobody has pushed yet.

If you would rather it were main only, that is a two-word change and I will make it — just be aware of that window.

Build-only validation is untouched:push=false still runs from any branch, by anyone, and the denial message says so explicitly. Both checks live in one step under a single inputs.push condition, so the gate cannot drift out of sync with what it guards.

Full matrix, run against the exact shell the step executes:

actortriggering_actorrefresult
LukasWodkaLukasWodkarefs/heads/mainALLOWED
LukasWodkaLukasWodkarefs/heads/stagingALLOWED
saadqbalsaadqbalrefs/heads/stagingALLOWED
LukasWodkaLukasWodkarefs/heads/developDENIED
LukasWodkaLukasWodkarefs/heads/feat/anythingDENIED
LukasWodkaLukasWodkarefs/pull/1/mergeDENIED
otherotherrefs/heads/mainDENIED
LukasWodkaotherrefs/heads/mainDENIED ← replay

What this still does not fix, filed as backend#1867: the tag remains mutable, and the installer pins it by tag, not digest. Two admins on a protected branch is a process control standing in for a technical guarantee — any future publish silently changes what every subsequent GPU install pulls, and a rollback is not expressible because there is no second identifier to point at. We have solved this twice already (prodDigest for the ingestor, RFC-BACKEND-1246 for training images); k3s-cuda is the last customer-pulled image that is neither digest-pinned nor immutable-tagged.

make check green, actionlint clean.

@saadqbalsaadqbal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate itself is tight: both github.actor and github.triggering_actor are exact-matched via case (so a re-run replay of an allowlisted dispatch is covered), the ref is pinned to staging/main, and all three publish signals — this gate, the GHCR login, and build.sh's docker push — are keyed to inputs.push, so they stay in sync. Fails closed, checkout is SHA-pinned, actionlint clean.

One non-blocking altitude note: an in-file if: gate on a workflow_dispatch that publishes with the ambient GITHUB_TOKEN only binds the honest dispatch path — it can't stop a write-access user who runs a modified copy of the workflow from their own ref, since dispatch uses the workflow file on the selected ref and the token's packages: write is always present. The durable control for that is a protected Environment with the push credential as an environment secret, or OIDC / package-level write restriction scoped to staging/main. Good to land this as defense-in-depth given backend#1867 already tracks the mutable-tag half — worth a follow-up ticket for the credential scoping. 👍

@LukasWodka
LukasWodka merged commit b606631 into developAug 13, 2026
37 checks passed
@LukasWodka
LukasWodka deleted the sec/1857-gate-k3s-cuda-dispatch branch August 13, 2026 07:14
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@LukasWodka@saadqbal