Uh oh!
There was an error while loading. Please reload this page.
refactor!: rework deploy/renovate as composite actions with validated targets - #162
Merged
Conversation
18 tasks
Deliberately a different input contract from deploy-shared.yml's per-target matrix, to compare the two before picking one: renovate.yml takes an app name and a targets directory, and deploy/renovate.py itself finds which targets currently run that app and re-pulls/recreates its containers there - no app_refs/env_refs re-resolution, no release rebuild, no vault decryption. Known gap: no Tailscale support yet.
…flow Match deploy.yml/deploy-shared.yml's two-file shape - renovate.yml is now just a workflow_dispatch trigger, and all the logic (checkout, running deploy/renovate.py) lives in renovate-shared.yml, called with secrets: inherit since it resolves secrets for matched targets itself rather than per-matrix-cell like deploy-shared.yml does.
Keep load-yaml-matrix a dumb, generic directory-to-matrix reader (no app-filtering added to it) - instead renovate-shared.yml gets a second job, mirroring deploy.yml/deploy-shared.yml's split: find-targets loads every target unfiltered, then renovate fans out over that matrix one job per target, with native per-matrix-cell secret/Tailscale resolution. Every target gets dispatched regardless of whether it runs the requested app; deploy/renovate.py itself decides to no-op when its one target's apps mapping doesn't contain it. This drops the directory-reading and GITHUB_SECRETS_JSON-parsing deploy/renovate.py used to do - it's now scoped to one already-resolved target, same shape as deploy/deploy.py.
deploy/renovate.py now compares `docker compose images -q` before/after the pull per host, so it can tell an actual update from a no-op pull, and reports updated/updated_hosts via $GITHUB_OUTPUT. renovate-shared.yml sends a Telegram message (via baseline's generic send-telegram-message action, same one the existing notify-telegram-*.yml workflows use) only when a host's image changed - silent on every no-op target-matrix cell.
…: inherit renovate.yml now computes the target matrix itself and resolves each matched target's secrets by name (secrets[matrix.credentials.secrets.X]), exactly like deploy.yml/deploy-shared.yml - renovate-shared.yml goes back to a single job with explicitly declared workflow_call secrets, so it only ever sees the one target's own resolved secret values, never the full secrets context. Dropped an earlier attempt to skip dispatching to a non-matching target via a matrix-conditioned job `if:` - actionlint (correctly) rejects `matrix` in the `if:` of a job that calls a reusable workflow via `uses:`, only github/inputs/needs/vars are available there. deploy/renovate.py's own `app in apps` check already makes a non-matching target a cheap no-op before any SSH connection happens, so this is a non-issue.
…ields
renovate.yml no longer exposes targets-directory as a dispatch input
(hardcoded to "targets", matching deploy.yml's own convention) and its
find-targets step id is now load-targets instead of the generic "matrix".
The bigger change: instead of flattening a target's hosts/apps/path into
separate with: inputs (each re-serialized via toJson(matrix.X)),
renovate.yml now just passes target-manifest: ${{ matrix.manifest }} -
the file path load-yaml-matrix already puts in every matrix item - and
renovate-shared.yml reads that file itself, the same way encrypt-env
takes a vault manifest path instead of flattened env fields. This drops
the with: block from 6 fields to 3 (app, target-manifest, tailscale
client id); the secrets: block is unchanged since secret values can only
ever come from an explicit named pass-through, never a re-read file.
renovate-shared.yml gets a second checkout (the caller's own repo, to
read that manifest) alongside the existing override checkout (flightdeck's
own code, now under .flightdeck/). deploy/renovate.py now takes
{app, target_manifest} on stdin and parses the YAML itself instead of
{app, hosts, path, apps} - it also derives and outputs target_name from
the manifest's filename, so the Telegram message no longer needs a
separate target-name input either.…elds
Apply the same simplification renovate-shared.yml just adopted, now that
the only real external caller (dupmachine/flightdeck) is also ours to
update: deploy-shared.yml's public contract changes from flattened
hosts/app-refs/apps/path/keep-releases inputs to a single target-manifest
path, read and parsed by the workflow itself - the same shape encrypt-env
already uses for vault manifests. Adds a second checkout (the caller's
own repo, to read that file) alongside the existing override checkout of
flightdeck's own code, now under .flightdeck/.
Also adds target-manifest-ref: release.yml pins deploy-targets' own
checkout to the just-published tag, so deploy-shared.yml's separate read
of the same manifest file needs to agree on that exact ref rather than
silently reading whatever the default branch's tip is by the time it runs.
release.yml's deploy job now also lists release in its own needs:, since
referencing needs.release.outputs requires a direct (not just transitive)
dependency.
deploy.py's main() now reads {target_manifest, sops_age_key} from stdin
and parses the YAML itself via a new load_target() - validate_config and
everything downstream is unchanged, since they already just consumed a
plain dict shaped this way regardless of where it came from.
deploy.yml also drops the "redeploy every target" (target: all) mode -
target is now a required input, no default - and renames its
find-targets step id from the generic "matrix" to "load-targets",
matching renovate.yml's own naming.…trix Target manifests drop the credentials: wrapper entirely. Tailscale config moves out of targets/*.yml altogether - it's one shared tailnet, not per-target - so deploy.yml/release.yml/renovate.yml now reference vars.TAILSCALE_OAUTH_CLIENT_ID/secrets.TAILSCALE_OAUTH_SECRET directly instead of resolving a per-target name for the same fixed values. ssh_private_key/sops_age_key move to the manifest root, renamed to ssh_private_key_secret/sops_age_key_secret - the _secret suffix makes it unambiguous these are GitHub Secret names, not values, now that they're no longer nested under credentials.secrets. This also sidesteps a real naming collision in deploy.py, which already uses the bare key "sops_age_key" internally for the actual decrypted value. New .github/actions/load-targets-matrix, a specialized sibling to load-yaml-matrix used only for targets/: same directory+name matrix contract, but validates hosts/app_refs/apps/ssh_private_key_secret/ sops_age_key_secret before ever producing a matrix item, so a broken target manifest fails immediately with a clear error instead of surfacing deep inside deploy.py/renovate.py after a checkout and dependency install on a different job. load-yaml-matrix itself is untouched and keeps validating nothing - it's still the right tool for vaults/, where the caller never touches manifest fields directly and encrypt-env already validates the same content one job later anyway. deploy.yml/release.yml/renovate.yml switch their targets/ loading to load-targets-matrix (dropping directory: targets, now the default) and simplify their with:/secrets: blocks to the flat field names.
Renamed app -> apps throughout (renovate.yml, renovate-shared.yml, deploy/renovate.py's stdin contract) so one run can target several apps at once - e.g. a future nightly cron renovating "traefik,rybbit" while leaving everything else alone - instead of one app per invocation. deploy/renovate.py splits the comma-separated list, renovates whichever requested apps are actually keys in the target's own apps mapping (still a clean no-op if none match), and loops hosts inside that. updated_hosts now reports app@host pairs instead of bare hosts, since more than one app can be renovated in the same run.
…ated Matches the JSON convention used everywhere else in this codebase (hosts, app_refs, apps mappings) rather than inventing a one-off comma-separated format just for this input. renovate-shared.yml passes it through with --argjson instead of --arg, and deploy/renovate.py reads config["apps"] directly as the already-parsed list.
…ectory It's had exactly one consumer since load-targets-matrix took over targets/ (release.yml's encrypt-vaults job) - naming it for what it actually loads and defaulting directory to vaults lets that call site drop the with: block entirely. No behavior change: still no schema validation, encrypt-env re-parses and validates each manifest itself. Test fixtures switched from generic hawkeye/mainframe examples to vault-shaped ones (asset/keys/env) to match what this action actually reads now that it's no longer shared with targets/.
…te actions Both were single-job workflow_call workflows - exactly the shape GitHub's own guidance says a composite action fits better than a reusable workflow (which earns its keep with multiple jobs or job-level config, neither of which these ever used). This also directly eliminates two problems the workflow_call boundary forced on us: - The second checkout (repository: job.workflow_repository, ref: job.workflow_sha) needed purely to get this repo's own deploy.py/ renovate.py onto the runner is gone - a composite action's own files come along automatically via $GITHUB_ACTION_PATH whenever it's referenced as owner/repo/.github/actions/X@ref. - target-manifest-ref is gone with it - it only existed to keep that second checkout's ref in sync with whatever ref the caller had already resolved its matrix from. With one checkout instead of two, there's nothing to keep in sync. deploy.py and its sibling modules (collisions.py, render.py, resolve.py, vault.py) move from deploy/ into .github/actions/deploy/, since a composite action's script has to live inside the action's own directory tree to be reachable via $GITHUB_ACTION_PATH; deploy/tests/ moves the same way. renovate.py and its tests move into .github/actions/renovate/ - it already had zero dependency on deploy.py's sibling modules, so it becomes its own action rather than living alongside deploy's. deploy.yml/release.yml/renovate.yml's matrix jobs now do their own actions/checkout, resolve secrets themselves, and call the action directly as one step among their own - no more uses:/secrets: mapping into a separate workflow file. No script logic changed, only where the files live and how they're invoked; all 89 tests pass unchanged in their new locations.
Matches its step id (already load-targets) and deploy.yml/release.yml's own job naming for the equivalent step (deploy-targets loads targets/ too, just paired with a different verb since it goes on to deploy them).
…ease.yml Matches its step id (already load-targets) and renovate.yml's own equivalent job, now that "deploy" as a verb belongs to the job that actually deploys, not the one that just loads the target matrix.
Matches load-targets: the job's own work is loading vaults/ into a matrix, not encrypting anything - that's the separate encrypt job that consumes it. Step id renamed from the generic "matrix" to load-vaults to match, same as load-targets-matrix's own step id.
…pip install Leftover from an old ansible-core install step, copy-pasted forward unchanged ever since - the actions written later (encrypt-env, load-targets-matrix, load-vaults-matrix) already use the correct form, `python3 -m pip install`, which targets the exact interpreter that later runs the script rather than whatever bare `pip` resolves to. That's already proven to work on this runner without needing to override PEP 668's externally-managed-environment protection at all.
The action now only reports what happened (updated/updated-hosts/ target-name as proper action outputs) instead of hard-requiring telegram-bot-token/telegram-chat-id and sending the message itself. Notifying is renovate.yml's own job now, as a separate step reading those outputs - matches deploy, which never had any notification logic of its own, and means a different caller can wire up a different channel (or none) without forking this action.
Same pattern just established for renovate: notification lives in the caller's own job, not baked into the deploy action. Unlike renovate, there's no "did anything change" gate needed - a deploy either fully lands or the job fails outright, so the Notify Telegram step runs unconditionally right after the deploy step, same as any other step that simply never executes if the one before it failed. release.yml's version also includes the release tag that was just deployed, since it already has that on hand from the release job.
…lter Single-target selection had exactly one consumer (deploy.yml's manual redeploy) and this repo has exactly one target with no second one planned - there was nothing left to disambiguate. deploy.yml now just redeploys everything, matching what release.yml's own deploy path already does; load-targets-matrix/load-vaults-matrix both drop the name input and its --name/selected filtering logic entirely, since neither has any remaining caller passing it. All 85 tests across deploy/renovate/load-targets-matrix/load-vaults- matrix pass unchanged in behavior for the "load everything" path - only the removed filtering path and its tests are gone.
ineedjetforce-pushed
the
feat/renovate-folder-prototype
branch
from
August 28, 2026 22:42
023b5ea to
4797067Comparedeploy.yml lost its target input in an earlier commit - it now always redeploys every target, exactly matching what release.yml's own load-targets+deploy job pair already did. That made the two genuinely identical operations, not just similar ones, so the duplication is worth removing: deploy.yml adds workflow_call (no inputs, same as its workflow_dispatch trigger) and release.yml calls it directly with secrets: inherit instead of running its own copy. secrets: inherit is fine here specifically because both workflows live in this same repository - deploy.yml already has native access to every one of this repo's own secrets when triggered directly, so inheriting them from release.yml doesn't broaden anything. No organization boundary is being crossed, unlike the renovate/dupmachine case that ruled out inherit there. Default (unpinned) checkouts inside deploy.yml's own jobs now resolve correctly even when called from release.yml's push-triggered run, because github.sha for that run already IS the commit release-please just tagged - no target-manifest-ref equivalent needed. Trade-off: release.yml's deploy no longer includes the release tag in its Telegram message, since deploy.yml's own generic message doesn't know it. Not worth an extra input and a conditional format(...) expression to preserve that one detail.
renovate.yml now also triggers on `schedule: 0 3 * * *`. A cron trigger can't supply workflow_dispatch inputs, so `apps` becomes optional (default "[]") both on the workflow input and the renovate action's own input, and an empty apps array now means "every app this target runs" instead of "no apps requested" - renovate.py already skipped apps not present on a given target, so this just changes what an empty selection means rather than adding new matching logic.
Mirrors load-targets-matrix's own validate_target: just type + non-empty checks on asset/keys/env, so a structurally broken vault manifest fails the matrix build immediately instead of reaching a checkout+dependency install on a different job (encrypt-env) before erroring out. encrypt-env's render-env.py still does the real field-level validation (name patterns, source references) - this only catches missing/empty top-level sections early, same division of labor as targets already had.
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Aug 28, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch has grown into a full reconsideration of how deploy/renovate are packaged.
deploy-shared.yml/renovate-shared.yml→ composite actionsBoth were single-job
workflow_callworkflows — exactly the shape GitHub's own guidance says a composite action fits better than a reusable workflow (which earns its keep with multiple jobs or job-level config, neither ever used here). Converting eliminates two problems theworkflow_callboundary forced on us:checkout(repository: job.workflow_repository, ref: job.workflow_sha) needed purely to get this repo's owndeploy.py/renovate.pyonto the runner is gone — a composite action's own files come along automatically via$GITHUB_ACTION_PATH.target-manifest-refis gone with it — it only existed to keep that second checkout's ref in sync with whatever ref the caller had already resolved its matrix from.deploy.py+ siblings (collisions.py,render.py,resolve.py,vault.py) moved fromdeploy/into.github/actions/deploy/(a composite action's script must live inside its own action directory to be reachable via$GITHUB_ACTION_PATH).renovate.pymoved into its own.github/actions/renovate/— it never depended ondeploy.py's sibling modules.deploy.yml/release.yml/renovate.yml's matrix jobs now checkout + resolve secrets themselves + call the action as one step, no moreuses:/secrets:mapping into a separate workflow file.dupmachine/flightdeckneeds updating to this contract too (on top of the earliertarget-manifestchange).Notifications moved out of the actions, into the callers
Neither
deploynorrenovatesends Telegram messages itself anymore.renovateexposesupdated/updated-hosts/target-nameas proper action outputs;renovate.ymlreads them in its own separate "Notify Telegram" step, silent when nothing changed.deploynever had a "did anything change" concept to begin with, sodeploy.yml/release.yml's own Notify Telegram step just runs unconditionally after a successful deploy step. A different caller can now wire up a different channel (or none) without forking either action.Job naming cleanup
Every "load a directory into a matrix" job across
deploy.yml/release.yml/renovate.ymlis now consistently namedload-targets/load-vaults, matching its own step id —deploy/encryptstay named for what they actually do.Also dropped
--user --break-system-packagesLeftover from an old
ansible-coreinstall step, copy-pasted forward unchanged for years. The actions written later (encrypt-env,load-targets-matrix,load-vaults-matrix) already used the correctpython3 -m pip installform, proven working on this runner without needing to override PEP 668 at all.deploy.ymlalso has notargetinput anymoreIt always redeploys every target under
targets/— a single-targetworkflow_dispatchinput never actually got used, and dropping it makesdeploy.ymlandrelease.yml's own post-release deploy step functionally identical (same "always all targets" semantics).release.ymlnow callsdeploy.ymlinstead of duplicating itOnce
deploy.ymllost itstargetinput, itsload-targets+deployjob pair became byte-for-byte the same thingrelease.ymlneeded after a release.release.yml'sdeployjob is now just:secrets: inherit's same-organization restriction isn't a concern here since both workflows live in this same repo — no org boundary is being crossed.deploy.ymlalso gained a bareworkflow_call: {}(no inputs) purely so this works. One accepted trade-off:release.yml's deploy no longer mentions the release tag in its Telegram message, sincedeploy.yml's own generic "{target}updated" text doesn't know it — not worth reintroducing an input just to preserve that detail.Earlier pieces already on this branch
appsinput is a JSON array so one run can renovate several apps at once.deploy/renovatecontracts take atarget-manifestpath instead of flattened fields (matchesencrypt-env's existing pattern for vault manifests).credentials:wrapper; Tailscale config moved out entirely (shared infra, not per-target);ssh_private_key_secret/sops_age_key_secretat the manifest root.load-targets-matrix(validates the schema above, no longer takes anamefilter — always loads every target) alongside the renamedload-vaults-matrix(unvalidated, unchanged behavior, also nonamefilter, just named for what it actually loads).Test plan
python3 -m unittest discover -s .github/actions/{deploy,renovate,load-targets-matrix,load-vaults-matrix}/tests— 85 tests pass total (55+11+12+7)heimdalltarget