refactor!: rework deploy/renovate as composite actions with validated targets

.github/actions/deploy/README.md

@@ -0,0 +1,36 @@
+# deploy
+
+Composite GitHub Action that runs a push-based deploy against a target manifest already present in the caller's own checkout - resolves and downloads every release ref, decrypts and renders each app's env and config, merges the release, and pushes the finished result to each host over SSH. See the main [README](../../../README.md#automated-deploy) for the full sequence.
+
+## Usage
+
+```yaml
+jobs:
+ deploy:
+ strategy:
+ matrix: ${{ fromJson(needs.load-targets.outputs.matrix) }}
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v7
+ - uses: rubykatzen/flightdeck/.github/actions/deploy@main
+ with:
+ target-manifest: ${{ matrix.manifest }} # required, path in this repository
+ ssh-private-key: ${{ secrets[matrix.ssh_private_key_secret] }}
+ sops-age-key: ${{ secrets[matrix.sops_age_key_secret] }}
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ # tailscale-oauth-client-id: ${{ vars.TAILSCALE_OAUTH_CLIENT_ID }} # optional, default: unset (skip joining a tailnet)
+ # tailscale-oauth-secret: ${{ secrets.TAILSCALE_OAUTH_SECRET }} # required only if tailscale-oauth-client-id is set
+ # tailscale-tags: tag:ci # default: tag:ci
+ - name: Notify Telegram
+ uses: rubykatzen/baseline/.github/actions/send-telegram-message@v0.16.1
+ with:
+ message: "Deploy: ${{ matrix.name }} updated"
+ telegram-bot-token: ${{ secrets.TELEGRAM_BOT_TOKEN }}
+ telegram-chat-id: ${{ secrets.TELEGRAM_CHAT_ID }}
+```
+
+Unlike a `workflow_call` reusable workflow, this action doesn't check out anything itself - it reads `target-manifest` from whatever the caller's own preceding `actions/checkout` step already put on disk, and its own code (`deploy.py` and friends) comes along automatically via `$GITHUB_ACTION_PATH` whenever it's referenced as `owner/repo/.github/actions/deploy@ref`. This is also why there's no `target-manifest-ref` input here: if the caller needs a specific ref (e.g. a just-published release tag), it just checks out that ref itself before this step runs, the same way every other job in this repository already does.
+
+`sops-age-key`/`ssh-private-key` are secret *values*, resolved by the caller from the target manifest's own `sops_age_key_secret`/`ssh_private_key_secret` fields (GitHub Secret *names* - see the main README's "Vaults And Targets" section) - this action never reads the manifest's credential fields itself, since it never has access to `secrets.*` by name.
+
+This action has no notification logic of its own, same as [`renovate`](../renovate) - notifying is the caller's job, as a plain following step. Unlike `renovate`, there's no "did anything change" output to gate it on: reaching that step at all already means the `deploy` step above it succeeded, so it just runs unconditionally.

.github/actions/deploy/action.yml

@@ -0,0 +1,65 @@
+name: Deploy
+description: Push-based deploy against a target manifest already present in the caller's own checkout.
+inputs:
+ target-manifest:
+ description: Path to the target manifest file, e.g. targets/heimdall.yml - already checked out by the caller.
+ required: true
+ ssh-private-key:
+ description: SSH private key used to connect to the hosts.
+ required: true
+ sops-age-key:
+ description: Private SOPS age key used to decrypt this target's vault-sourced env, on the runner.
+ required: true
+ github-token:
+ description: Token used to resolve/download release refs (app bundles, encrypted env sources).
+ required: true
+ tailscale-oauth-client-id:
+ description: Tailscale OAuth client ID used to join the tailnet. Leave unset to skip joining a tailnet (e.g. when the runner already has network access to the hosts).
+ required: false
+ default: ""
+ tailscale-oauth-secret:
+ description: Tailscale OAuth client secret used to join the tailnet. Required only when tailscale-oauth-client-id is set.
+ required: false
+ default: ""
+ tailscale-tags:
+ description: Comma-separated ACL tags for the ephemeral tailnet node.
+ required: false
+ default: tag:ci
+runs:
+ using: composite
+ steps:
+ - name: Install deploy dependencies
+ shell: bash
+ run: python3 -m pip install --disable-pip-version-check --requirement "$GITHUB_ACTION_PATH/requirements.txt"
+ - name: Install sops
+ shell: bash
+ env:
+ SOPS_VERSION: "3.13.1"
+ run: |
+ curl --fail --location --silent --show-error \
+ "https://github.com/getsops/sops/releases/download/v${SOPS_VERSION}/sops-v${SOPS_VERSION}.linux.amd64" \
+ --output /usr/local/bin/sops
+ chmod +x /usr/local/bin/sops
+ - uses: tailscale/github-action@v4
+ if: inputs.tailscale-oauth-client-id != ''
+ with:
+ oauth-client-id: ${{ inputs.tailscale-oauth-client-id }}
+ oauth-secret: ${{ inputs.tailscale-oauth-secret }}
+ tags: ${{ inputs.tailscale-tags }}
+ - name: Load SSH key
+ shell: bash
+ run: |
+ eval "$(ssh-agent -s)"
+ echo "SSH_AUTH_SOCK=$SSH_AUTH_SOCK" >> "$GITHUB_ENV"
+ echo "SSH_AGENT_PID=$SSH_AGENT_PID" >> "$GITHUB_ENV"
+ ssh-add - <<< "${{ inputs.ssh-private-key }}"
+ - name: Run deploy
+ shell: bash
+ env:
+ TARGET_MANIFEST: ${{ inputs.target-manifest }}
+ SOPS_AGE_KEY: ${{ inputs.sops-age-key }}
+ GH_TOKEN: ${{ inputs.github-token }}
+ run: |
+ jq -n --arg target_manifest "$TARGET_MANIFEST" --arg sops_age_key "$SOPS_AGE_KEY" \
+ '{target_manifest: $target_manifest, sops_age_key: $sops_age_key}' \
+ | python3 "$GITHUB_ACTION_PATH/deploy.py"

deploy/deploy.py → .github/actions/deploy/deploy.py

@@ -6,8 +6,13 @@
 nothing but Docker and Docker Compose: no sops, no age key, no gh, no
 flightdeck scripts of any kind.
 
-Reads a JSON config from stdin (see README's "deploy-shared.yml" section
-for the exact shape).
+Reads a JSON config from stdin: {"target_manifest": "targets/heimdall.yml",
+"sops_age_key": "..."}. `target_manifest` is a path to that target's own
+manifest, parsed here rather than flattened into separate fields by the
+caller - the same shape encrypt-env's render-env.py already uses for
+vault manifests. `sops_age_key` can't come from that file, since it's a
+secret value, not YAML-safe config - the calling `deploy` action passes
+it separately (see README's "deploy" section for the exact contract).
 """
 import json
 import shlex
@@ -192,6 +197,10 @@ def deploy_to_host(host, archive_path, apps, networks, config, release_name):
  connection.run("docker container prune -f && docker image prune -a -f")
 
 
+def load_target(path):
+ return yaml.safe_load(Path(path).read_text())
+
+
 def validate_config(config):
  if not config.get("hosts"):
  raise DeployError("Config must set hosts to a non-empty list")
@@ -204,7 +213,9 @@ def validate_config(config):
 
 
 def main():
- config = json.load(sys.stdin)
+ stdin_config = json.load(sys.stdin)
+ config = load_target(stdin_config["target_manifest"])
+ config["sops_age_key"] = stdin_config.get("sops_age_key")
  validate_config(config)
  with tempfile.TemporaryDirectory(prefix="flightdeck-deploy-") as raw_dir:
  work_dir = Path(raw_dir)

deploy/tests/test_deploy.py → .github/actions/deploy/tests/test_deploy.py

@@ -1,4 +1,5 @@
 import importlib.util
+import io
 import json
 import sys
 import tarfile
@@ -42,6 +43,22 @@ def make_zip(path, files):
  return path
 
 
+class LoadTargetTest(unittest.TestCase):
+ def test_parses_a_target_manifest(self):
+ with tempfile.TemporaryDirectory() as directory:
+ manifest_path = Path(directory) / "heimdall.yml"
+ manifest_path.write_text(
+ "hosts: [user@host]\napp_refs: [owner/repo@latest]\n"
+ "apps:\n traefik:\n env_refs: [owner/repo@latest:a.sops.env]\n"
+ )
+
+ target = deploy.load_target(manifest_path)
+
+ self.assertEqual(target["hosts"], ["user@host"])
+ self.assertEqual(target["app_refs"], ["owner/repo@latest"])
+ self.assertEqual(target["apps"]["traefik"]["env_refs"], ["owner/repo@latest:a.sops.env"])
+
+
 class ValidateConfigTest(unittest.TestCase):
  VALID = {
  "hosts": ["user@host"],
@@ -456,5 +473,57 @@ def test_no_previous_manifest_stops_nothing(self):
  self.assertNotIn("docker compose down", joined)
 
 
+class MainTest(unittest.TestCase):
+ def test_reads_target_manifest_and_merges_sops_age_key(self):
+ with tempfile.TemporaryDirectory() as directory:
+ manifest_path = Path(directory) / "heimdall.yml"
+ manifest_path.write_text(
+ "hosts: [deploy@host]\n"
+ "app_refs: [owner/repo@latest]\n"
+ "apps:\n traefik:\n env_refs: [owner/repo@latest:a.sops.env]\n"
+ )
+ stdin_config = {"target_manifest": str(manifest_path), "sops_age_key": "AGE-SECRET-KEY-1..."}
+
+ with (
+ patch.object(sys, "stdin", io.StringIO(json.dumps(stdin_config))),
+ patch.object(
+ deploy, "build_release", return_value=(Path(directory) / "release", ["owner/repo@v1.0.0"])
+ ) as fake_build_release,
+ patch.object(
+ deploy,
+ "resolve_app_envs",
+ return_value={"traefik": ["owner/repo@v1.0.0:a.sops.env"]},
+ ) as fake_resolve_app_envs,
+ patch.object(deploy, "write_release_manifest"),
+ patch.object(deploy, "archive_release", return_value=Path(directory) / "release.tar.gz"),
+ patch.object(deploy, "list_required_networks", return_value=[]),
+ patch.object(deploy, "deploy_to_host") as fake_deploy_to_host,
+ ):
+ deploy.main()
+
+ config_arg = fake_build_release.call_args[0][0]
+ self.assertEqual(config_arg["hosts"], ["deploy@host"])
+ self.assertEqual(config_arg["app_refs"], ["owner/repo@latest"])
+ self.assertEqual(config_arg["sops_age_key"], "AGE-SECRET-KEY-1...")
+
+ fake_resolve_app_envs.assert_called_once()
+ fake_deploy_to_host.assert_called_once()
+ host, archive_path, apps, networks, deploy_config = fake_deploy_to_host.call_args[0][:5]
+ expected = ("deploy@host", Path(directory) / "release.tar.gz", ["traefik"], [], config_arg)
+ self.assertEqual((host, archive_path, apps, networks, deploy_config), expected)
+
+ def test_raises_when_sops_age_key_missing(self):
+ with tempfile.TemporaryDirectory() as directory:
+ manifest_path = Path(directory) / "heimdall.yml"
+ manifest_path.write_text("hosts: [deploy@host]\napp_refs: [owner/repo@latest]\napps:\n traefik: {}\n")
+ stdin_config = {"target_manifest": str(manifest_path), "sops_age_key": ""}
+
+ with (
+ patch.object(sys, "stdin", io.StringIO(json.dumps(stdin_config))),
+ self.assertRaises(deploy.DeployError),
+ ):
+ deploy.main()
+
+
 if __name__ == "__main__":
  unittest.main()

.github/actions/load-targets-matrix/README.md

@@ -0,0 +1,18 @@
+# load-targets-matrix
+
+Composite GitHub Action that reads every target manifest in a directory into a GitHub Actions matrix, validating each one's required shape. Sibling to [`load-vaults-matrix`](../load-vaults-matrix), same shape, for `vaults/*.yml` instead.
+
+## Usage
+
+```yaml
+- uses: rubykatzen/flightdeck/.github/actions/load-targets-matrix@main
+ id: load-targets
+ with:
+ # directory: targets # optional, default shown
+```
+
+The action exposes `matrix`, containing `{ "include": [...] }`, and `count`. Each matrix item merges the manifest's own top-level YAML fields with `name` (the file's basename) and `manifest` (its path).
+
+Every manifest must set `hosts` (non-empty list), `app_refs` (non-empty list), `apps` (non-empty object), `ssh_private_key_secret` (non-empty string - the name of the GitHub Secret holding this target's SSH private key), and `sops_age_key_secret` (non-empty string - the name of the GitHub Secret holding this target's private SOPS age key). This fails the matrix build immediately, with an error naming the specific broken target, rather than letting a malformed manifest reach the [`deploy`](../deploy)/[`renovate`](../renovate) actions much later, after a checkout and dependency install on a different job entirely.
+
+Files may use either the `.yml` or `.yaml` extension. Filenames must match `^[a-z0-9][a-z0-9-]*$` and be unique per directory; duplicate top-level YAML keys within a manifest are rejected.

.github/actions/load-targets-matrix/action.yml

@@ -0,0 +1,27 @@
+name: Load targets matrix
+description: Read every target manifest in a directory into a GitHub Actions matrix, validating each one's required shape.
+inputs:
+ directory:
+ description: Directory containing target manifests.
+ required: false
+ default: targets
+outputs:
+ matrix:
+ description: JSON strategy matrix, one item per target manifest, merging its parsed YAML fields with name and manifest.
+ value: ${{ steps.load.outputs.matrix }}
+ count:
+ description: Number of targets in the matrix.
+ value: ${{ steps.load.outputs.count }}
+runs:
+ using: composite
+ steps:
+ - name: Install dependencies
+ shell: bash
+ run: python3 -m pip install --disable-pip-version-check --requirement "$GITHUB_ACTION_PATH/requirements.txt"
+ - name: Load matrix
+ id: load
+ shell: bash
+ env:
+ DIRECTORY: ${{ inputs.directory }}
+ run: |
+ python3 "$GITHUB_ACTION_PATH/scripts/load-targets-matrix.py" --directory "$DIRECTORY"

.github/actions/load-targets-matrix/scripts/load-targets-matrix.py

@@ -0,0 +1,105 @@
+#!/usr/bin/env python3
+import argparse
+import json
+import os
+import re
+import sys
+from pathlib import Path
+
+import yaml
+
+NAME_RE = re.compile(r"^[a-z0-9][a-z0-9-]*$")
+
+
+class ManifestError(Exception):
+ pass
+
+
+class UniqueKeyLoader(yaml.SafeLoader):
+ pass
+
+
+def construct_mapping(loader, node, deep=False):
+ mapping = {}
+ for key_node, value_node in node.value:
+ key = loader.construct_object(key_node, deep=deep)
+ if key in mapping:
+ raise ManifestError(f"duplicate YAML key: {key}")
+ mapping[key] = loader.construct_object(value_node, deep=deep)
+ return mapping
+
+
+UniqueKeyLoader.add_constructor(
+ yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
+ construct_mapping,
+)
+
+
+def load_manifest(path):
+ try:
+ value = yaml.load(path.read_text(), Loader=UniqueKeyLoader)
+ except yaml.YAMLError as exc:
+ raise ManifestError(f"{path} is not valid YAML: {exc}") from exc
+ if not isinstance(value, dict):
+ raise ManifestError(f"{path} must contain a YAML mapping")
+ return value
+
+
+def validate_target(name, manifest):
+ if not isinstance(manifest.get("hosts"), list) or not manifest["hosts"]:
+ raise ManifestError(f"target {name!r} must set hosts to a non-empty list")
+ if not isinstance(manifest.get("app_refs"), list) or not manifest["app_refs"]:
+ raise ManifestError(f"target {name!r} must set app_refs to a non-empty list")
+ if not isinstance(manifest.get("apps"), dict) or not manifest["apps"]:
+ raise ManifestError(f"target {name!r} must set apps to a non-empty object")
+ if not isinstance(manifest.get("ssh_private_key_secret"), str) or not manifest["ssh_private_key_secret"]:
+ raise ManifestError(f"target {name!r} must set ssh_private_key_secret to a non-empty string")
+ if not isinstance(manifest.get("sops_age_key_secret"), str) or not manifest["sops_age_key_secret"]:
+ raise ManifestError(f"target {name!r} must set sops_age_key_secret to a non-empty string")
+
+
+def build_matrix(directory):
+ paths = sorted(directory.glob("*.yml")) + sorted(directory.glob("*.yaml"))
+ if not paths:
+ raise ManifestError(f"no manifests found in {directory}")
+ include = []
+ seen_names = set()
+ for path in paths:
+ name = path.stem
+ if not NAME_RE.fullmatch(name):
+ raise ManifestError(f"invalid manifest filename: {path.name}")
+ if name in seen_names:
+ raise ManifestError(f"duplicate manifest name: {name}")
+ seen_names.add(name)
+ manifest = load_manifest(path)
+ validate_target(name, manifest)
+ item = {"name": name, "manifest": str(path)}
+ item.update(manifest)
+ include.append(item)
+ return {"include": include}
+
+
+def write_github_output(name, value):
+ output_path = os.environ.get("GITHUB_OUTPUT")
+ if output_path:
+ with open(output_path, "a", encoding="utf-8") as output:
+ output.write(f"{name}={value}\n")
+
+
+def main(argv=None):
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--directory", required=True, type=Path)
+ args = parser.parse_args(argv)
+ try:
+ matrix = build_matrix(args.directory)
+ encoded = json.dumps(matrix, separators=(",", ":"))
+ write_github_output("matrix", encoded)
+ write_github_output("count", len(matrix["include"]))
+ except ManifestError as exc:
+ print(f"error: {exc}", file=sys.stderr)
+ return 1
+ return 0
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())