Uh oh!
There was an error while loading. Please reload this page.
feat: deploy rybbit for rubykatzen.com through flightdeck itself - #102
Conversation
First real (non-CI-test) self-consumption of deploy-shared.yml: flightdeck deploys its own rybbit instance at rybbit.rubykatzen.com, via an isolated rubykatzen-com user on the existing hawkeye host (Tailscale IP 100.75.50.2), separate from dupmachine's own services on that box (own traefik on 8080/8443 to avoid the port conflict, own age keypair, own env manifest). - projects/rubykatzen/rubykatzen-com.yml: publish-sops-env manifest (APPS=traefik,rybbit, Cloudflare DNS challenge cert resolver). - keys/rubykatzen-com.pub: age recipient for this deployment's encrypted env. Private key generated locally, not committed -- goes on the server at flightdeck_sops_age_key_file, out of band. - publish-rubykatzen-com-env.yml: renders + encrypts the manifest, publishes it to the rubykatzen-com-config release tag. - deploy-rubykatzen-com.yml: calls deploy-shared.yml@v0.3.0 (pinned, same as a real external consumer would) against 100.75.50.2. Blocked on, not part of this diff: DEPLOY_SSH_PRIVATE_KEY secret, TAILSCALE_OAUTH_CLIENT_ID/SECRET, RUBYKATZEN_COM_CLOUDFLARE_TOKEN secret, and server-side prep (create rubykatzen-com user, install the age private key, cloudflared tunnel). workflow_dispatch-only until those land. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…wkeye.yml The manifest identity is the server + isolated user + age key it decrypts on (hawkeye), not the domain/service currently running there (rubykatzen.com/rybbit) or a repo-grouping level that doesn't apply to self-hosting flightdeck's own deploys. Matches the existing convention elsewhere of naming manifests/assets after the server (mainframe, hawkeye), and drops the redundant projects/rubykatzen/ nesting -- one flat targets/ directory, one file per deploy target. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same reasoning as the targets/ rename: the age key, release tag, and asset identify the deploy target (server + isolated user + key), not the service currently running there. Renamed rubykatzen-com.pub -> hawkeye.pub, rubykatzen-com-config -> hawkeye-config, rubykatzen-com.sops.env -> hawkeye.sops.env. The RUBYKATZEN_COM_* GitHub variable/secret names are untouched -- those describe the service's own config content, not the target identity, and the rubykatzen-com OS username/home path on the server is a separate, correctly-still-rubykatzen-com concept. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
We were the odd one out: deploy-rubykatzen-com.yml already calls deploy-shared.yml as a real interface, but publish-rubykatzen-com-env.yml hand-assembled its own "ensure release exists" step around the local publish-sops-env composite action instead of going through an equivalent reusable workflow -- because that reusable workflow didn't exist yet, only the composite action did. Add publish-sops-env-shared.yml (checkout, resolve release tag -- input or read from the manifest's own release_tag, ensure the release exists, call publish-sops-env) and rewrite our own publish-rubykatzen-com-env.yml to just call it, matching the same consumer interface pattern deploy-rubykatzen-com.yml already uses. Note: this one needs `secrets: inherit` on the caller, unlike deploy-shared.yml's explicit named secrets -- the manifest can reference an arbitrary, per-target set of secret/variable names, so there's no fixed schema to declare individually. Pinned to @main for now since this file doesn't exist in any released tag yet; re-pin to a real version once the next release ships, same as deploy-rubykatzen-com.yml does for deploy-shared.yml@v0.3.0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…an interface The old interface leaked Ansible directly to callers -- extra-vars was a hand-written JSON blob using flightdeck_* variable names lifted straight from ansible/deploy.yml. Callers shouldn't need to know this runs on Ansible at all. Replace with named inputs in plain deploy vocabulary: app-ref, env-ref, extra-refs (comma-separated), path, keep-releases, sops-age-key-file. The workflow builds the ansible -e JSON internally via jq. No behavior change for existing values, just the surface. Added defaults for the ones that have a sensible one: - path: ~/flightdeck (matches every example/consumer so far) - keep-releases: 5 - sops-age-key-file: ~/.config/sops/age/keys.txt for the connecting `user`, computed rather than hardcoded since it depends on which user connects -- can't be a static workflow_call default. We have no external consumers of this interface yet to break, so doing this now rather than carrying the leaky version forward. Also updates our own deploy-rubykatzen-com.yml to the new interface (drops explicit path/keep-releases/sops-age-key-file entirely since the new defaults already match what we were passing), and README's deploy-shared.yml documentation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same reasoning as the earlier targets/keys/release-tag rename: these workflows deploy a whole target's APPS list, not one specific service. If hawkeye ever hosts a second app alongside rybbit, it's the same workflow, same target -- naming it after rubykatzen.com (the service currently there) would become misleading. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
One deploy entry point for the whole repo instead of one workflow file per target. Each target is its own job (currently just hawkeye); a workflow_dispatch `target` input picks which job(s) run -- "all" (default) runs every target, a specific name runs just that job. This is also the pattern we want client repos to copy: a system that knows exactly which target it just pushed config for can `gh workflow run deploy.yml -f target=<name>` instead of redeploying the whole fleet. No scheduling added yet -- that depends on the idempotent reconciliation work in #100/#101 landing first, otherwise a cron trigger would just force-restart everything on every tick. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…fter it - Rename release-please.yml -> release.yml (name: Release Please -> Release), matching the file. - deploy.yml now also fires on workflow_run of Release (in addition to manual workflow_dispatch). Auto-redeploys hawkeye whenever Release completes successfully -- including no-op runs where release-please found nothing to release, since workflow_run doesn't expose that workflow's own job outputs. Accepted as an MVP tradeoff: harmless once deploy.sh is idempotent (#100), just an unnecessary restart until then. - Fixed a real bug this surfaced: deploy-shared.yml's app-ref input was reused as both the checkout ref (needs a real git ref: branch, tag, SHA) and the flightdeck_app_ref bundle to deploy (can be 'latest', a GitHub Releases API concept, not a git ref). Passing app-ref: latest previously would have broken the checkout step. Split "Resolve checkout ref" (always derived from workflow_ref, used only for actions/checkout) from app-ref resolution inside "Build extra-vars" (input override, defaulting to the checkout ref if not given). deploy.yml's hawkeye job now passes app-ref: latest so it always deploys the newest release regardless of which ref runs the playbook mechanism itself. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
No external consumers yet, so no reason to carry the extra branch: "if app-ref is empty, reuse whatever ref runs the playbook mechanism" was solving a DRY concern (avoid writing the version twice) at the cost of conflating two genuinely different things -- which ref checks out ansible/deploy.yml vs which release gets deployed. Now that we need them to diverge (app-ref: latest while pinned to a stable mechanism version), the implicit default is just a trap. Require app-ref explicitly instead; deploy.yml already passes it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
release.yml's upload job was hand-assembled from checkout + build-bundle + gh release upload steps directly, the same pattern we already moved away from for publish-sops-env. Extract it into a generic reusable workflow: checkout a ref, build a zip via the build-bundle composite action, upload it to an existing release. release.yml's own upload job now just calls it via a local path (./.github/workflows/upload-bundle-shared.yml) -- same-repo self-reference, no ref-pinning concern since it always runs from whatever commit release.yml itself runs from. Also generic enough for external use: a consumer repo packaging its own apps/ directory as an extra bundle for flightdeck_extra_refs can call it the same way (checkout defaults to the caller's own repo, same as any reusable workflow without an explicit repository: override). Docs: added README sections for both upload-bundle-shared.yml and publish-sops-env-shared.yml (the latter existed but was never documented), updated the GitHub Actions intro sentence to list all three reusable workflows, and fixed a stale release-please.yml filename left over in the project tree from the earlier rename. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…f upload-apps-shared.yml upload-bundle-shared.yml took bundle-name/paths as caller inputs, but in practice there's exactly one caller (this repo's own release.yml) with fixed, well-known content -- flightdeck.zip is core scripts + apps/ + README, not something that varies per call. Generic parameterization here just pushed config a consumer shouldn't need to know onto the call site. Baked it in; the workflow now only takes ref/release-tag. Split the actual generic use case -- a consumer repo bundling its own apps/ as an extra bundle for flightdeck_extra_refs -- into its own purpose-built upload-apps-shared.yml (fixed to apps.zip / apps/ only). Added a comment at release.yml's upload job pointing consumers at it, so reading our own release flow makes the intended split obvious rather than inviting a copy-paste of flightdeck-specific config. Also: switched every local composite-action reference inside these shared workflows (publish-sops-env-shared.yml, upload-bundle-shared.yml, upload-apps-shared.yml) from ./.github/actions/x to $/.github/actions/x -- GitHub's new (July 2026) self-repository syntax. ./ inside a reusable workflow resolves against the CALLER's repo, which is wrong here and would have broken the moment an external repo actually called publish-sops-env-shared.yml or upload-apps-shared.yml (our own usage so far has been same-repo, so this was latent). $/ always resolves against the workflow's own defining repo at the exact ref running, regardless of caller -- matches the pattern rubykatzen/baseline already uses in lint-shared.yml. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… a separate trigger publish-hawkeye-env.yml had its own independent trigger (push when targets/hawkeye.yml or keys/hawkeye.pub changed), decoupled from whether a release actually happened. Config changes for a target are conceptually part of the release cycle here, same as the bundle -- fold it into release.yml as another job gated on release_created, same shape as upload. Delete the standalone workflow; a per-target manual/path-based publish trigger isn't needed once every target's env publish just rides along with the release. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missed staging this alongside the publish-hawkeye-env.yml removal in 2b7b16a -- this is the actual job that replaces it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Predates the explicit per-target job pattern (targets/hawkeye.yml, named jobs in release.yml/deploy.yml) this repo now actually uses -- it was built for a dynamic glob-based matrix discovery approach we never ended up using here, and nothing in this repo calls it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missed staging this alongside the action removal in d1472bb. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
build-bundle was the odd one out -- publish-sops-env already does render+encrypt+upload as one action, but build-bundle only built the zip and left the gh release upload as a separate step in each caller. Fold upload into build-bundle itself (new release-tag/token inputs), so upload-bundle-shared.yml and upload-apps-shared.yml both drop to checkout + one action call. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
deploy.yml previously fired on every completed "Release" workflow_run, including runs where release-please created no release - which meant every ordinary merge to main would redeploy hawkeye and bounce traefik/rybbit. Move the automatic deploy into release.yml, gated on release_created like upload/targets/encrypt, and leave deploy.yml with only the manual workflow_dispatch redeploy path.
"targets" already means deploy destinations (servers). Reusing that name for the generic config-matrix reader used by both encrypt and deploy modes was confusing. Rename the action to load-matrix, its name-selector input/flag from target to name, and the matrix item's target field to name. Also rename release.yml's encrypt-mode job from targets to encrypt-configs so it doesn't collide semantically with the new deploy-targets job.
Rename load-matrix to load-yaml-matrix and strip out its encrypt/deploy schema knowledge. It now just reads every YAML file in a directory, merges each file's top-level fields with name/manifest, and builds a matrix - no mode, no field validation, no defaults. encrypt-env already re-parses and validates its own manifest from the file path, so the encrypt/ path loses nothing. The targets/ path had no other validator, so move what it relied on into the callers: release.yml and deploy.yml now apply the path/keep-releases/sops-age- key-file defaults inline and read credentials.secrets/credentials.variables directly off the raw matrix item instead of the flattened fields load-matrix used to produce. This drops schema validation entirely (SSH destination format, ref format, duplicate hosts, unknown-key typos) - a malformed targets/*.yml now only surfaces once deploy-shared.yml or ansible/deploy.yml actually runs against production hosts, not before.
targets/ is a plural noun naming what it holds (deploy targets). encrypt/ was a singular verb, inconsistent with that pattern and with its own contents (env asset configs, one per encrypt-env manifest). Rename to envs/ to match.
envs/ read unnaturally next to targets/ and keys/. sops/ matches the asset naming convention already used throughout (*.sops.env) and pairs directly with keys/, which already holds the age recipients used to encrypt into that directory's assets.
One repo prepares vaults from a pool of secrets/vars; different consumers (targets) unpack the vault meant for them. Also just a better English word for "a named container of credentials" than sops/, which only referenced the encryption mechanism.
encrypt-configs -> encrypt-vaults, matching the deploy-targets job it sits next to (both <verb>-<directory> now). deploy.yml's targets job becomes deploy-targets to match release.yml's job of the same purpose, and its step id follows the matrix convention used everywhere else.
Core apps and third-party extra apps were structurally identical (an apps/ directory) but packaged asymmetrically: core was baked into flightdeck.zip, extras went through a separate build-bundle+merge path. Publish flightdeck's own apps/ as flightdeck-apps.zip via the same build-bundle action, and merge it through the exact same ansible path as any other app bundle - removing the special-cased "extract core zip as the apps base" step entirely. extra_refs is retired in favor of app_refs, now required and non-empty throughout (targets/*.yml, deploy-shared.yml's app-refs input, and ansible's flightdeck_app_refs): a target must explicitly list every app bundle it wants, including flightdeck's own - there's no more implicit apps/ riding along with the machinery bundle.
… defaults build-bundle's paths and bundle-name now default to Flightdeck's own machinery bundle instead of being required on every call, so the canonical file list lives in one place (the action) instead of inline in release.yml. build-apps-bundle is a new, thin defaults wrapper around build-bundle (paths: apps, bundle-name: flightdeck-apps.zip) used both for flightdeck's own apps/ catalog and by any consumer repo publishing its own app bundle the same way.
There was a problem hiding this comment.
Pull request overview
This PR evolves Flightdeck into a self-hosting setup that can publish its own machinery/apps bundles, encrypt env assets from vaults/ manifests, and deploy targets/—including a concrete hawkeye target for running traefik + rybbit for rubykatzen.com.
Changes:
- Introduces a
vaults/+targets/configuration model (withhawkeyemanifests) and updates docs to match. - Adds new composite actions (
encrypt-env,load-yaml-matrix,build-apps-bundle) and updates release automation to publish bothflightdeck.zipandflightdeck-apps.zip. - Updates the Ansible deploy flow to merge required app bundles (
flightdeck_app_refs) into the release and expands~paths per SSH user.
Reviewed changes
Copilot reviewed 26 out of 27 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
vaults/hawkeye.yml | Adds a concrete vault manifest mapping GitHub Secrets/Vars into an encrypted env asset and defining the app list. |
targets/hawkeye.yml | Adds a concrete deploy target selecting bundles, env asset ref, SSH destination, and credential names. |
keys/hawkeye.pub | Adds the age recipient public key used by encrypt-env for hawkeye. |
ansible/deploy.yml | Requires flightdeck_app_refs and merges app bundles into releases/<ts>/apps; expands ~ paths per SSH user. |
README.md | Documents the new deploy interface (flightdeck_app_refs) and the vaults/targets model + new actions/workflows. |
AGENTS.md | Updates internal architecture docs to reflect the new bundle/env/target workflow model. |
.github/workflows/release.yml | Replaces the Release Please workflow and adds jobs to upload both bundles, encrypt vaults, and deploy targets on release_created. |
.github/workflows/release-please.yml | Removes the old Release Please workflow file. |
.github/workflows/deploy.yml | Adds a manual workflow_dispatch redeploy workflow that selects one target or all targets. |
.github/workflows/deploy-shared.yml | Redesigns the reusable workflow interface to accept hosts, app-ref, env-ref, and non-empty app-refs. |
.github/actions/build-bundle/action.yml | Extends the bundle builder to upload to an existing release and provides defaults for Flightdeck’s machinery bundle. |
.github/actions/build-apps-bundle/action.yml | Adds a thin wrapper that publishes an apps/-only bundle (flightdeck-apps.zip). |
.github/actions/encrypt-env/action.yml | Adds composite action to render env from vaults/*, encrypt via SOPS age, and upload to an existing release. |
.github/actions/encrypt-env/README.md | Documents usage and the new manifest format (asset, keys, apps, env). |
.github/actions/encrypt-env/requirements.txt | Pins PyYAML for the action implementation. |
.github/actions/encrypt-env/scripts/render-env.py | Enforces the new manifest schema, renders apps as APPS=..., and writes outputs for downstream steps. |
.github/actions/encrypt-env/tests/test_render_env.py | Updates/extends unit tests for the new manifest schema and APPS rendering rules. |
.github/actions/load-yaml-matrix/action.yml | Adds composite action to load a directory of YAML files into a matrix + count outputs. |
.github/actions/load-yaml-matrix/README.md | Documents load-yaml-matrix behavior and filename/key validation rules. |
.github/actions/load-yaml-matrix/requirements.txt | Pins PyYAML for the action implementation. |
.github/actions/load-yaml-matrix/scripts/load-yaml-matrix.py | Implements matrix building with duplicate-key detection and filename validation. |
.github/actions/load-yaml-matrix/tests/test_load_yaml_matrix.py | Adds unit tests covering discovery, filtering, and validation failures. |
.github/actions/publish-sops-env/action.yml | Removes the superseded publish-sops-env action. |
.github/actions/publish-sops-env/README.md | Removes the superseded publish-sops-env docs. |
.github/actions/discover-manifest-matrix/action.yml | Removes the unused manifest discovery action. |
.github/actions/discover-manifest-matrix/README.md | Removes the unused manifest discovery docs. |
.github/actionlint.yaml | Adds actionlint ignores for newer job context properties used in deploy-shared.yml. |
Suppressed comments (6)
.github/workflows/release.yml:40
- 🥽 [rubykatzen/flightdeck]:
uses: $/.github/...is not a valid local action reference in GitHub Actions and will fail to load this action. Use a relative path starting with./instead.
🛠️ 🧪 📦
- uses: $/.github/actions/build-apps-bundle
.github/workflows/release.yml:69
- 🥽 [rubykatzen/flightdeck]:
uses: $/.github/...is not a valid local action reference in GitHub Actions and will fail to load this action. Use a relative path starting with./instead.
🛠️ 🧪 📦
- uses: $/.github/actions/encrypt-env
.github/workflows/release.yml:55
- 🥽 [rubykatzen/flightdeck]:
uses: $/.github/...is not a valid local action reference in GitHub Actions and will fail to load this action. Use a relative path starting with./instead.
🛠️ 🧪 📦
- uses: $/.github/actions/load-yaml-matrix
.github/workflows/release.yml:88
- 🥽 [rubykatzen/flightdeck]:
uses: $/.github/...is not a valid local action reference in GitHub Actions and will fail to load this action. Use a relative path starting with./instead.
🛠️ 🧪 📦
- uses: $/.github/actions/load-yaml-matrix
.github/workflows/release.yml:97
- 🥽 [rubykatzen/flightdeck]:
uses: $/.github/...is not a valid local reusable-workflow reference in GitHub Actions and will fail to call the workflow. Use a relative path starting with./instead.
🛠️ 🧪 📦
uses: $/.github/workflows/deploy-shared.yml
.github/workflows/deploy.yml:27
- 🥽 [rubykatzen/flightdeck]:
uses: $/.github/...is not a valid local reusable-workflow reference in GitHub Actions and will fail to call the workflow. Use a relative path starting with./instead.
🛠️ 🧪 📦
uses: $/.github/workflows/deploy-shared.yml
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| - uses: actions/checkout@v7 | ||
| with: | ||
| ref: ${{ needs.release.outputs.tag_name }} | ||
| - uses: $/.github/actions/build-bundle |
| count: ${{ steps.matrix.outputs.count }} | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| - uses: $/.github/actions/load-yaml-matrix |
Summary
Flightdeck now uses its own release and deployment tooling to manage Rybbit for
rubykatzen.comon the existinghawkeyeserver.The final configuration model separates two independent concerns:
vaults/describes encrypted env assets.targets/describes deployments and explicitly selects a machinery bundle, app bundles, an env asset, and SSH destinations.The matching
hawkeye.ymlfilenames are a convention only. The actual relationship is the explicitenv_refin the deployment target, so either collection can live in a repository without the other.Flightdeck's own
apps/catalog is published as its own release asset (flightdeck-apps.zip), merged into a deploy the same way as any other app bundle. There's no special-cased "core" bundle that apps ride along with implicitly — a target must list every app bundle it wants, flightdeck's own included.Configuration
vaults/hawkeye.ymldefines:hawkeye.sops.envrelease asset;hawkeyeage recipient;traefik,rybbit), rendered asAPPS=traefik,rybbit;targets/hawkeye.ymldefines:rubykatzen/flightdeck@latest;app_refs, required, non-empty):rubykatzen/flightdeck@latest;rubykatzen/flightdeck@latest:hawkeye.sops.env;rubykatzen-com@100.75.50.2;Credential values are never stored in target files.
Automation
build-bundle, which builds and uploads a bundle to an existing release;paths/bundle-namedefault to Flightdeck's own machinery bundle (flightdeck.zip) but are fully overridable.build-apps-bundle, a thin defaults wrapper aroundbuild-bundle(paths: apps,bundle-name: flightdeck-apps.zip) used both for flightdeck's ownapps/catalog and any consumer repo's own app bundle.publish-sops-envwithencrypt-env, which renders a flat config from avaults/manifest, encrypts it with SOPS age recipients, and uploads it to an existing release.load-yaml-matrix, a schema-agnostic action that reads every YAML file in a caller-given directory into a GitHub Actions matrix (no directory default, no validation — callers own their manifest shape).release.yml. A created release uploadsflightdeck.zipandflightdeck-apps.zip, processes every config invaults/, and deploys every config intargets/— all gated onrelease_created, so an ordinary merge tomainthat doesn't cut a release does nothing.deploy.ymlfor manual redeploys only (workflow_dispatch, select a target or "all"). It is deliberately not triggered automatically — an earlier version fired on every completed Release workflow run regardless of whether a release was actually created, which would have redeployedhawkeye(and bouncedtraefik/rybbit) on every ordinary merge tomain.Deployment Interface
deploy-shared.ymlexposes deployment vocabulary rather than Ansible implementation details:user@hostSSH destinations;owner/repo@tag[:asset]refs;app-refsis a required, non-empty JSON array;path, retention, and the SOPS age key path have defaults and remain overridable;user@hostvalues are converted to an internal Ansible JSON inventory;~paths are expanded per SSH user on each target host.The default deployment path is
~/flightdeck. The default age key path is~/.config/sops/age/keys.txt, matching the normal Linux SOPS location for the connecting user while still working under Ansible privilege escalation.External Prerequisites
DEPLOY_SSH_PRIVATE_KEY.TAILSCALE_OAUTH_CLIENT_IDand SecretTAILSCALE_OAUTH_SECRETwhen Tailscale is used.vaults/hawkeye.yml.rubykatzen-comuser, corresponding SSH access and age private key, Docker, GitHub CLI, SOPS, and the external Cloudflare Tunnel configuration.Verification
load-yaml-matrixunit testsencrypt-envunit testsvaults/hawkeye.ymlandtargets/hawkeye.ymlansible-inventoryansible-playbook --syntax-check ansible/deploy.ymlpre-commit run --all-files