adr: minimum external security smoke test before a deployment is declared healthy

[tucktuck101]

Context and problem statement

Ruling 12 of #5 lists eight things external verification should establish where practical, and #5's success criteria require that an external port scan reaches only the intentionally public services. The PRD's open question asks something narrower: what the minimum external security smoke test is before a deployment can be declared healthy. #35 builds the suite; this decision sets the floor that suite must clear before anyone says the relay is deployed.

The floor matters because the bundle's own validation is an inside-the-host check. deploy/compose/README.md tells the operator to run curl -fsS "http://127.0.0.1:$(grep -E '^BUZZ_HTTP_PORT=' .env | cut -d= -f2-)/_liveness" — loopback, plaintext, on the machine being tested. That proves the relay process is up and proves nothing about what the internet can reach, which is exactly the gap Ruling 12 exists to close.

What "public" should mean is readable from the bundle. Under compose.caddy.yml the relay's own port is removed with ports: !reset [] and Caddy publishes 80 and 443; Caddyfile declares a single site for {$BUZZ_DOMAIN} whose only route is reverse_proxy relay:3000, so TLS terminates at Caddy and nothing else should answer. The relay's health port 8080 and metrics port 9102 are configured but never published, and compose.dev.yml would publish 5432, 6379, 9000, 9001, 8082 and 9090 if it were ever enabled on the VPS.

This is kept separate from the companion ADR on CI/CD deployment gates. That one decides what blocks the pipeline and is implemented by #37, which is blocked on the access question in #13. This one defines the health bar itself, is implemented by #35, and can be decided and used now — including by hand against the destroyable VM before any pipeline exists.

Decision drivers

• Ruling 12: the system must be checked from the perspective of an untrusted internet client, not only an authorised one
• # prd-03 — reproducible and hardened Buzz deployment: rebuild the cohort server from a bare Ubuntu host #5's success criteria name the specific negative result: PostgreSQL, Redis and object-storage administration unreachable from the internet
• # prd-03 — reproducible and hardened Buzz deployment: rebuild the cohort server from a bare Ubuntu host #5 also names TLS explicitly: a plaintext connection attempt to the public endpoint must fail
• A minimum has to be small enough to actually run on every deployment, or it will not be run
• task: add an external verification suite for the deployed host #35's definition of done requires the suite to be proven to fail against a deliberately misconfigured host, so the floor must be expressible as checks that can be made to fail on purpose
• Ruling 12 also asks that Buzz be running the expected deployment version, which matters because .env.example ships BUZZ_IMAGE=ghcr.io/block/buzz:main, a floating tag
• Port scanning against a provider's network may be constrained by that provider's acceptable-use policy

Considered options

• Floor A — reachability and TLS only: the public endpoint answers over TLS, a plaintext attempt fails, and the relay reports healthy through that endpoint
• Floor B — Floor A plus the negative assertions: an off-host scan showing that 5432, 6379, 9000, 9001, 8082, 9090 and the relay's 8080/9102 are not reachable, and that no other unexpected port answers
• Floor C — Floor B plus the deployed version matching the pinned BUZZ_IMAGE, and remote privileged access matching the policy decided in adr: administrative access model after bootstrap #26
• Floor D — the whole of Ruling 12's list, including the two-person connection check from task: verify two people on two machines can talk on the deployed relay #23, before the word "healthy" is used at all
• Expected rejection: Floor A on its own. "The relay answers over HTTPS" passes cleanly on a host that also has PostgreSQL open to the internet, so it certifies the one thing that was never in doubt while missing the failure # prd-03 — reproducible and hardened Buzz deployment: rebuild the cohort server from a bare Ubuntu host #5's success criteria are written around.
• Expected rejection: the bundle's existing loopback check as the health bar — curl -fsS "http://127.0.0.1:.../_liveness" from deploy/compose/README.md. It is a useful functional check and a worthless security one: it runs on the host, over plaintext loopback, and would pass identically whether the firewall were correct or absent.
• Expected rejection: Floor D. Setting the minimum equal to task: add an external verification suite for the deployed host #35's full scope means there is no minimum — nothing can be declared healthy until the last check is written, and task: deploy the relay to the VPS with real DNS and TLS #22 has no bar to be held to in the meantime. A floor that is never reached does not gate anything.

Decision outcome

Left blank deliberately. launchpad/AGENTS.md rule 1 for agents: draft everything, approve nothing.

Consequences

Good: a named floor turns "deployed" into a claim with evidence behind it, and gives #35 a subset to build first and #22 something to satisfy before the relay is handed to the cohort. It also lets the rest of Ruling 12's list be recorded as deferred rather than quietly dropped, which #35's definition of done already requires.

Bad: any floor is also a ceiling in practice — checks outside it tend not to get written. Setting it too low certifies a host that is only superficially safe; setting it high enough to be meaningful means the check must run from off-host, so declaring health depends on a second machine being available. The negative assertions are the expensive part: proving a port is closed requires scanning rather than reading configuration, which takes longer and can be affected by provider-side filtering that makes a misconfigured host look correctly closed.

Security implications

The purpose of this decision is to make the negative claims testable. Positive checks — the relay answers, TLS works — fail loudly and are hard to get wrong; the claims that actually carry the security posture are the ones about what is not reachable, and those fail silently by simply never being made. That is why an off-host scan rather than a configuration read is the load-bearing part, and why #35 requires proof that the suite fails against a deliberately broken host: a negative check that has only ever passed may not be a check at all. Two cautions attach to it. Scanning must be scoped to the cohort's own host and should be confirmed against the provider's acceptable-use policy before it runs from CI. And the smoke test's output describes the host's exposure, so it must not be published anywhere the public repository would carry it, per launchpad/AGENTS.md section 8.

Supersedes

none

---

_{Filed by an AI agent (Claude Opus 5) on behalf of @tucktuck101, as an ADR raised from #5's open questions per launchpad/AGENTS.md section 4. Verified by reading the repository at launchpad-issue-pr-scaffold: the loopback validation command and production notes in deploy/compose/README.md, the !reset and 80/443 publishing in compose.caddy.yml, the reverse_proxy relay:3000 line in Caddyfile, the unpublished BUZZ_HEALTH_PORT/BUZZ_METRICS_PORT and published relay port in compose.yml, the ports in compose.dev.yml, the floating BUZZ_IMAGE=ghcr.io/block/buzz:main default in .env.example, and the text of #5, #22, #23, #26 and #35. Not verified: any of these ports' actual reachability on a deployed host, the cohort VPS provider's acceptable-use policy on port scanning, and whether a public endpoint or DNS name exists yet — nothing was scanned or executed.}

[serina-mcfall]

Recommendation (for confirmation — not a decision; per §5 rule 1 I'm drafting, not deciding):

Floor B — reachability/TLS (Floor A) plus the negative assertions: an off-host scan confirming PostgreSQL, Redis, object-storage admin, and the relay's own health/metrics ports are not reachable from the internet, and no other unexpected port answers. Floor A alone is already flagged as insufficient (it would pass cleanly even with Postgres open to the internet — it certifies the one thing that was never in doubt). Floor D is flagged as unreachable-therefore-meaningless (setting the minimum equal to the full verification suite's scope means nothing can be declared healthy until the last check exists, which leaves the deploy step with no bar at all in the meantime).

Floor C (version-pin + admin-access-policy checks) is the natural next step once #26 (admin access model) is decided, but the issue is explicit that Floor B "can be decided and used now" — including by hand against the destroyable VM, before any CI pipeline exists — so there's no reason to wait on #26 to lock in the floor itself.

Worth flagging: the issue notes port-scanning may be constrained by the VPS provider's acceptable-use policy, and that hasn't been checked yet — worth a quick look before this runs anywhere for real, separate from deciding the floor.

If this matches your read, this is ready to move to a PR that writes launchpad/decisions/ADR-00XX-external-security-smoke-test-floor.md and closes this issue. Flag me if you'd rather go a different direction.

[serina-mcfall]

Follow-up to the recommendation above: a contingency plan for Floor B, to fold into this ADR's Consequences section.

Floor B confirms reachability/TLS and that the internal-only ports aren't exposed — it does not confirm the deployed relay is actually running the pinned image version. That's Floor C's job, and it's deliberately deferred here since it also needs #26 (admin access model) decided first for its remote-access-policy check.

The accepted gap, stated rather than hidden: a deployment could pass every Floor B check — TLS works, nothing internal is exposed — while still running a stale or wrong image, and Floor B alone would call that "healthy."

Trigger for upgrading to Floor C: either #26 lands (making the remote-access-policy half of Floor C decidable), or a real incident where a deployment passed Floor B while running the wrong version — whichever comes first.

The fix, once the trigger fires: add the two checks Floor C names on top of Floor B — confirm the deployed image matches the pinned BUZZ_IMAGE, and confirm remote privileged access matches whatever #26 decides. Nothing about Floor B needs reworking; Floor C is additive.

Safety net in the meantime: the image-pin check isn't exclusively an external smoke-test problem — it's largely covered already by #141's fix (this fork's deploy guard rejects floating tags) and the ADR-0005 boundary check, so the gap Floor B leaves open is narrower in practice than it sounds; Floor C mainly adds external, off-host confirmation of what those internal checks already assert.