Add mix mob.attest — prove the device is running the code you just pushed (MOB-152)

CHANGELOG.md

@@ -1,17 +1,19 @@
-# Changelog
-
-All notable changes to **mob_dev** are documented here.
-
-Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [SemVer](https://semver.org/spec/v2.0.0.html).
-
-Full module documentation: [hexdocs.pm/mob_dev](https://hexdocs.pm/mob_dev).
-
----
-
 ## [Unreleased]
 
 ### Added
 
+- **`mix mob.attest`** — prove the device is running the code you just pushed.
+  Compares `module_info(:md5)` on the device against `:beam_lib.md5/1` of the
+  local `.beam`, so a push that never landed, landed in the wrong place, or
+  landed and was never loaded all show up. Exits non-zero on a mismatch, and
+  also when the check could not run — a check that could not run is not a check
+  that passed. Modules the device has not loaded yet are reported and are not a
+  failure. Defaults to the exact set `mix mob.deploy` pushes, so what is
+  attested cannot drift from what was shipped. Written after a bundle-id
+  divergence let a BEAM push succeed against the wrong app's container,
+  printing a tick while the app kept running old code; it found a second live
+  instance (MOB-161) on its first real use.
+
 - **`mix mob.mutate`** — mutation testing for the lines a branch changed.
   Breaks the production code one line at a time and reports the changes nothing
   noticed, which is the only cheap way to tell a test that guards something
@@ -21,13 +23,20 @@ Full module documentation: [hexdocs.pm/mob_dev](https://hexdocs.pm/mob_dev).
   running, never supplied. `--file`, `--base`, `--test-command`, `--max`,
   `--json`; exits non-zero when anything survives.
 
-### Fixed
+- **`mix mob.deploy --json`** — a machine-readable result on stdout listing the
+  deployed, failed and skipped devices with their per-device reasons, and an
+  `outcome` that mirrors the exit status. Progress output is redirected to
+  stderr for the run, so `mix mob.deploy --json | jq` receives exactly one
+  document. Emitted on the native-build failure path too, which is when a
+  caller most needs it.
 
-- **`mix mob.deploy` silently ignored unknown options.** `-d` was never aliased
-  to `--device` here — `mob.connect` has aliased it all along — so
-  `mix mob.deploy -d <udid>` deployed to *every* connected device instead of
-  the one named, without a word. A typo'd `--devcie` did the same. Parsing is
-  strict now and the run refuses, naming the offending options.
+- **`:ios_bundle_id`** in `mob.exs`, for when Android's `applicationId` is not a
+  legal Apple bundle id. Apple forbids the underscores Android allows, and
+  `com.example.*` is often already claimed by another Apple team, so the two
+  frequently cannot be the same string. Every iOS path — deploy, connect,
+  provision, battery bench, uninstall, the simulator and device builds,
+  code-signing, and the release IPA — resolves `:ios_bundle_id || :bundle_id`.
+  Android keeps using `:bundle_id`.
 
 ### Changed
 
@@ -49,17 +58,6 @@ Full module documentation: [hexdocs.pm/mob_dev](https://hexdocs.pm/mob_dev).
   A `--native` run that built the artifact and found no device to push it to
   still exits 0 — "build the APK now, attach the phone after" is unchanged.
 
-### Added
-
-- **`mix mob.deploy --json`** — a machine-readable result on stdout listing the
-  deployed, failed and skipped devices with their per-device reasons, and an
-  `outcome` that mirrors the exit status. Progress output is redirected to
-  stderr for the run, so `mix mob.deploy --json | jq` receives exactly one
-  document. Emitted on the native-build failure path too, which is when a
-  caller most needs it.
-
-### Changed
-
 - **`mix mob.deploy` now exits non-zero when a device fails.** A run that
   printed `Failed on 1 device(s)` previously still returned status 0, so CI and
   wrapper scripts read a failed deploy as success. Every device is still
@@ -69,18 +67,14 @@ Full module documentation: [hexdocs.pm/mob_dev](https://hexdocs.pm/mob_dev).
   devices and has been passing, it may now fail** — check whether it was
   passing on a partial deploy.
 
-### Added
-
-- **`:ios_bundle_id`** in `mob.exs`, for when Android's `applicationId` is not a
-  legal Apple bundle id. Apple forbids the underscores Android allows, and
-  `com.example.*` is often already claimed by another Apple team, so the two
-  frequently cannot be the same string. Every iOS path — deploy, connect,
-  provision, battery bench, uninstall, the simulator and device builds,
-  code-signing, and the release IPA — resolves `:ios_bundle_id || :bundle_id`.
-  Android keeps using `:bundle_id`.
-
 ### Fixed
 
+- **`mix mob.deploy` silently ignored unknown options.** `-d` was never aliased
+  to `--device` here — `mob.connect` has aliased it all along — so
+  `mix mob.deploy -d <udid>` deployed to *every* connected device instead of
+  the one named, without a word. A typo'd `--devcie` did the same. Parsing is
+  strict now and the run refuses, naming the offending options.
+
 - **iOS deploys installed the app under one bundle id and pushed BEAMs at
   another.** `:ios_bundle_id` was resolved by the build but discarded by the
   deployer and connector, so `mix mob.deploy --native --device <udid>` installed

README.md

@@ -35,6 +35,7 @@ end
 | `mix mob.watch` | Auto-push BEAMs on file save |
 | `mix mob.watch_stop` | Stop a running `mix mob.watch` |
 | `mix mob.devices` | List connected devices and their status |
+| `mix mob.attest` | Prove a device is running the code you just pushed — compares module digests, not artifacts ([see below](#did-that-deploy-actually-land-mix-mobattest)) |
 | `mix mob.push` | Hot-push only changed modules (no restart) |
 | `mix mob.enable <feature>...` | Wire up an optional Mob feature — platform-manifest entries, Elixir stubs, dep injections ([see below](#mix-mobenable-feature)) |
 | `mix mob.add_nif <name>` | Scaffold a statically-linked NIF — Elixir stub + `mob.exs` `:static_nifs` append + optional native skeleton ([see below](#mix-mobadd_nif-name)) |
@@ -93,6 +94,41 @@ If dist is not reachable (first deploy, app not running), it falls back to `adb
 
 **Requirements:** The app must call `Mob.Dist.ensure_started/1` at startup, and the cookie must match the one in `mob.exs` (default `:mob_secret`).
 
+## Did that deploy actually land? (`mix mob.attest`)
+
+`mix mob.deploy` reports what it *did*. It does not report what is now *true*,
+and the two come apart more often than the exit code suggests.
+
+Two real cases: a bundle-id divergence sent the BEAM push into one app's
+container while a *different* app was running — it succeeded and printed a tick,
+because both containers existed on the device. And a plain dist deploy reported
+success while twelve modules on the device kept their old code.
+
+```bash
+mix mob.connect --no-iex     # set up the tunnel
+mix mob.attest               # compare the device against this build
+mix mob.attest --json        # machine-readable, for CI or an agent
+```
+
+```
+mob_plugin_demo_ios_8a4250e9@127.0.0.1: 55 match, 12 stale, 0 not loaded, 0 unreadable
+  stale: Mob.Renderer
+  12 module(s) on the device do not match this build. The app is running code
+  you did not just push.
+```
+
+It compares `module_info(:md5)` on the device against `:beam_lib.md5/1` of the
+local `.beam` — the same digest for the same bytes. Deliberately **not** an
+artifact hash: two builds of the same source differ in timestamps and paths, so
+that would report a mismatch on every rebuild, and a check that cries wolf gets
+switched off.
+
+Exits non-zero when a module differs, and also when the check could not run —
+a check that could not run is not a check that passed. Modules the device has
+not loaded yet are reported and are **not** a failure: interactive BEAM loads a
+module when something first calls it, so most of a bundle is legitimately
+unloaded at any moment.
+
 ## `mix mob.enable <feature>`
 
 Wires up an optional Mob feature in one command — platform-manifest

decisions/2026-09-05-attest-what-is-running.md

@@ -0,0 +1,87 @@
+# Deployment attestation compares module digests, not artifacts
+
+- Date: 2026-09-05
+- Status: accepted
+
+## Context
+
+`mix mob.deploy` reports what it did. It does not report what is now true, and
+the two come apart more often than the exit code suggests.
+
+Two instances from one session. A bundle-id divergence sent the BEAM push into
+one app's container while a different app was running: it did not fail with
+"not installed", it succeeded and printed a tick, because both containers
+existed on the device. And a plain dist deploy reported success while twelve
+`mob` modules on the device kept their old digests (MOB-161).
+
+In both cases every individual step was honest about itself and the run as a
+whole was wrong. Every guide in these repos says "verify effects, not exit
+codes" — an instruction that exists precisely because the tools cannot be
+trusted, and that only works while someone remembers to follow it.
+
+## Decision
+
+Compare `module_info(:md5)` on the device against `:beam_lib.md5/1` of the
+local `.beam`. They are the same digest for the same bytes, so a module that
+never arrived, arrived in the wrong place, or arrived and was never loaded all
+show up, and no new machinery has to be invented to produce a fingerprint.
+
+Deliberately not hashing artifacts, directories or the IPA/APK. Two builds of
+the same source differ in timestamps and paths, so an artifact hash reports a
+mismatch on every rebuild; a check that cries wolf gets switched off, and a
+switched-off check is worth less than none because its absence is invisible.
+
+Four verdicts rather than two:
+
+* `:stale` is fatal. The device is running code we did not build.
+* `:unreadable` is fatal. The check could not run, and a check that could not
+  run must not report success — the same rule as a deploy exiting 0 having
+  shipped nothing.
+* `:missing` is reported and is **not** fatal. Interactive BEAM loads a module
+  when something first calls it, so most of a bundle is legitimately unloaded
+  at any moment. Failing on that would make the tool unusable within a day.
+* `:match` is the only pass.
+
+Attestation connects without restarting. `Connector.connect_all/1` restarts the
+app, which reloads every module and would destroy the evidence being read. The
+tunnels a previous `mix mob.connect` set up are device-level and outlive it, so
+a plain `Node.connect/1` is enough.
+
+A run that reaches no device raises rather than reporting success, for the same
+reason `:unreadable` is fatal.
+
+## Consequences
+
+It is a separate task, not a step inside `mob.deploy`. Deploy cannot always
+reach the node — a physical iPhone with USB attached suspends its BEAM — and
+folding a check that sometimes cannot run into the thing it checks would either
+weaken the deploy's exit code or produce false failures. Wiring it in as an
+opt-in deploy flag is the obvious follow-up.
+
+It only sees modules the device has loaded, so it cannot distinguish "shipped
+but not yet loaded" from "never shipped". `mix mob.snapshot_loaded` answers the
+shipped-set question and the two are complementary.
+
+It found MOB-161 on its first real use, which is the argument for building it
+and also a caution: the failure it found had been happening silently, and there
+is no way to know for how long.
+
+That sentence needs a correction, though. The first version of this task scoped
+itself to the project's own application, and MOB-161's twelve stale modules
+were in `mob` — a dependency. So the default would have reported `:ok`, and the
+evidence for the design working was only obtainable by passing `--app mob`,
+which nothing suggested doing. The default is now the exact set
+`mix mob.deploy` pushes (`HotPush.runtime_beam_dirs/0`), so what is attested
+and what was shipped cannot drift apart, and `--app` narrows rather than
+replaces.
+
+A second correction, from the same review. This record originally justified
+`:missing` being non-fatal with "interactive BEAM loads a module when something
+first calls it, so most of a bundle is legitimately unloaded". Measured on a
+device, the inference runs the other way: the code server *is* interactive, so
+the probe itself triggers the load and returns a digest. `:undef` therefore
+means the module is on no code path at all, which for something that was just
+pushed is a real failure — it is now fatal. The check is correspondingly
+stronger than described, comparing the file the device would load rather than
+the resident set, and it has a side effect worth stating: probing an unloaded
+module loads it.