Skip to content

Mob.Test.capabilities/1 — ask the app what it can be probed with (MOB-151) - #133

Merged
GenericJam merged 1 commit into
masterfrom
feat/mob-151-capabilities
Sep 5, 2026
Merged

Mob.Test.capabilities/1 — ask the app what it can be probed with (MOB-151)#133
GenericJam merged 1 commit into
masterfrom
feat/mob-151-capabilities

Conversation

@GenericJam

Copy link
Copy Markdown
Owner

Every helper in Mob.Test is a thin :rpc.call into :mob_nif, and which of them an app can serve is a runtime fact, not a property of the platform:

  • Android: each harness NIF bails with {:error, :not_loaded} when its cached MobBridge method is absent. MobBridge.kt is app-owned, generated once, never re-rendered — the same drift that crash-looped an app on MobBridge.torch.
  • iOS: the whole harness sits inside #if !MOB_RELEASE, so a release build leaves the Erlang stubs behind.

That's why this is a NIF reading the bridge cache on Android and the gate macros on iOS, rather than an Elixir table — a table would reproduce exactly the drift the ticket exists to remove.

It found something immediately

A freshly generated Android app has no synthetic input at all:

iOS debug simAndroid (generated)
screen_info, element_frames, screenshot, scroll_info, scroll_to
tap_xy, tap_by_label, swipe_xy, long_press_xy, type_text, delete_backward, clear_text
view_tree, ui_tree, ax_action, sample_region

The mob_new template's MobBridge ships none of the input methods. Filed as MOB-160. The support matrix in this module listed tap_xy on Android as n/a, which reads as "not applicable" rather than "not implemented"; it now says so, and says plainly that the table is a snapshot and the probe is the truth.

What the review caught

A false positive, in the direction that hurts. The first version reported sample_region: true on Android from Bridge.screenshot, on the theory that the crop is served from a screenshot as it is on iOS. Android has no sample_region NIF at all — one grep hit, the atom I'd just added. An agent would read true, commit to pixel verification, and hit a stub raise mid-run: precisely the failure this removes, now with a false assurance attached. Worth stating plainly that my device verification did not catch this — I read sample_region: true in the output as a pass.

The iOS rationale was wrong. I claimed the NIF must sit outside the release gate or it'd be missing from the build whose reduced capability it reports. But mob_beam.m drops -name/-setcookie under MOB_RELEASE, so a release build has no distribution and the call returns {:badrpc, :nodedown} before reaching it. The placement is still right; the reason wasn't, and the comment now says what's true.

:rpc.call/4 waits forever — on the first call an agent makes, a wedged-but-reachable node (a plugged-in iPhone whose BEAM suspends) would hang indefinitely, worse than the error it replaces. It takes a timeout now, which also makes the existing :timeout guard reachable. And badrpc shapes are classified rather than lumped: an app predating the NIF is :unknown; a failed load_nif is false with dist_rpc: true, since every NIF is down and :unknown would send an agent off to try probes that cannot work.

The tests were largely vacuous, and the review proved it by running the mutations. Deleting the -export line left them green — the highest-consequence mutation in the diff, since load_nif then fails for the whole module and every app crashes at boot — because the regex was dotall-greedy and matched the -nifs block. Moving the iOS registration inside the gate left them green, because the assertion checked where the definition sits, not the registration; it now uses the same #if-nesting parser as Mob.ReleaseScreenshotTest. The Android assertions checked string presence rather than key/value pairing, so swapping two lines of a parallel array was invisible. All nine mutations are now caught against an exact baseline.

Verification

Device-verified on both platforms after the fixes: iOS debug simulator reports all sixteen probes true; the generated Android app reports the five above with sample_region correctly false. Degradation paths verified on live nodes too — an app without the NIF returns :unknown per probe with dist_rpc: true; an unreachable node returns all false.

1539 tests, credo and format clean.

Every helper in Mob.Test is a thin :rpc.call into :mob_nif, and which of them
an app can serve is a runtime fact rather than a property of the platform. On
Android each harness NIF bails with {:error, :not_loaded} when its cached
MobBridge method is absent, and MobBridge.kt is app-owned, generated once and
never re-rendered — the same drift that crash-looped an app on
MobBridge.torch. On iOS the whole harness lives inside #if !MOB_RELEASE, so a
release build leaves the Erlang stubs behind.
So the answer has to come from the running app. A table in the docs would
reproduce exactly the problem this removes, which is why capabilities/0 is a
NIF reading the bridge cache on Android and the gate macros on iOS, rather
than an Elixir lookup.
It found something immediately. On a freshly generated Android app only
element_frames, screen_info, screenshot, scroll_info and scroll_to are
available: there is no synthetic input at all — no tap_xy, swipe_xy,
long_press_xy, type_text, delete_backward or clear_text, because the mob_new
template's MobBridge ships none of them. Filed as MOB-160. The support matrix
in this module listed tap_xy on Android as "n/a", which reads as "not
applicable" rather than "not implemented"; it now says so, and says plainly
that the table is a snapshot and the probe is the truth.
An adversarial review found the first version reported sample_region: true on
Android from Bridge.screenshot, on the theory that the crop is served from a
screenshot the way it is on iOS. Android has no sample_region NIF at all — one
grep hit, the atom I had just added. An agent would have read `true`, committed
to pixel verification, and hit a stub raise mid-run, which is the precise
failure this feature exists to prevent, now with a false assurance attached.
Worth recording that my device verification did not catch this: I read
sample_region: true in the output as a pass.
The same review dismantled the iOS rationale. The comment claimed the NIF must
sit outside the release gate or it would be missing from the build whose
reduced capability it reports — but mob_beam.m drops -name and -setcookie under
MOB_RELEASE, so a release build has no distribution and capabilities/1 gets
{:badrpc, :nodedown} before reaching it. The placement is still right; the
reason was wrong, and the comment now says what is actually true.
Also from the review: :rpc.call/4 waits forever, which on the first call an
agent makes turns a wedged-but-reachable node into an indefinite hang — worse
than the error it replaces. It takes a timeout now, which also makes the
existing :timeout guard reachable. And the badrpc shapes are classified rather
than lumped: an app predating the NIF is :unknown, a failed load_nif is false
with dist_rpc: true (every NIF is down, so :unknown would send an agent off to
try probes that cannot work), and anything else is unreachable.
The tests were largely vacuous and the review proved it by running the
mutations. Deleting the -export line left them green — the highest-consequence
mutation in the diff, since load_nif then fails for the whole module and every
app crashes at boot — because the regex was dotall-greedy and matched the -nifs
block instead. Moving the iOS registration inside the gate left them green,
because the assertion checked where the definition sits, not the registration;
it now uses the same #if-nesting parser as Mob.ReleaseScreenshotTest. The
Android assertions checked that strings were present, not that keys and values
were paired, so swapping two lines of a parallel array was invisible. All nine
mutations are now caught against an exact baseline.
Device-verified on both platforms after the fixes: iOS debug simulator reports
all sixteen probes true; the generated Android app reports the five above, with
sample_region now correctly false.
Refs MOB-151, MOB-160
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@GenericJam
GenericJam merged commit 01815bf into masterSep 5, 2026
4 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@GenericJam