From 143a76d23f3f07cf37fa7c54f50c34ce47735342 Mon Sep 17 00:00:00 2001 From: GenericJam Date: Sat, 5 Sep 2026 13:34:46 -0600 Subject: [PATCH] Run the input NIFs on a dirty scheduler instead of stalling the device MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Android's default BEAM argv is `-S 1:1` — one normal scheduler. Every harness input NIF blocks it waiting on the platform UI thread, and all but one were registered with `.flags = 0`. MOB-160 made this acute: Android gestures now hold the pointer for their real duration, because the platform's detectors wait on posted callbacks and ignore synthesised timestamps. `long_press_xy(node, x, y, 800)` blocks for 800ms, roughly 800x the budget a NIF is supposed to take. For that window nothing on the device runs — no timers, no renders, no :rpc, no PubSub. An agent driving the UI would be pausing the app it is trying to observe. iOS was already there and nobody had noticed. On the device branch nif_long_press_xy sleeps the caller's full duration via [NSThread sleepForTimeInterval:], and nif_ax_action_at_xy sleeps up to 4 x 50ms retrying its accessibility lookup — 150ms on the only scheduler, behind Mob.Test.toggle/2, dismiss_alert/2 and adjust_slider/4. So: tap, tap_xy, long_press_xy, swipe_xy, type_text, delete_backward and clear_text go IO_BOUND on both platforms, plus key_press, ax_action and ax_action_at_xy on iOS. IO_BOUND rather than CPU_BOUND because they are waiting on another thread, not computing. The flags diverge by platform because the code does: Android's key_press is a hardcoded :not_implemented stub that never touches the UI thread, so a dirty hop would buy nothing, and Android has no accessibility path at all. This reverses the note above nif_funcs[], which held these on regular schedulers on the grounds that the harness calls them in tight loops and dirty-dispatch overhead would add up, pending benchmarks that were never run. That trade is the wrong way round: the cost is a thread wakeup per call, and the thing traded away is the whole VM. The note is rewritten rather than left to contradict the table ten lines below it. Safety checked empirically rather than inferred: registering dirty flags on an ERTS built without dirty scheduler support fails module load and bricks every app at boot. The running device reports dirty_io_schedulers: 1 and dirty_cpu_schedulers: 1 on OTP 29, and master already ships dirty NIFs on the boot DNS path. Also documents the user-facing half, which was unshipped: - The platform matrix covers long_press_xy/4, type_text/2, delete_backward/1 and clear_text/1, and marks tap_xy/3 and swipe/5 working on Android — for apps generated by mob_new 0.4.32+, since the methods live in the app's own generated bridge. The table cannot know that; capabilities/1 can. - What Android synthetic input costs: gestures block for their real duration, only the activity's own window is reachable (not dialogs or modal sheets), type_text is ASCII-only and rejects a whole string over one bad character, clear_text is deliberately absent. - scroll_info/2 returns device PIXELS while element_frames/1 returns dp and tap_xy/swipe take dp. Feeding one to the other overshoots by the display density — 2.75x on a moto g power. Newly reachable now that Android can swipe. - The ✱ footnote no longer blames the IOHID injection path for type_text, delete_backward and clear_text, which do not use it. Test asserts the registration flags on both platforms, and asserts the inverse for Android's key_press so the "it is dirty because it waits" rule stays true of its own members. Mutation-checked: flipping a flag back to 0 fails it. Rebuild native to pick this up — the registration table compiles into each app. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 32 ++++++ android/jni/mob_nif.zig | 16 +-- .../2026-09-05-input-nifs-are-dirty-io.md | 87 +++++++++++++++ ios/mob_nif.m | 40 ++++--- lib/mob/test.ex | 82 ++++++++++++-- test/mob/input_nif_scheduling_test.exs | 104 ++++++++++++++++++ 6 files changed, 328 insertions(+), 33 deletions(-) create mode 100644 decisions/2026-09-05-input-nifs-are-dirty-io.md create mode 100644 test/mob/input_nif_scheduling_test.exs diff --git a/CHANGELOG.md b/CHANGELOG.md index ce1cea7..a6a1834 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,38 @@ Full module documentation: [hexdocs.pm/mob](https://hexdocs.pm/mob). ## [Unreleased] +### Changed +- **Input NIFs now run on a dirty IO scheduler.** `tap`, `tap_xy`, + `long_press_xy`, `swipe_xy`, `type_text`, `delete_backward` and `clear_text` + on both platforms, plus `key_press`, `ax_action` and `ax_action_at_xy` on + iOS, are registered `ERL_NIF_DIRTY_JOB_IO_BOUND`. Every one of them blocks + waiting on the platform UI thread, and Android runs with a single normal + scheduler (`-S 1:1`), so an 800ms `long_press_xy` was stopping every process + on the device — timers, renders, `:rpc`, PubSub — for the duration. iOS's + `nif_long_press_xy` had been sleeping the caller's full duration on a normal + scheduler since it was written, and `nif_ax_action_at_xy` sleeps up to + 4 x 50ms retrying its lookup. + + **Rebuild native** to pick this up (`mix mob.deploy --native`): the NIF + registration table is compiled into each app. See + `decisions/2026-09-05-input-nifs-are-dirty-io.md`. + +### Documented +- **Android synthetic input in `Mob.Test`.** The platform matrix now covers + `long_press_xy/4`, `type_text/2`, `delete_backward/1` and `clear_text/1`, and + marks `tap_xy/3` and `swipe/5` as working on Android — for apps generated by + `mob_new` 0.4.32 or newer, since the methods live in the app's own generated + bridge. The new ⊕ footnote covers what that costs: gestures block for their + real duration, only the activity's own window is reachable (not dialogs or + modal sheets), `type_text/2` is ASCII-only and rejects a whole string + containing one unmappable character, and `clear_text/1` is deliberately + absent on Android rather than broken. +- **`scroll_info/2` returns device pixels**, while `element_frames/1` returns + dp and `tap_xy/3` and `swipe/5` take dp. Feeding one into the other + overshoots by the display density. +- The `✱` footnote no longer attributes `type_text/2`, `delete_backward/1` and + `clear_text/1` to the IOHID injection path they do not use. + ### Added - **`Mob.Test.capabilities/1`** — ask a build which test-harness probes it can actually serve, before choosing how to drive it. Which probes work is a diff --git a/android/jni/mob_nif.zig b/android/jni/mob_nif.zig index d85c98a..f12fb64 100644 --- a/android/jni/mob_nif.zig +++ b/android/jni/mob_nif.zig @@ -4262,14 +4262,16 @@ const nif_funcs = [_]erts.ErlNifFunc{ .{ .name = "ax_action_at_xy", .arity = 3, .fptr = nif_ax_action_at_xy, .flags = 0 }, .{ .name = "ui_debug", .arity = 0, .fptr = nif_ui_debug, .flags = erts.ERL_NIF_DIRTY_JOB_CPU_BOUND }, .{ .name = "screen_info", .arity = 0, .fptr = nif_screen_info, .flags = 0 }, - .{ .name = "tap", .arity = 1, .fptr = nif_tap, .flags = 0 }, - .{ .name = "tap_xy", .arity = 2, .fptr = nif_tap_xy, .flags = 0 }, - .{ .name = "type_text", .arity = 1, .fptr = nif_type_text, .flags = 0 }, - .{ .name = "delete_backward", .arity = 0, .fptr = nif_delete_backward, .flags = 0 }, + .{ .name = "tap", .arity = 1, .fptr = nif_tap, .flags = erts.ERL_NIF_DIRTY_JOB_IO_BOUND }, + .{ .name = "tap_xy", .arity = 2, .fptr = nif_tap_xy, .flags = erts.ERL_NIF_DIRTY_JOB_IO_BOUND }, + .{ .name = "type_text", .arity = 1, .fptr = nif_type_text, .flags = erts.ERL_NIF_DIRTY_JOB_IO_BOUND }, + .{ .name = "delete_backward", .arity = 0, .fptr = nif_delete_backward, .flags = erts.ERL_NIF_DIRTY_JOB_IO_BOUND }, + // Not dirty: this one is a stub that returns :not_implemented without + // touching the UI thread. Flag it when it grows a real body. .{ .name = "key_press", .arity = 1, .fptr = nif_key_press, .flags = 0 }, - .{ .name = "clear_text", .arity = 0, .fptr = nif_clear_text, .flags = 0 }, - .{ .name = "long_press_xy", .arity = 3, .fptr = nif_long_press_xy, .flags = 0 }, - .{ .name = "swipe_xy", .arity = 4, .fptr = nif_swipe_xy, .flags = 0 }, + .{ .name = "clear_text", .arity = 0, .fptr = nif_clear_text, .flags = erts.ERL_NIF_DIRTY_JOB_IO_BOUND }, + .{ .name = "long_press_xy", .arity = 3, .fptr = nif_long_press_xy, .flags = erts.ERL_NIF_DIRTY_JOB_IO_BOUND }, + .{ .name = "swipe_xy", .arity = 4, .fptr = nif_swipe_xy, .flags = erts.ERL_NIF_DIRTY_JOB_IO_BOUND }, .{ .name = "screenshot", .arity = 3, .fptr = nif_screenshot, .flags = erts.ERL_NIF_DIRTY_JOB_CPU_BOUND }, .{ .name = "scroll_info", .arity = 1, .fptr = nif_scroll_info, .flags = 0 }, .{ .name = "scroll_to", .arity = 3, .fptr = nif_scroll_to, .flags = 0 }, diff --git a/decisions/2026-09-05-input-nifs-are-dirty-io.md b/decisions/2026-09-05-input-nifs-are-dirty-io.md new file mode 100644 index 0000000..f3cd40f --- /dev/null +++ b/decisions/2026-09-05-input-nifs-are-dirty-io.md @@ -0,0 +1,87 @@ +# Harness input NIFs run on a dirty IO scheduler + +Date: 2026-09-05 +Status: accepted +Ticket: MOB-160 + +## Context + +Android's default BEAM argv is `-S 1:1 -SDcpu 1:1 -SDio 1 -A 1` +(`android/jni/mob_beam.zig`). One normal scheduler. Whatever blocks it blocks +every process on the device. + +Every harness input NIF blocks. They hand work to the platform UI thread and +wait for the answer: Android through a `CountDownLatch` against a main-thread +coroutine, iOS through `dispatch_sync` to the main queue. Until now all of +them except iOS's `tap_xy` were registered with `.flags = 0`. + +MOB-160 made this acute rather than theoretical. Android gestures now have to +hold the pointer for their real duration, because the platform's long-press +and drag detectors wait on posted callbacks and frame boundaries and ignore +synthesised timestamps. `long_press_xy(node, x, y, 800)` blocks for 800ms. +iOS was already there and nobody noticed: on the device branch +`nif_long_press_xy` calls `[NSThread sleepForTimeInterval:]` for the caller's +full duration on a normal scheduler. (The simulator branch drives the press +through `_setState:` instead and does not sleep. The device sleep is doubly +wasteful, since per the 2026-08-09 decision that injection path is accepted +and never delivered — it is 800ms of dead time.) + +800ms is roughly 800 times the ~1ms budget the Erlang efficiency guide gives a +NIF. For that window the device does nothing at all: no timers, no renders, no +`:rpc`, no PubSub. An agent driving the UI would be pausing the app it is +trying to observe. + +## Decision + +`tap`, `tap_xy`, `long_press_xy`, `swipe_xy`, `type_text`, `delete_backward` +and `clear_text` are registered `ERL_NIF_DIRTY_JOB_IO_BOUND` on both platforms. +iOS additionally flags `key_press`, `ax_action` and `ax_action_at_xy`. + +(There is no `tap_by_label` NIF on either platform — it is a capability atom +and a Kotlin method name; the NIF behind it is `tap/1`.) + +The flags diverge by platform because the code does. Android's `key_press` is +a hardcoded `:not_implemented` stub that never touches the UI thread, so a +dirty hop would buy nothing; iOS's `dispatch_sync`s. Android has no +accessibility path at all, while iOS's `nif_ax_action_at_xy` retries its +lookup four times with `[NSThread sleepForTimeInterval:0.05]` between +attempts — up to ~150ms of literal sleep, which is the same bug in a place +nobody was looking. + +IO-bound rather than CPU-bound: they are not computing, they are waiting on +another thread, which is what that flag is for. The default argv already +provisions a dirty IO scheduler (`-SDio 1`), so they get a thread that is not +the one running everyone else's processes. + +This extends `decisions/2026-08-09-tap-xy-reports-observed-effect.md`, which +moved iOS's `tap_xy` because it blocks for the settle window. That reasoning +was right and was applied too narrowly — Android's copy of `tap_xy` never +followed at all — but it does not simply generalise: only `tap_xy` has a settle +window. What covers the rest is plainer, and was true the whole time: they +block on `dispatch_sync` or on a latch. + +The previous note above `nif_funcs[]` in `ios/mob_nif.m` did state a principle, +so this is a reversal rather than a gap. It held that the harness calls these +in tight loops and that dirty-dispatch overhead would add up, pending +benchmarks that were never run. That trade is the wrong way round: the cost is +a thread wakeup per call (real — `-sbwtdio none` means the dirty scheduler +does not busy-wait, so each call pays one), and the thing being traded away is +the whole VM for the duration. We have not measured the wakeup cost, and are +accepting it deliberately rather than claiming it is free. + +## Consequences + +- A blocking input NIF costs a dirty IO scheduler slot, not the VM. +- There is exactly one dirty IO scheduler (`-SDio 1`), so this converts a + total stall into head-of-line blocking on a resource of size one. An 800ms + long press now delays anything else that is IO-dirty — on Android that + includes `resolve_ipv4` (the DNS path), `audio_output_level` and + `vendor_usb_bulk_write`; on iOS, `safe_area`, which is on a layout path. + Strictly better than what it replaces, but it is a new contention edge, and + the answer if it bites is to raise `-SDio`, not to go back. +- `long_press_xy` still takes its full duration. That is inherent — a long + press that returns early is not a long press. The call is now merely slow + rather than globally stalling. +- The rule to apply to anything added here later: **if it waits on the UI + thread, it is dirty.** The old table had no principle, which is how one + member of a group of nine ended up flagged correctly and eight did not. diff --git a/ios/mob_nif.m b/ios/mob_nif.m index 52a7753..9934f95 100644 --- a/ios/mob_nif.m +++ b/ios/mob_nif.m @@ -7969,15 +7969,21 @@ static ERL_NIF_TERM nif_vendor_usb_close(ErlNifEnv *env, int argc, const ERL_NIF // * ui_tree — recursive UIAccessibility walk (variable, can be 10s of ms) // * ui_debug — same walk, more output // -// Synthetic-input NIFs (swipe_xy, long_press_xy, type_text, key_press, -// delete_backward, clear_text) dispatch_sync to the main queue but also do -// some pre-dispatch work; they're left on regular schedulers for now because -// the test harness calls them in tight loops and dirty-dispatch overhead would -// add up. Re-evaluate if benchmarks show scheduler stalls under heavy harness use. +// The input NIFs (tap, tap_xy, swipe_xy, long_press_xy, type_text, key_press, +// delete_backward, clear_text, ax_action, ax_action_at_xy) are all +// ERL_NIF_DIRTY_JOB_IO_BOUND: every one of them blocks a scheduler waiting on +// the main queue. See decisions/2026-09-05-input-nifs-are-dirty-io.md. // -// tap_xy is the exception: it blocks up to MOB_TAP_SETTLE_MS waiting for the -// app to react (that wait is what makes its :ok trustworthy), which is far too -// long to hold a normal scheduler. +// This reverses an earlier note here, which kept them on regular schedulers on +// the grounds that the harness calls them in tight loops and dirty-dispatch +// overhead would add up, pending benchmarks. That trade was the wrong way +// round: the overhead is a thread wakeup, while the stall is the whole VM. +// nif_long_press_xy sleeps the caller's full duration on the device branch, +// and nif_ax_action_at_xy sleeps up to 4 x 50ms retrying its lookup — on +// Android there is exactly one normal scheduler, so that is the entire device. +// +// The rule for anything added below: if it waits on the main queue, it is +// dirty. IO_BOUND rather than CPU_BOUND, because it is waiting, not computing. static ErlNifFunc nif_funcs[] = { #if !MOB_RELEASE // ── Test harness (listed first to survive linker dead-code stripping) ────── @@ -7990,16 +7996,16 @@ static ERL_NIF_TERM nif_vendor_usb_close(ErlNifEnv *env, int argc, const ERL_NIF {"ui_paint_debug", 0, nif_ui_paint_debug, ERL_NIF_DIRTY_JOB_CPU_BOUND}, {"ui_debug", 0, nif_ui_debug, ERL_NIF_DIRTY_JOB_CPU_BOUND}, {"screen_info", 0, nif_screen_info, 0}, - {"tap", 1, nif_tap, 0}, - {"ax_action", 2, nif_ax_action, 0}, - {"ax_action_at_xy", 3, nif_ax_action_at_xy, 0}, + {"tap", 1, nif_tap, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"ax_action", 2, nif_ax_action, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"ax_action_at_xy", 3, nif_ax_action_at_xy, ERL_NIF_DIRTY_JOB_IO_BOUND}, {"tap_xy", 2, nif_tap_xy, ERL_NIF_DIRTY_JOB_IO_BOUND}, - {"type_text", 1, nif_type_text, 0}, - {"delete_backward", 0, nif_delete_backward, 0}, - {"key_press", 1, nif_key_press, 0}, - {"clear_text", 0, nif_clear_text, 0}, - {"long_press_xy", 3, nif_long_press_xy, 0}, - {"swipe_xy", 4, nif_swipe_xy, 0}, + {"type_text", 1, nif_type_text, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"delete_backward", 0, nif_delete_backward, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"key_press", 1, nif_key_press, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"clear_text", 0, nif_clear_text, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"long_press_xy", 3, nif_long_press_xy, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"swipe_xy", 4, nif_swipe_xy, ERL_NIF_DIRTY_JOB_IO_BOUND}, #endif {"capabilities", 0, nif_capabilities, 0}, #if !MOB_RELEASE || defined(MOB_ENABLE_SCREENSHOT) diff --git a/lib/mob/test.ex b/lib/mob/test.ex index 1f70e3e..13c3987 100644 --- a/lib/mob/test.ex +++ b/lib/mob/test.ex @@ -132,8 +132,12 @@ defmodule Mob.Test do | `toggle/2` | ⚠️ AX active§ | ⚠️ AX active§ | ❌ ui_tree_unavailable | | `dismiss_alert/2` | ⚠️ AX active§ | ⚠️ AX active§ | ❌ ui_tree_unavailable | | `adjust_slider/4` | ⚠️ AX active§ | ⚠️ AX active§ | ❌ ui_tree_unavailable | - | `tap_xy/3` | ⚠️ AX-activatable only¶ | ❌ no_effect¶ | n/a | - | `swipe/5` | ⚠️ scroll only| ⚠️ unverified✱| n/a | + | `tap_xy/3` | ⚠️ AX-activatable only¶ | ❌ no_effect¶ | ✅ ⊕ | + | `long_press_xy/4` | ⚠️ acceptance only✱| ⚠️ acceptance only✱| ✅ ⊕ | + | `swipe/5` | ⚠️ scroll only| ⚠️ acceptance only✱| ✅ ⊕ | + | `type_text/2` | ⚠️ acceptance only✱| ⚠️ acceptance only✱| ✅ ASCII only⊕ | + | `delete_backward/1` | ⚠️ acceptance only✱| ⚠️ acceptance only✱| ✅ ⊕ | + | `clear_text/1` | ⚠️ acceptance only✱| ⚠️ acceptance only✱| ❌ not_loaded⊕ | | `capabilities/1` | ✅ | ✅ | ✅ | **This table is a snapshot, and snapshots drift.** Ask the running app @@ -141,9 +145,9 @@ defmodule Mob.Test do Android that is a per-*app* fact, since each harness NIF bails when its cached `MobBridge` method is absent and the bridge is generated once and never re-rendered; on iOS it is per-*configuration*, since the whole harness - is compiled out of release builds. Measured on a freshly generated Android - app, `tap_xy` and `type_text` are both unavailable (MOB-160) — the `n/a` in - the rows above reads as "not applicable" but means "no bridge method". + is compiled out of release builds. Apps generated before `mob_new` 0.4.32 + have no synthetic-input methods in their bridge at all and report `false` for + every one of them; regenerate with a new enough `mob_new` to pick them up. - **†** SwiftUI doesn't expose its content as separate UIView instances — `view_tree` reaches the SwiftUI hosting view's container and stops. @@ -172,9 +176,54 @@ defmodule Mob.Test do coordinate returns `{:error, :no_effect}`. Drive taps with `tap/2` (by tag); see `tap_xy/3` and `decisions/2026-08-09-ios-device-tap-injection-has-no-effect.md`. - - **✱** `swipe/5` and `long_press/4` use the same device injection path as - `tap_xy/3` and still report `:ok` on acceptance rather than on effect. - Same root cause, not yet converted — treat their `:ok` as unverified. + - **⊕** Android synthesises input in-process, dispatching `MotionEvent`s at + the activity's decor view and `KeyEvent`s at the activity. No `adb`, no + `INJECT_EVENTS` (a signature permission no app can hold). + + **The ✅ is a property of the app, not of mob.** These call methods on the + app's own generated `MobBridge`, which ships in `mob_new` — they work in an + app generated by `mob_new` 0.4.32 or newer, and return + `{:error, :not_loaded}` in every app generated before that, however new the + `mob` it runs. `MobBridge.kt` is generated once and never re-rendered, so + an existing app needs regenerating. `capabilities/1` answers this for the + build in front of you; the table cannot. + + Four more consequences worth knowing before you rely on it: + + * **Gestures cost real wall-clock.** A long press or swipe has to hold + the pointer for its real duration, because Android's detectors wait on + posted callbacks and frame boundaries — synthesised timestamps are + ignored. `long_press_xy(node, x, y, 800)` blocks for 800ms. These NIFs + run on a dirty IO scheduler for that reason. + * **Only the activity's own window is reachable.** A `Dialog` or a + Material `ModalBottomSheet` renders in its own window, so a tap aimed + at one lands on the dimmed activity behind it. + * **`type_text/2` is ASCII-only.** The virtual keyboard has no key + sequence for emoji or accented Latin, and one unmappable character + rejects the whole string, so nothing is typed. + * **`clear_text/1` is absent, not broken.** Two implementations reported + success while clearing nothing (events coalesce faster than the field + recomposes), so the bridge ships without it and the call returns + `{:error, :not_loaded}`. Select-all-and-delete by hand, or rebuild the + field's state through your own event. + + Verified on a physical device: tap navigates, long press fires + `on_long_press`, swipe scrolls a scroll view, typing and backspace change + the field. `on_long_press` fires on `column`, `row`, `text`, `icon` and + `box` — not `button`, matching iOS — so a long press on a `button` does + nothing by design. + + - **✱** These report `:ok` when the OS *accepted* the input, not when the app + was observed to react — they have not been converted to the observation + model `tap_xy/3` uses. Treat their `:ok` as "sent", not "worked". + + Two different reasons sit behind that, and they lead to different bugs. + `swipe/5` and `long_press_xy/4` ride the same IOHID injection path as + `tap_xy/3`, which on a physical device is accepted and never delivered — + so their `:ok` there is actively misleading. `type_text/2`, + `delete_backward/1` and `clear_text/1` do NOT touch that path; they + `dispatch_sync` and message the first responder directly, so they do + something real, and merely fail to confirm it. Helpers that depend on AX return clear error tuples on Android instead of raising. Callers should match on `{:error, :not_supported_on_android}` and @@ -1197,7 +1246,14 @@ defmodule Mob.Test do :rpc.call(node, :mob_nif, :key_press, [key]) end - @doc "Clear all text in the focused input (select-all + delete)." + @doc """ + Clear all text in the focused input (select-all + delete). + + On Android this returns `{:error, :not_loaded}`: the generated bridge ships + without a `clearText` method on purpose, because both implementations tried + reported success while clearing nothing. See the `⊕` note on the platform + matrix above. + """ @spec clear_text(node()) :: :ok | {:error, atom()} def clear_text(node) do :rpc.call(node, :mob_nif, :clear_text, []) @@ -1379,6 +1435,14 @@ defmodule Mob.Test do Mob.Test.scroll_info(node, "feed") #=> %{offset: {0.0, 0.0}, content: {393.0, 2400.0}, viewport: {393.0, 756.0}, # max_offset: {0.0, 1644.0}, kind: :pixel} + + > #### These are device pixels, not dp {: .warning} + > + > `element_frames/1` returns dp, and `tap_xy/3` and `swipe/5` take dp. A + > `:pixel` scroll offset is in raw device pixels. Feeding one straight into + > the other overshoots by the display density — 2.75x on a moto g power. + > Divide by the density, or drive scrolling with `scroll_to/4`, which works + > in whatever unit `:kind` reports. """ @spec scroll_info(node(), String.t() | atom()) :: map() | {:error, term()} def scroll_info(node, id) do diff --git a/test/mob/input_nif_scheduling_test.exs b/test/mob/input_nif_scheduling_test.exs new file mode 100644 index 0000000..0e88399 --- /dev/null +++ b/test/mob/input_nif_scheduling_test.exs @@ -0,0 +1,104 @@ +# Source-contract test: NIF registration flags are a property of the native +# tables, which Elixir cannot execute. Guards MOB-160. +# credo:disable-for-this-file Jump.CredoChecks.VacuousTest +defmodule Mob.InputNifSchedulingTest do + use ExUnit.Case, async: true + + @root Path.expand("../..", __DIR__) + + # Every one of these blocks a scheduler waiting on the platform UI thread. + # See decisions/2026-09-05-input-nifs-are-dirty-io.md. + # `tap_by_label` is deliberately absent: it appears in the capabilities + # atom list but has no registration entry on either platform. + @blocking_input_nifs ~w( + tap tap_xy long_press_xy swipe_xy + type_text delete_backward clear_text + ) + + # iOS-only, and the asymmetry is real rather than an oversight: iOS's + # nif_key_press dispatch_syncs to the main queue, while Android's is a + # hardcoded :not_implemented stub that never touches the UI thread. iOS also + # blocks in its accessibility lookups — nif_ax_action_at_xy sleeps up to + # 4 x 50ms retrying — where Android has no AX path at all. + @ios_only_blocking ~w(key_press ax_action ax_action_at_xy) + + describe "Android" do + setup do + %{source: File.read!(Path.join(@root, "android/jni/mob_nif.zig"))} + end + + test "every blocking input NIF is registered dirty IO-bound", %{source: source} do + for name <- @blocking_input_nifs do + entry = registration(source, ~r/\.\{ \.name = "#{name}", .*?\}/s) + + assert entry =~ "ERL_NIF_DIRTY_JOB_IO_BOUND", + """ + #{name} is registered on a normal scheduler. + + Android's default argv is `-S 1:1` — one normal scheduler — so a + NIF that waits on the main thread stops every process on the + device for as long as it waits. A long press holds for its full + duration by design. + + #{entry} + """ + end + end + + test "the app's own default argv still has exactly one normal scheduler" do + # If this ever changes, the reasoning above gets weaker, not wrong — + # but the decision record should be revisited rather than silently drift. + # Matched loosely: the spacing is `zig fmt` column alignment computed + # from the longest key in that struct, so renaming a neighbouring flag + # would otherwise fail this test with a message about scheduler counts + # that had not changed. + beam = File.read!(Path.join(@root, "android/jni/mob_beam.zig")) + assert beam =~ ~r/"-S",\s+"1:1"/ + end + + test "key_press is NOT dirty, because Android's is a stub" do + # Guards the inverse claim. A dirty scheduler hop to return an atom is + # pure overhead, and flagging it would make the rule above ("it is dirty + # because it waits") false of one of its own members. + source = File.read!(Path.join(@root, "android/jni/mob_nif.zig")) + entry = registration(source, ~r/\.\{ \.name = "key_press", .*?\}/s) + + assert entry =~ ".flags = 0", + "Android's nif_key_press returns :not_implemented without touching " <> + "the UI thread; it has nothing to wait for.\n\n #{entry}" + end + end + + describe "iOS" do + setup do + %{source: File.read!(Path.join(@root, "ios/mob_nif.m"))} + end + + test "every blocking input NIF is registered dirty IO-bound", %{source: source} do + for name <- @blocking_input_nifs ++ @ios_only_blocking do + entry = registration(source, ~r/\{"#{name}", \d+, nif_\w+, [^}]*\}/) + + assert entry =~ "ERL_NIF_DIRTY_JOB_IO_BOUND", + """ + #{name} is registered on a normal scheduler. + + nif_long_press_xy sleeps for the caller's full duration via + [NSThread sleepForTimeInterval:], and the rest dispatch_sync to + the main queue. + + #{entry} + """ + end + end + end + + # Matches the registration entry, not merely the name: the name also appears + # in the capabilities list and in doc comments, and asserting against those + # would pass no matter what the flags said. + defp registration(source, regex) do + case Regex.run(regex, source) do + [entry] -> entry + nil -> flunk("no NIF registration entry matched #{inspect(regex.source)}") + end + end +end