Skip to content

Run the input NIFs on a dirty scheduler instead of stalling the device - #135

Merged
GenericJam merged 1 commit into
masterfrom
fix/mob-160-input-nif-scheduling
Sep 6, 2026
Merged

Run the input NIFs on a dirty scheduler instead of stalling the device#135
GenericJam merged 1 commit into
masterfrom
fix/mob-160-input-nif-scheduling

Conversation

@GenericJam

Copy link
Copy Markdown
Owner

Android's default BEAM argv is -S 1:1one normal scheduler (android/jni/mob_beam.zig:732). Every harness input NIF blocks it waiting on the platform UI thread, and all but one were registered .flags = 0.

GenericJam/mob_new#56 made this acute: Android gestures now hold the pointer for their real duration, because the platform's detectors wait on posted callbacks and ignore synthesised timestamps. long_press_xy(node, x, y, 800) blocks for 800ms — roughly 800x the budget a NIF is supposed to take. For that window nothing on the device runs: no timers, no renders, no :rpc, no PubSub. An agent driving the UI would be pausing the app it is trying to observe.

iOS was already there and nobody had noticed. On the device branch nif_long_press_xy sleeps the caller's full duration via [NSThread sleepForTimeInterval:], and nif_ax_action_at_xy sleeps up to 4 × 50ms retrying its accessibility lookup — 150ms on the only scheduler, sitting behind Mob.Test.toggle/2, dismiss_alert/2 and adjust_slider/4.

The change

tap, tap_xy, long_press_xy, swipe_xy, type_text, delete_backward, clear_textERL_NIF_DIRTY_JOB_IO_BOUND on both platforms. Plus key_press, ax_action, ax_action_at_xy on iOS.

IO_BOUND rather than CPU_BOUND: they are waiting on another thread, not computing.

The flags diverge by platform because the code does. Android's key_press is a hardcoded :not_implemented stub that never touches the UI thread, so a dirty hop would buy nothing; Android has no accessibility path at all.

This reverses the note above nif_funcs[], which held these on regular schedulers because "the harness calls them in tight loops and dirty-dispatch overhead would add up", pending benchmarks that were never run. That trade is the wrong way round: the cost is a thread wakeup per call, and the thing traded away is the whole VM. The note is rewritten rather than left contradicting the table ten lines below it.

Safety checked empirically, not inferred

Registering dirty flags on an ERTS built without dirty scheduler support fails module load and bricks every app at boot — a failure this project has hit before. Rather than reason about build flags, I asked the running device:

dirty_io_schedulers: 1
dirty_cpu_schedulers: 1
otp_release: "29"

Support is compiled in, and master already ships dirty NIFs on the boot DNS path (resolve_ipv4).

Documentation — the user-facing half, previously unshipped

  • Platform matrix covers long_press_xy/4, type_text/2, delete_backward/1, clear_text/1, and marks tap_xy/3 and swipe/5 working on Android for apps generated by mob_new 0.4.32+ — the methods live in the app's own generated bridge, so the ✅ is a property of the app, not of mob. The table cannot know that; capabilities/1 can.
  • What Android synthetic input costs: gestures block for their real duration, only the activity's own window is reachable (not dialogs or modal sheets), type_text is ASCII-only, clear_text deliberately absent.
  • scroll_info/2 returns device pixels while element_frames/1 returns dp and tap_xy/swipe take dp. Feeding one to the other overshoots by the display density — 2.75x on a moto g power. Newly reachable now that Android can swipe.
  • The footnote no longer blames the IOHID injection path for type_text, delete_backward and clear_text, which do not use it.

Trade-off worth knowing

There is exactly one dirty IO scheduler (-SDio 1), so this converts a total stall into head-of-line blocking on a resource of size one. An 800ms long press now delays other IO-dirty work — resolve_ipv4, audio_output_level, vendor_usb_bulk_write on Android; safe_area on iOS. Strictly better than what it replaces, and if it bites the answer is to raise -SDio, not to go back. Recorded in the decision's Consequences.

Tests

Asserts registration flags on both platforms, and asserts the inverse for Android's key_press so the "it is dirty because it waits" rule stays true of its own members. Mutation-checked: flipping a flag back to 0 fails it.

Suite: 1543 passed, credo clean.

⚠️Rebuild native to pick this up — the registration table compiles into each app.

Pairs with GenericJam/mob_new#56.

🤖 Generated with Claude Code

Android's default BEAM argv is `-S 1:1` — one normal scheduler. Every harness
input NIF blocks it waiting on the platform UI thread, and all but one were
registered with `.flags = 0`.
MOB-160 made this acute: Android gestures now hold the pointer for their real
duration, because the platform's detectors wait on posted callbacks and ignore
synthesised timestamps. `long_press_xy(node, x, y, 800)` blocks for 800ms,
roughly 800x the budget a NIF is supposed to take. For that window nothing on
the device runs — no timers, no renders, no :rpc, no PubSub. An agent driving
the UI would be pausing the app it is trying to observe.
iOS was already there and nobody had noticed. On the device branch
nif_long_press_xy sleeps the caller's full duration via
[NSThread sleepForTimeInterval:], and nif_ax_action_at_xy sleeps up to 4 x 50ms
retrying its accessibility lookup — 150ms on the only scheduler, behind
Mob.Test.toggle/2, dismiss_alert/2 and adjust_slider/4.
So: tap, tap_xy, long_press_xy, swipe_xy, type_text, delete_backward and
clear_text go IO_BOUND on both platforms, plus key_press, ax_action and
ax_action_at_xy on iOS. IO_BOUND rather than CPU_BOUND because they are waiting
on another thread, not computing.
The flags diverge by platform because the code does: Android's key_press is a
hardcoded :not_implemented stub that never touches the UI thread, so a dirty
hop would buy nothing, and Android has no accessibility path at all.
This reverses the note above nif_funcs[], which held these on regular
schedulers on the grounds that the harness calls them in tight loops and
dirty-dispatch overhead would add up, pending benchmarks that were never run.
That trade is the wrong way round: the cost is a thread wakeup per call, and
the thing traded away is the whole VM. The note is rewritten rather than left
to contradict the table ten lines below it.
Safety checked empirically rather than inferred: registering dirty flags on an
ERTS built without dirty scheduler support fails module load and bricks every
app at boot. The running device reports dirty_io_schedulers: 1 and
dirty_cpu_schedulers: 1 on OTP 29, and master already ships dirty NIFs on the
boot DNS path.
Also documents the user-facing half, which was unshipped:
- The platform matrix covers long_press_xy/4, type_text/2, delete_backward/1
and clear_text/1, and marks tap_xy/3 and swipe/5 working on Android — for
apps generated by mob_new 0.4.32+, since the methods live in the app's own
generated bridge. The table cannot know that; capabilities/1 can.
- What Android synthetic input costs: gestures block for their real duration,
only the activity's own window is reachable (not dialogs or modal sheets),
type_text is ASCII-only and rejects a whole string over one bad character,
clear_text is deliberately absent.
- scroll_info/2 returns device PIXELS while element_frames/1 returns dp and
tap_xy/swipe take dp. Feeding one to the other overshoots by the display
density — 2.75x on a moto g power. Newly reachable now that Android can
swipe.
- The ✱ footnote no longer blames the IOHID injection path for type_text,
delete_backward and clear_text, which do not use it.
Test asserts the registration flags on both platforms, and asserts the inverse
for Android's key_press so the "it is dirty because it waits" rule stays true
of its own members. Mutation-checked: flipping a flag back to 0 fails it.
Rebuild native to pick this up — the registration table compiles into each app.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@GenericJam
GenericJam merged commit 03f8eb3 into masterSep 6, 2026
4 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@GenericJam