Uh oh!
There was an error while loading. Please reload this page.
fix(cu): harden merged executor boundaries - #910
Conversation
94a0305 to
158474fComparehqhq1025
commented
Jul 13, 2026
@Astro-Han Updated on current main after #895/#896/#897. CI typecheck/test/e2e are green. This PR now also absorbs the two actionable executor P2s from #905: verified Electron CDP semantic click/text remains reachable while CGEvent fallback stays disabled, and native semantic AX dispatch emits a privacy-safe trace for restart oracles. Recommended merge queue: #910, #911, then #913. |
hqhq1025
commented
Jul 13, 2026
@astrohan Real-model AX integration found a remaining semantic/refetch boundary on current #910 tip. Scenario: real gpt-5.6-sol observes one |
Astro-Han
left a comment
There was a problem hiding this comment.
Approved on dae38967. I found no P0 or P1. CI is green, the full workspace rebuild and stale check passed locally, and the Computer Use suite passed 119/119. The findings below are non-blocking.
P2
The window fingerprint now includes every AX label and value (
packages/computer-use/src/cua-driver-backend.ts:350-364). An unrelated clock, progress label, or status counter changing between observation and action changes the whole fingerprint, so a bound coordinate action or Electron semantic pointer action returnstarget_changedeven when its target is unchanged. The tests replace the production fingerprint with a constant and do not cover this case.Several failures after delivery still lose the delivered state. Native and Electron text writes return
capture_failedwhen readback cannot confirm the write (cua-driver-backend.ts:1551-1563,:1652-1668), and a successful native semantic action can becomecapture_failedorsensitivity_blockedwhen its fresh screenshot fails (:1965-1980). These should preserveoutcome_unknownso the model is not encouraged to repeat a write or click that may already have happened.Keyboard ownership validates only the turn and PID/window ID (
cua-driver-backend.ts:1468-1482). If that window moves or its fields reflow after the click,typereuses the old screen point and can select a different empty field in the same window. A successful readback then reports success for the wrong field.When the original observation could not establish Electron page identity, a later bound action can resolve a current CDP target and dispatch without a document identity check (
cua-driver-backend.ts:929-939,:1716-1726,:2075-2101). A reload or navigation to a structurally identical page can therefore operate on a different document.
P3
observeResolvedWindow()stores the observation before screenshot normalization. A missing or oversized screenshot throws without returning the ID, but the unreachable observation still consumes one of the 16 FIFO slots and can evict an earlier usable observation.The model-facing tool description in
packages/runtime/src/computer-use-tools.ts:1009-1011still says Electron text targets are refused, although this change keeps the CDP click and text path reachable. Models following the description will not use the new path.
Uh oh!
There was an error while loading. Please reload this page.
Summary
Post-merge hardening for the actionable findings recorded on #893 and the remaining executor findings from #905 review.
outcome_unknownafter delivered but unverifiable actionsDeliberate boundaries
See
docs/computer-use-executor-hardening.mdfor the finding-by-finding rationale and remaining interface limitation.Verification
npm --workspace @maka/core run buildnpm --workspace @maka/runtime run buildnpm --workspace @maka/computer-use run typecheckgit diff --checkFollow-up to #893. Supersedes the two actionable P2 executor findings on #905.