test(cu): extend real AX provider evidence

[hqhq1025]

Summary

Extends merged provider-evidence #913 with an external AppKit AX runner and Anthropic-compatible provider-loop coverage.

The existing #913 Desktop runner remains the canonical OpenAI L0/L1 evidence path. This follow-up adds hazards and providers that runner does not cover yet.

Real AppKit product path

The opt-in harness uses:

live provider
-> getAIModel
-> AiSdkBackend / streamText
-> ToolRuntime
-> maka_computer
-> pinned cua-driver
-> owned Codex CUA Lab.app AX tree/actions

It launches the fixture without explicit activation, owns caffeinate/cleanup, monitors lock and fixture focus, and uses a read-only physical-input age probe. Scenario allowlists exclude coordinate, scroll, drag, type, key, and pixel fallback.

Qualifying real runs

OpenAI gpt-5.6-sol through the local Azure Responses bridge:

• observe-only: zero dispatch;
• AX set_value: one AX dispatch, fresh AX value matched, zero pixel dispatch;
• user intervention recovery: first mutation returned reobserve_required, then observe + success;
• process restart recovery: old action returned target_missing, model rediscovered the new PID/window and succeeded;
• AX click: one click_element AX dispatch and external AppKit count 1;
• multi-step semantic task: set_value then click_element using the fresh observation chain, one dispatch each and zero pixel dispatch.

Anthropic claude-sonnet-4-6 through coproxy:

• AX set_value through the same production Runtime path;
• user intervention recovery;
• multi-step set_value then click_element;
• one AX dispatch per mutation and zero pixel dispatch.

Dynamic structure evidence

After the model observed one stale target, an independent AX setup action inserted a duplicate semantic target. The current executor conservatively returned target_changed before identity refetch, with zero model dispatch and zero target mutation. This remains safe; the semantic-refetch ordering finding was reported on #910.

Other providers

Kimi Coding Plan and MiniMax Coding Plan complete four-stage product loops through their exact Anthropic-compatible streaming URL/auth contracts:

list_apps -> observe -> set_value -> final response

These are hermetic-protocol evidence because no live credentials are configured on this Mac; they are not labeled real-runtime.

Evidence/privacy fixes

• sanitized direct reports preserve explicit producer/provider identity instead of hard-coding OpenAI;
• reports retain action types, result codes, latency, aggregate state, and allowlisted traces;
• reports omit prompts, screenshots, coordinates, typed values, AX/UI text, credentials, and raw provider payloads;
• setup-only AX traces are excluded from model-dispatch evidence.

Verification

• scripts: 60/60;
• Runtime provider protocol: Kimi/MiniMax 2/2;
• Computer Use: 124/124;
• full repository build and typecheck pass;
• all listed OpenAI and Anthropic real runs completed locally;
• git diff --check passes.

[hqhq1025]

Ready for review. CI is green and this PR adds real AppKit AX evidence with zero pixel dispatch for OpenAI and Anthropic product Runtime runs. cc @Astro-Han

[Astro-Han]

Approved. I did not find a P0 or P1. These findings are non-blocking for this opt-in real-model harness:

• P2 — scripts/cu-real-ax-model-e2e.mjs:342-376,475-501: the runner keeps only each event type, accepts any final complete, and then hard-codes loopStatus: "completed". A run that reaches step_limit or error after the expected mutation can therefore be published as completed. Keep the final stop reason and require end_turn.

• P2 — scripts/cu-real-ax-model-e2e.mjs:240-283,398-457: the multi-step check independently counts one set_value and one click_element, but it does not prove their order or that the click used the observation returned by set_value. Record request/result observation IDs and verify the chain.

• P2 — scripts/cu-real-ax-model-e2e.mjs:431-450: the ambiguity scenario passes when the model observes once and stops without attempting click_element. That produces an ambiguity evidence report without exercising either target_changed or ambiguous stale_frame; treat a safe decline as inconclusive or require an actual rejection outcome.

• P2 — scripts/cu-physical-input-age.swift:4-13: the input-age probe omits .scrollWheel. Trackpad or wheel-only activity can therefore look idle and allow real AX dispatch while the user is actively scrolling.

• P3 — scripts/cu-real-ax-model-e2e.mjs:190-203 and docs/computer-use-real-ax-model-e2e.md:42-47: intervention recovery is injected by the wrapper before the real backend and input probe run. The result is useful Runtime fault-injection evidence, but calling it physical-intervention evidence overstates what was tested.

• P3 — packages/runtime/src/__tests__/computer-use-provider-protocol.test.ts:44-123: the protocol fixtures do not assert body.model, the final set_value tool-result reinjection, or the collected toolResults[].isError, despite the documentation claiming those checks.

• P3 — scripts/cu-real-ax-model-e2e.mjs:252-265,404-423: per-scenario mutation counts are checked only after execution. A model can dispatch repeated allowed mutations before the report fails; enforce the stated exactly-once limits before dispatch.

• P3 — scripts/cu-real-ax-model-e2e-launcher.mjs:141-149: READY parsing reads field 4 as bundleIdentifier, while the monitor emits input age in field 4 and the bundle ID in field 5. The field is currently unused, but the recorded monitor identity is wrong.