test(cu): consolidate qualification evidence

[hqhq1025]

Summary

Consolidates Computer Use qualification evidence into one fail-closed contract.

• require explicit producer/model/provider/transport/policy/terminal provenance;
• independently collect fixture PID/window identity;
• require latest-observation target lineage and invalidate it after target loss;
• consume one exact dispatch trace per successful mutation;
• prevent screenshot actions from inventing observation lineage;
• preserve full model recovery text in provider error tool results;
• retain the five-round restart runner as a non-qualifying soak while keeping one qualification path.

Verification

• scripts: 92 pass
• Runtime: 1599 pass, 7 skip
• CI typecheck/test/e2e: pass

[Astro-Han]

Approving — findings confirmed but all non-blocking: they're confined to the manually-run real-model / real-macOS qualification scripts, which CI does not execute, and there's no product/runtime regression. Follow-ups worth addressing since they touch the qualification contract this PR delivers:

• [P2] restart-recovery has no passing path (scripts/cu-real-ax-model-e2e.mjs): actionBudgetFor grants set_value: 1, but the pass condition needs a stale set_value failing target_missing AND a later successful set_value (two attempts); the second is rejected by the budget check before dispatch, so e2e:computer-use-process-restart / -real-ax-restart can never pass. Fix: allow set_value: 2 for restart-recovery and authorize the stale target_missing attempt.
• [P2] Over-budget / disallowed attempts are erased from the qualification report: the budget and allowed-action throws run before actions.push (and before totalActionAttempts = nextTotal), so validateRealReport never sees them — an auditability gap versus the stated fail-closed goal (the action is still blocked, so not a safety issue). Fix: record the attempt before throwing.
• [P2] launcher and matrix disagree on the pass verdict: cu-real-model-launcher.mjs omits the expectedActionSequence / lineage checks that validateRealReport enforces, so it can exit 0 on a report the matrix rejects as invalid. Fix: share one verdict.

[hqhq1025]

@Astro-Han All three qualification follow-ups are fixed in 3c9a3d93: restart recovery has the required two-attempt budget, rejected attempts remain in the evidence report, and launcher/matrix now share the same verdict contract. The branch was rebased onto current main; scripts, runtime, desktop, lint/typecheck, and latest CI are green. Ready for merge.

[Astro-Han]

Both product changes are correct. ai-sdk-backend.ts preserves the full modelText recovery text with no truncation and only throws on outputs already classified as tool errors, so normal flow is unchanged. The wait/cursor_position ownership exemption in computer-use-real-model-policy.ts matches the runtime's own non-mutating classification (both take an observation lease, not an action lease), so no mutating action loses its owned-observation requirement. The consolidation keeps validateRealReport running on the sanitized report, and the provenance and screenshot-lineage axes come out stronger than before (the sanitizer strips a screenshot's resultObservationId at cu-report-sanitize.mjs:112, so a screenshot cannot set the latest observation lineage).

A cluster of non-blocking robustness and coverage notes:

• Matrix dispatch-trace check is fail-open on extras (cu-provider-matrix.mjs:336-359). It rejects zero dispatch and prevents one trace serving two mutations, but does not reject a duplicate/extra trace or a dispatch trace on a success:false action. The exact-count assertion survives only in the AX harness runtime, not the matrix path.
• target_missing invalidation is enforced but untested (cu-provider-matrix.mjs:401-404). The reset-on-target-loss branch has no unit exercising it; removing those lines would fail nothing. Behavioral coverage rests on the real-runtime restart scenario.
• Declared restart failure is not required to occur (cu-provider-matrix.mjs:372-378). expectedFailures is an allowlist, so a report with only successful mutations can satisfy the type sequence, budget, and final state without ever observing the set_value/target_missing the scenario is meant to force.
• Coordinate-attempt coverage weakened. The deleted real-e2e harness actively attempted a coordinate action and asserted zero dispatch and zero side effects; the consolidated default relies on the model following "never use coordinates" rather than deterministically attempting one. The compatibilityInputBlocked guard itself is still unit-tested in the backend, so this is reduced e2e coverage rather than a lost guard.
• Monitor baseline accepts malformed numeric fields (cu-real-ax-model-e2e-launcher.mjs:176-191). validateMonitorBaseline does not require frontmostPID to be a positive integer or the pointer x/y to be finite, so a NaN PID or pointer passes the READY check the test names "validates every READY field."
• Minor sanitizer inconsistency (cu-report-sanitize.mjs:11). toolCallId is allowlisted and passed verbatim in the trace path while sanitizeCuActionRecord runs it through safeId; tool-call IDs are SDK-generated so the risk is negligible, but the two paths could match. (The unbounded string actual in sanitizeAssertionResult predates this PR.)

[Astro-Han]

Approving. The core behaviors are verified correct; the notes posted are non-blocking (P2/P3 follow-ups).

[hqhq1025]

Rebased onto current main and addressed the additional adversarial qualification findings. Latest head 74fcca88 now records every rejected attempt in the canonical ledger, recomputes counts from that ledger, requires restart target_missing → fresh observation → successful AX retry, binds reports to run/commit/timestamp/generator lineage, preserves the report outside the temp directory, and supports relocated fixtures plus candidate-driver qualification only when path + SHA-256 + server version are all supplied. Validation: scripts 108/108; Runtime 1941 pass, 7 skip, 0 fail. New CI is running.

[hqhq1025]

Summary

Consolidates Computer Use qualification evidence into one fail-closed contract.

• require explicit producer/model/provider/transport/policy/terminal provenance;
• independently collect fixture PID/window identity;
• require latest-observation target lineage and invalidate it after target loss;
• consume one exact dispatch trace per successful mutation;
• prevent screenshot actions from inventing observation lineage;
• preserve full model recovery text in provider error tool results;
• retain the five-round restart runner as a non-qualifying soak while keeping one qualification path.

Verification

• scripts: 92 pass
• Runtime: 1599 pass, 7 skip
• CI typecheck/test/e2e: pass

[Astro-Han]

Approving — findings confirmed but all non-blocking: they're confined to the manually-run real-model / real-macOS qualification scripts, which CI does not execute, and there's no product/runtime regression. Follow-ups worth addressing since they touch the qualification contract this PR delivers:

• [P2] restart-recovery has no passing path (scripts/cu-real-ax-model-e2e.mjs): actionBudgetFor grants set_value: 1, but the pass condition needs a stale set_value failing target_missing AND a later successful set_value (two attempts); the second is rejected by the budget check before dispatch, so e2e:computer-use-process-restart / -real-ax-restart can never pass. Fix: allow set_value: 2 for restart-recovery and authorize the stale target_missing attempt.
• [P2] Over-budget / disallowed attempts are erased from the qualification report: the budget and allowed-action throws run before actions.push (and before totalActionAttempts = nextTotal), so validateRealReport never sees them — an auditability gap versus the stated fail-closed goal (the action is still blocked, so not a safety issue). Fix: record the attempt before throwing.
• [P2] launcher and matrix disagree on the pass verdict: cu-real-model-launcher.mjs omits the expectedActionSequence / lineage checks that validateRealReport enforces, so it can exit 0 on a report the matrix rejects as invalid. Fix: share one verdict.

[hqhq1025]

@Astro-Han All three qualification follow-ups are fixed in 3c9a3d93: restart recovery has the required two-attempt budget, rejected attempts remain in the evidence report, and launcher/matrix now share the same verdict contract. The branch was rebased onto current main; scripts, runtime, desktop, lint/typecheck, and latest CI are green. Ready for merge.

[Astro-Han]

Both product changes are correct. ai-sdk-backend.ts preserves the full modelText recovery text with no truncation and only throws on outputs already classified as tool errors, so normal flow is unchanged. The wait/cursor_position ownership exemption in computer-use-real-model-policy.ts matches the runtime's own non-mutating classification (both take an observation lease, not an action lease), so no mutating action loses its owned-observation requirement. The consolidation keeps validateRealReport running on the sanitized report, and the provenance and screenshot-lineage axes come out stronger than before (the sanitizer strips a screenshot's resultObservationId at cu-report-sanitize.mjs:112, so a screenshot cannot set the latest observation lineage).

A cluster of non-blocking robustness and coverage notes:

• Matrix dispatch-trace check is fail-open on extras (cu-provider-matrix.mjs:336-359). It rejects zero dispatch and prevents one trace serving two mutations, but does not reject a duplicate/extra trace or a dispatch trace on a success:false action. The exact-count assertion survives only in the AX harness runtime, not the matrix path.
• target_missing invalidation is enforced but untested (cu-provider-matrix.mjs:401-404). The reset-on-target-loss branch has no unit exercising it; removing those lines would fail nothing. Behavioral coverage rests on the real-runtime restart scenario.
• Declared restart failure is not required to occur (cu-provider-matrix.mjs:372-378). expectedFailures is an allowlist, so a report with only successful mutations can satisfy the type sequence, budget, and final state without ever observing the set_value/target_missing the scenario is meant to force.
• Coordinate-attempt coverage weakened. The deleted real-e2e harness actively attempted a coordinate action and asserted zero dispatch and zero side effects; the consolidated default relies on the model following "never use coordinates" rather than deterministically attempting one. The compatibilityInputBlocked guard itself is still unit-tested in the backend, so this is reduced e2e coverage rather than a lost guard.
• Monitor baseline accepts malformed numeric fields (cu-real-ax-model-e2e-launcher.mjs:176-191). validateMonitorBaseline does not require frontmostPID to be a positive integer or the pointer x/y to be finite, so a NaN PID or pointer passes the READY check the test names "validates every READY field."
• Minor sanitizer inconsistency (cu-report-sanitize.mjs:11). toolCallId is allowlisted and passed verbatim in the trace path while sanitizeCuActionRecord runs it through safeId; tool-call IDs are SDK-generated so the risk is negligible, but the two paths could match. (The unbounded string actual in sanitizeAssertionResult predates this PR.)

[Astro-Han]

Approving. The core behaviors are verified correct; the notes posted are non-blocking (P2/P3 follow-ups).

[hqhq1025]

Rebased onto current main and addressed the additional adversarial qualification findings. Latest head 74fcca88 now records every rejected attempt in the canonical ledger, recomputes counts from that ledger, requires restart target_missing → fresh observation → successful AX retry, binds reports to run/commit/timestamp/generator lineage, preserves the report outside the temp directory, and supports relocated fixtures plus candidate-driver qualification only when path + SHA-256 + server version are all supplied. Validation: scripts 108/108; Runtime 1941 pass, 7 skip, 0 fail. New CI is running.

[hqhq1025]

Summary

Consolidates Computer Use qualification evidence into one fail-closed contract.

• require explicit producer/model/provider/transport/policy/terminal provenance;
• independently collect fixture PID/window identity;
• require latest-observation target lineage and invalidate it after target loss;
• consume one exact dispatch trace per successful mutation;
• prevent screenshot actions from inventing observation lineage;
• preserve full model recovery text in provider error tool results;
• retain the five-round restart runner as a non-qualifying soak while keeping one qualification path.

Verification

• scripts: 92 pass
• Runtime: 1599 pass, 7 skip
• CI typecheck/test/e2e: pass

[Astro-Han]

Approving — findings confirmed but all non-blocking: they're confined to the manually-run real-model / real-macOS qualification scripts, which CI does not execute, and there's no product/runtime regression. Follow-ups worth addressing since they touch the qualification contract this PR delivers:

• [P2] restart-recovery has no passing path (scripts/cu-real-ax-model-e2e.mjs): actionBudgetFor grants set_value: 1, but the pass condition needs a stale set_value failing target_missing AND a later successful set_value (two attempts); the second is rejected by the budget check before dispatch, so e2e:computer-use-process-restart / -real-ax-restart can never pass. Fix: allow set_value: 2 for restart-recovery and authorize the stale target_missing attempt.
• [P2] Over-budget / disallowed attempts are erased from the qualification report: the budget and allowed-action throws run before actions.push (and before totalActionAttempts = nextTotal), so validateRealReport never sees them — an auditability gap versus the stated fail-closed goal (the action is still blocked, so not a safety issue). Fix: record the attempt before throwing.
• [P2] launcher and matrix disagree on the pass verdict: cu-real-model-launcher.mjs omits the expectedActionSequence / lineage checks that validateRealReport enforces, so it can exit 0 on a report the matrix rejects as invalid. Fix: share one verdict.

[hqhq1025]

@Astro-Han All three qualification follow-ups are fixed in 3c9a3d93: restart recovery has the required two-attempt budget, rejected attempts remain in the evidence report, and launcher/matrix now share the same verdict contract. The branch was rebased onto current main; scripts, runtime, desktop, lint/typecheck, and latest CI are green. Ready for merge.

[Astro-Han]

Both product changes are correct. ai-sdk-backend.ts preserves the full modelText recovery text with no truncation and only throws on outputs already classified as tool errors, so normal flow is unchanged. The wait/cursor_position ownership exemption in computer-use-real-model-policy.ts matches the runtime's own non-mutating classification (both take an observation lease, not an action lease), so no mutating action loses its owned-observation requirement. The consolidation keeps validateRealReport running on the sanitized report, and the provenance and screenshot-lineage axes come out stronger than before (the sanitizer strips a screenshot's resultObservationId at cu-report-sanitize.mjs:112, so a screenshot cannot set the latest observation lineage).

A cluster of non-blocking robustness and coverage notes:

• Matrix dispatch-trace check is fail-open on extras (cu-provider-matrix.mjs:336-359). It rejects zero dispatch and prevents one trace serving two mutations, but does not reject a duplicate/extra trace or a dispatch trace on a success:false action. The exact-count assertion survives only in the AX harness runtime, not the matrix path.
• target_missing invalidation is enforced but untested (cu-provider-matrix.mjs:401-404). The reset-on-target-loss branch has no unit exercising it; removing those lines would fail nothing. Behavioral coverage rests on the real-runtime restart scenario.
• Declared restart failure is not required to occur (cu-provider-matrix.mjs:372-378). expectedFailures is an allowlist, so a report with only successful mutations can satisfy the type sequence, budget, and final state without ever observing the set_value/target_missing the scenario is meant to force.
• Coordinate-attempt coverage weakened. The deleted real-e2e harness actively attempted a coordinate action and asserted zero dispatch and zero side effects; the consolidated default relies on the model following "never use coordinates" rather than deterministically attempting one. The compatibilityInputBlocked guard itself is still unit-tested in the backend, so this is reduced e2e coverage rather than a lost guard.
• Monitor baseline accepts malformed numeric fields (cu-real-ax-model-e2e-launcher.mjs:176-191). validateMonitorBaseline does not require frontmostPID to be a positive integer or the pointer x/y to be finite, so a NaN PID or pointer passes the READY check the test names "validates every READY field."
• Minor sanitizer inconsistency (cu-report-sanitize.mjs:11). toolCallId is allowlisted and passed verbatim in the trace path while sanitizeCuActionRecord runs it through safeId; tool-call IDs are SDK-generated so the risk is negligible, but the two paths could match. (The unbounded string actual in sanitizeAssertionResult predates this PR.)

[Astro-Han]

Approving. The core behaviors are verified correct; the notes posted are non-blocking (P2/P3 follow-ups).

[hqhq1025]

Rebased onto current main and addressed the additional adversarial qualification findings. Latest head 74fcca88 now records every rejected attempt in the canonical ledger, recomputes counts from that ledger, requires restart target_missing → fresh observation → successful AX retry, binds reports to run/commit/timestamp/generator lineage, preserves the report outside the temp directory, and supports relocated fixtures plus candidate-driver qualification only when path + SHA-256 + server version are all supplied. Validation: scripts 108/108; Runtime 1941 pass, 7 skip, 0 fail. New CI is running.

[hqhq1025]

Summary

Consolidates Computer Use qualification evidence into one fail-closed contract.

• require explicit producer/model/provider/transport/policy/terminal provenance;
• independently collect fixture PID/window identity;
• require latest-observation target lineage and invalidate it after target loss;
• consume one exact dispatch trace per successful mutation;
• prevent screenshot actions from inventing observation lineage;
• preserve full model recovery text in provider error tool results;
• retain the five-round restart runner as a non-qualifying soak while keeping one qualification path.

Verification

• scripts: 92 pass
• Runtime: 1599 pass, 7 skip
• CI typecheck/test/e2e: pass

[Astro-Han]

Approving — findings confirmed but all non-blocking: they're confined to the manually-run real-model / real-macOS qualification scripts, which CI does not execute, and there's no product/runtime regression. Follow-ups worth addressing since they touch the qualification contract this PR delivers:

• [P2] restart-recovery has no passing path (scripts/cu-real-ax-model-e2e.mjs): actionBudgetFor grants set_value: 1, but the pass condition needs a stale set_value failing target_missing AND a later successful set_value (two attempts); the second is rejected by the budget check before dispatch, so e2e:computer-use-process-restart / -real-ax-restart can never pass. Fix: allow set_value: 2 for restart-recovery and authorize the stale target_missing attempt.
• [P2] Over-budget / disallowed attempts are erased from the qualification report: the budget and allowed-action throws run before actions.push (and before totalActionAttempts = nextTotal), so validateRealReport never sees them — an auditability gap versus the stated fail-closed goal (the action is still blocked, so not a safety issue). Fix: record the attempt before throwing.
• [P2] launcher and matrix disagree on the pass verdict: cu-real-model-launcher.mjs omits the expectedActionSequence / lineage checks that validateRealReport enforces, so it can exit 0 on a report the matrix rejects as invalid. Fix: share one verdict.

[hqhq1025]

@Astro-Han All three qualification follow-ups are fixed in 3c9a3d93: restart recovery has the required two-attempt budget, rejected attempts remain in the evidence report, and launcher/matrix now share the same verdict contract. The branch was rebased onto current main; scripts, runtime, desktop, lint/typecheck, and latest CI are green. Ready for merge.

[Astro-Han]

Both product changes are correct. ai-sdk-backend.ts preserves the full modelText recovery text with no truncation and only throws on outputs already classified as tool errors, so normal flow is unchanged. The wait/cursor_position ownership exemption in computer-use-real-model-policy.ts matches the runtime's own non-mutating classification (both take an observation lease, not an action lease), so no mutating action loses its owned-observation requirement. The consolidation keeps validateRealReport running on the sanitized report, and the provenance and screenshot-lineage axes come out stronger than before (the sanitizer strips a screenshot's resultObservationId at cu-report-sanitize.mjs:112, so a screenshot cannot set the latest observation lineage).

A cluster of non-blocking robustness and coverage notes:

• Matrix dispatch-trace check is fail-open on extras (cu-provider-matrix.mjs:336-359). It rejects zero dispatch and prevents one trace serving two mutations, but does not reject a duplicate/extra trace or a dispatch trace on a success:false action. The exact-count assertion survives only in the AX harness runtime, not the matrix path.
• target_missing invalidation is enforced but untested (cu-provider-matrix.mjs:401-404). The reset-on-target-loss branch has no unit exercising it; removing those lines would fail nothing. Behavioral coverage rests on the real-runtime restart scenario.
• Declared restart failure is not required to occur (cu-provider-matrix.mjs:372-378). expectedFailures is an allowlist, so a report with only successful mutations can satisfy the type sequence, budget, and final state without ever observing the set_value/target_missing the scenario is meant to force.
• Coordinate-attempt coverage weakened. The deleted real-e2e harness actively attempted a coordinate action and asserted zero dispatch and zero side effects; the consolidated default relies on the model following "never use coordinates" rather than deterministically attempting one. The compatibilityInputBlocked guard itself is still unit-tested in the backend, so this is reduced e2e coverage rather than a lost guard.
• Monitor baseline accepts malformed numeric fields (cu-real-ax-model-e2e-launcher.mjs:176-191). validateMonitorBaseline does not require frontmostPID to be a positive integer or the pointer x/y to be finite, so a NaN PID or pointer passes the READY check the test names "validates every READY field."
• Minor sanitizer inconsistency (cu-report-sanitize.mjs:11). toolCallId is allowlisted and passed verbatim in the trace path while sanitizeCuActionRecord runs it through safeId; tool-call IDs are SDK-generated so the risk is negligible, but the two paths could match. (The unbounded string actual in sanitizeAssertionResult predates this PR.)

[Astro-Han]

Approving. The core behaviors are verified correct; the notes posted are non-blocking (P2/P3 follow-ups).

[hqhq1025]

Rebased onto current main and addressed the additional adversarial qualification findings. Latest head 74fcca88 now records every rejected attempt in the canonical ledger, recomputes counts from that ledger, requires restart target_missing → fresh observation → successful AX retry, binds reports to run/commit/timestamp/generator lineage, preserves the report outside the temp directory, and supports relocated fixtures plus candidate-driver qualification only when path + SHA-256 + server version are all supplied. Validation: scripts 108/108; Runtime 1941 pass, 7 skip, 0 fail. New CI is running.

[hqhq1025]

Summary

Consolidates Computer Use qualification evidence into one fail-closed contract.

• require explicit producer/model/provider/transport/policy/terminal provenance;
• independently collect fixture PID/window identity;
• require latest-observation target lineage and invalidate it after target loss;
• consume one exact dispatch trace per successful mutation;
• prevent screenshot actions from inventing observation lineage;
• preserve full model recovery text in provider error tool results;
• retain the five-round restart runner as a non-qualifying soak while keeping one qualification path.

Verification

• scripts: 92 pass
• Runtime: 1599 pass, 7 skip
• CI typecheck/test/e2e: pass

[Astro-Han]

Approving — findings confirmed but all non-blocking: they're confined to the manually-run real-model / real-macOS qualification scripts, which CI does not execute, and there's no product/runtime regression. Follow-ups worth addressing since they touch the qualification contract this PR delivers:

• [P2] restart-recovery has no passing path (scripts/cu-real-ax-model-e2e.mjs): actionBudgetFor grants set_value: 1, but the pass condition needs a stale set_value failing target_missing AND a later successful set_value (two attempts); the second is rejected by the budget check before dispatch, so e2e:computer-use-process-restart / -real-ax-restart can never pass. Fix: allow set_value: 2 for restart-recovery and authorize the stale target_missing attempt.
• [P2] Over-budget / disallowed attempts are erased from the qualification report: the budget and allowed-action throws run before actions.push (and before totalActionAttempts = nextTotal), so validateRealReport never sees them — an auditability gap versus the stated fail-closed goal (the action is still blocked, so not a safety issue). Fix: record the attempt before throwing.
• [P2] launcher and matrix disagree on the pass verdict: cu-real-model-launcher.mjs omits the expectedActionSequence / lineage checks that validateRealReport enforces, so it can exit 0 on a report the matrix rejects as invalid. Fix: share one verdict.

[hqhq1025]

@Astro-Han All three qualification follow-ups are fixed in 3c9a3d93: restart recovery has the required two-attempt budget, rejected attempts remain in the evidence report, and launcher/matrix now share the same verdict contract. The branch was rebased onto current main; scripts, runtime, desktop, lint/typecheck, and latest CI are green. Ready for merge.

[Astro-Han]

Both product changes are correct. ai-sdk-backend.ts preserves the full modelText recovery text with no truncation and only throws on outputs already classified as tool errors, so normal flow is unchanged. The wait/cursor_position ownership exemption in computer-use-real-model-policy.ts matches the runtime's own non-mutating classification (both take an observation lease, not an action lease), so no mutating action loses its owned-observation requirement. The consolidation keeps validateRealReport running on the sanitized report, and the provenance and screenshot-lineage axes come out stronger than before (the sanitizer strips a screenshot's resultObservationId at cu-report-sanitize.mjs:112, so a screenshot cannot set the latest observation lineage).

A cluster of non-blocking robustness and coverage notes:

• Matrix dispatch-trace check is fail-open on extras (cu-provider-matrix.mjs:336-359). It rejects zero dispatch and prevents one trace serving two mutations, but does not reject a duplicate/extra trace or a dispatch trace on a success:false action. The exact-count assertion survives only in the AX harness runtime, not the matrix path.
• target_missing invalidation is enforced but untested (cu-provider-matrix.mjs:401-404). The reset-on-target-loss branch has no unit exercising it; removing those lines would fail nothing. Behavioral coverage rests on the real-runtime restart scenario.
• Declared restart failure is not required to occur (cu-provider-matrix.mjs:372-378). expectedFailures is an allowlist, so a report with only successful mutations can satisfy the type sequence, budget, and final state without ever observing the set_value/target_missing the scenario is meant to force.
• Coordinate-attempt coverage weakened. The deleted real-e2e harness actively attempted a coordinate action and asserted zero dispatch and zero side effects; the consolidated default relies on the model following "never use coordinates" rather than deterministically attempting one. The compatibilityInputBlocked guard itself is still unit-tested in the backend, so this is reduced e2e coverage rather than a lost guard.
• Monitor baseline accepts malformed numeric fields (cu-real-ax-model-e2e-launcher.mjs:176-191). validateMonitorBaseline does not require frontmostPID to be a positive integer or the pointer x/y to be finite, so a NaN PID or pointer passes the READY check the test names "validates every READY field."
• Minor sanitizer inconsistency (cu-report-sanitize.mjs:11). toolCallId is allowlisted and passed verbatim in the trace path while sanitizeCuActionRecord runs it through safeId; tool-call IDs are SDK-generated so the risk is negligible, but the two paths could match. (The unbounded string actual in sanitizeAssertionResult predates this PR.)

[Astro-Han]

Approving. The core behaviors are verified correct; the notes posted are non-blocking (P2/P3 follow-ups).

[hqhq1025]

Rebased onto current main and addressed the additional adversarial qualification findings. Latest head 74fcca88 now records every rejected attempt in the canonical ledger, recomputes counts from that ledger, requires restart target_missing → fresh observation → successful AX retry, binds reports to run/commit/timestamp/generator lineage, preserves the report outside the temp directory, and supports relocated fixtures plus candidate-driver qualification only when path + SHA-256 + server version are all supplied. Validation: scripts 108/108; Runtime 1941 pass, 7 skip, 0 fail. New CI is running.

[hqhq1025]

Summary

Consolidates Computer Use qualification evidence into one fail-closed contract.

• require explicit producer/model/provider/transport/policy/terminal provenance;
• independently collect fixture PID/window identity;
• require latest-observation target lineage and invalidate it after target loss;
• consume one exact dispatch trace per successful mutation;
• prevent screenshot actions from inventing observation lineage;
• preserve full model recovery text in provider error tool results;
• retain the five-round restart runner as a non-qualifying soak while keeping one qualification path.

Verification

• scripts: 92 pass
• Runtime: 1599 pass, 7 skip
• CI typecheck/test/e2e: pass

[Astro-Han]

Approving — findings confirmed but all non-blocking: they're confined to the manually-run real-model / real-macOS qualification scripts, which CI does not execute, and there's no product/runtime regression. Follow-ups worth addressing since they touch the qualification contract this PR delivers:

• [P2] restart-recovery has no passing path (scripts/cu-real-ax-model-e2e.mjs): actionBudgetFor grants set_value: 1, but the pass condition needs a stale set_value failing target_missing AND a later successful set_value (two attempts); the second is rejected by the budget check before dispatch, so e2e:computer-use-process-restart / -real-ax-restart can never pass. Fix: allow set_value: 2 for restart-recovery and authorize the stale target_missing attempt.
• [P2] Over-budget / disallowed attempts are erased from the qualification report: the budget and allowed-action throws run before actions.push (and before totalActionAttempts = nextTotal), so validateRealReport never sees them — an auditability gap versus the stated fail-closed goal (the action is still blocked, so not a safety issue). Fix: record the attempt before throwing.
• [P2] launcher and matrix disagree on the pass verdict: cu-real-model-launcher.mjs omits the expectedActionSequence / lineage checks that validateRealReport enforces, so it can exit 0 on a report the matrix rejects as invalid. Fix: share one verdict.

[hqhq1025]

@Astro-Han All three qualification follow-ups are fixed in 3c9a3d93: restart recovery has the required two-attempt budget, rejected attempts remain in the evidence report, and launcher/matrix now share the same verdict contract. The branch was rebased onto current main; scripts, runtime, desktop, lint/typecheck, and latest CI are green. Ready for merge.

[Astro-Han]

Both product changes are correct. ai-sdk-backend.ts preserves the full modelText recovery text with no truncation and only throws on outputs already classified as tool errors, so normal flow is unchanged. The wait/cursor_position ownership exemption in computer-use-real-model-policy.ts matches the runtime's own non-mutating classification (both take an observation lease, not an action lease), so no mutating action loses its owned-observation requirement. The consolidation keeps validateRealReport running on the sanitized report, and the provenance and screenshot-lineage axes come out stronger than before (the sanitizer strips a screenshot's resultObservationId at cu-report-sanitize.mjs:112, so a screenshot cannot set the latest observation lineage).

A cluster of non-blocking robustness and coverage notes:

• Matrix dispatch-trace check is fail-open on extras (cu-provider-matrix.mjs:336-359). It rejects zero dispatch and prevents one trace serving two mutations, but does not reject a duplicate/extra trace or a dispatch trace on a success:false action. The exact-count assertion survives only in the AX harness runtime, not the matrix path.
• target_missing invalidation is enforced but untested (cu-provider-matrix.mjs:401-404). The reset-on-target-loss branch has no unit exercising it; removing those lines would fail nothing. Behavioral coverage rests on the real-runtime restart scenario.
• Declared restart failure is not required to occur (cu-provider-matrix.mjs:372-378). expectedFailures is an allowlist, so a report with only successful mutations can satisfy the type sequence, budget, and final state without ever observing the set_value/target_missing the scenario is meant to force.
• Coordinate-attempt coverage weakened. The deleted real-e2e harness actively attempted a coordinate action and asserted zero dispatch and zero side effects; the consolidated default relies on the model following "never use coordinates" rather than deterministically attempting one. The compatibilityInputBlocked guard itself is still unit-tested in the backend, so this is reduced e2e coverage rather than a lost guard.
• Monitor baseline accepts malformed numeric fields (cu-real-ax-model-e2e-launcher.mjs:176-191). validateMonitorBaseline does not require frontmostPID to be a positive integer or the pointer x/y to be finite, so a NaN PID or pointer passes the READY check the test names "validates every READY field."
• Minor sanitizer inconsistency (cu-report-sanitize.mjs:11). toolCallId is allowlisted and passed verbatim in the trace path while sanitizeCuActionRecord runs it through safeId; tool-call IDs are SDK-generated so the risk is negligible, but the two paths could match. (The unbounded string actual in sanitizeAssertionResult predates this PR.)

[Astro-Han]

Approving. The core behaviors are verified correct; the notes posted are non-blocking (P2/P3 follow-ups).

[hqhq1025]

Rebased onto current main and addressed the additional adversarial qualification findings. Latest head 74fcca88 now records every rejected attempt in the canonical ledger, recomputes counts from that ledger, requires restart target_missing → fresh observation → successful AX retry, binds reports to run/commit/timestamp/generator lineage, preserves the report outside the temp directory, and supports relocated fixtures plus candidate-driver qualification only when path + SHA-256 + server version are all supplied. Validation: scripts 108/108; Runtime 1941 pass, 7 skip, 0 fail. New CI is running.

[hqhq1025]

Summary

Consolidates Computer Use qualification evidence into one fail-closed contract.

• require explicit producer/model/provider/transport/policy/terminal provenance;
• independently collect fixture PID/window identity;
• require latest-observation target lineage and invalidate it after target loss;
• consume one exact dispatch trace per successful mutation;
• prevent screenshot actions from inventing observation lineage;
• preserve full model recovery text in provider error tool results;
• retain the five-round restart runner as a non-qualifying soak while keeping one qualification path.

Verification

• scripts: 92 pass
• Runtime: 1599 pass, 7 skip
• CI typecheck/test/e2e: pass

[Astro-Han]

Approving — findings confirmed but all non-blocking: they're confined to the manually-run real-model / real-macOS qualification scripts, which CI does not execute, and there's no product/runtime regression. Follow-ups worth addressing since they touch the qualification contract this PR delivers:

• [P2] restart-recovery has no passing path (scripts/cu-real-ax-model-e2e.mjs): actionBudgetFor grants set_value: 1, but the pass condition needs a stale set_value failing target_missing AND a later successful set_value (two attempts); the second is rejected by the budget check before dispatch, so e2e:computer-use-process-restart / -real-ax-restart can never pass. Fix: allow set_value: 2 for restart-recovery and authorize the stale target_missing attempt.
• [P2] Over-budget / disallowed attempts are erased from the qualification report: the budget and allowed-action throws run before actions.push (and before totalActionAttempts = nextTotal), so validateRealReport never sees them — an auditability gap versus the stated fail-closed goal (the action is still blocked, so not a safety issue). Fix: record the attempt before throwing.
• [P2] launcher and matrix disagree on the pass verdict: cu-real-model-launcher.mjs omits the expectedActionSequence / lineage checks that validateRealReport enforces, so it can exit 0 on a report the matrix rejects as invalid. Fix: share one verdict.

[hqhq1025]

@Astro-Han All three qualification follow-ups are fixed in 3c9a3d93: restart recovery has the required two-attempt budget, rejected attempts remain in the evidence report, and launcher/matrix now share the same verdict contract. The branch was rebased onto current main; scripts, runtime, desktop, lint/typecheck, and latest CI are green. Ready for merge.

[Astro-Han]

Both product changes are correct. ai-sdk-backend.ts preserves the full modelText recovery text with no truncation and only throws on outputs already classified as tool errors, so normal flow is unchanged. The wait/cursor_position ownership exemption in computer-use-real-model-policy.ts matches the runtime's own non-mutating classification (both take an observation lease, not an action lease), so no mutating action loses its owned-observation requirement. The consolidation keeps validateRealReport running on the sanitized report, and the provenance and screenshot-lineage axes come out stronger than before (the sanitizer strips a screenshot's resultObservationId at cu-report-sanitize.mjs:112, so a screenshot cannot set the latest observation lineage).

A cluster of non-blocking robustness and coverage notes:

• Matrix dispatch-trace check is fail-open on extras (cu-provider-matrix.mjs:336-359). It rejects zero dispatch and prevents one trace serving two mutations, but does not reject a duplicate/extra trace or a dispatch trace on a success:false action. The exact-count assertion survives only in the AX harness runtime, not the matrix path.
• target_missing invalidation is enforced but untested (cu-provider-matrix.mjs:401-404). The reset-on-target-loss branch has no unit exercising it; removing those lines would fail nothing. Behavioral coverage rests on the real-runtime restart scenario.
• Declared restart failure is not required to occur (cu-provider-matrix.mjs:372-378). expectedFailures is an allowlist, so a report with only successful mutations can satisfy the type sequence, budget, and final state without ever observing the set_value/target_missing the scenario is meant to force.
• Coordinate-attempt coverage weakened. The deleted real-e2e harness actively attempted a coordinate action and asserted zero dispatch and zero side effects; the consolidated default relies on the model following "never use coordinates" rather than deterministically attempting one. The compatibilityInputBlocked guard itself is still unit-tested in the backend, so this is reduced e2e coverage rather than a lost guard.
• Monitor baseline accepts malformed numeric fields (cu-real-ax-model-e2e-launcher.mjs:176-191). validateMonitorBaseline does not require frontmostPID to be a positive integer or the pointer x/y to be finite, so a NaN PID or pointer passes the READY check the test names "validates every READY field."
• Minor sanitizer inconsistency (cu-report-sanitize.mjs:11). toolCallId is allowlisted and passed verbatim in the trace path while sanitizeCuActionRecord runs it through safeId; tool-call IDs are SDK-generated so the risk is negligible, but the two paths could match. (The unbounded string actual in sanitizeAssertionResult predates this PR.)

[Astro-Han]

Approving. The core behaviors are verified correct; the notes posted are non-blocking (P2/P3 follow-ups).

[hqhq1025]

Rebased onto current main and addressed the additional adversarial qualification findings. Latest head 74fcca88 now records every rejected attempt in the canonical ledger, recomputes counts from that ledger, requires restart target_missing → fresh observation → successful AX retry, binds reports to run/commit/timestamp/generator lineage, preserves the report outside the temp directory, and supports relocated fixtures plus candidate-driver qualification only when path + SHA-256 + server version are all supplied. Validation: scripts 108/108; Runtime 1941 pass, 7 skip, 0 fail. New CI is running.

[hqhq1025]

Summary

Consolidates Computer Use qualification evidence into one fail-closed contract.

• require explicit producer/model/provider/transport/policy/terminal provenance;
• independently collect fixture PID/window identity;
• require latest-observation target lineage and invalidate it after target loss;
• consume one exact dispatch trace per successful mutation;
• prevent screenshot actions from inventing observation lineage;
• preserve full model recovery text in provider error tool results;
• retain the five-round restart runner as a non-qualifying soak while keeping one qualification path.

Verification

• scripts: 92 pass
• Runtime: 1599 pass, 7 skip
• CI typecheck/test/e2e: pass

[Astro-Han]

Approving — findings confirmed but all non-blocking: they're confined to the manually-run real-model / real-macOS qualification scripts, which CI does not execute, and there's no product/runtime regression. Follow-ups worth addressing since they touch the qualification contract this PR delivers:

• [P2] restart-recovery has no passing path (scripts/cu-real-ax-model-e2e.mjs): actionBudgetFor grants set_value: 1, but the pass condition needs a stale set_value failing target_missing AND a later successful set_value (two attempts); the second is rejected by the budget check before dispatch, so e2e:computer-use-process-restart / -real-ax-restart can never pass. Fix: allow set_value: 2 for restart-recovery and authorize the stale target_missing attempt.
• [P2] Over-budget / disallowed attempts are erased from the qualification report: the budget and allowed-action throws run before actions.push (and before totalActionAttempts = nextTotal), so validateRealReport never sees them — an auditability gap versus the stated fail-closed goal (the action is still blocked, so not a safety issue). Fix: record the attempt before throwing.
• [P2] launcher and matrix disagree on the pass verdict: cu-real-model-launcher.mjs omits the expectedActionSequence / lineage checks that validateRealReport enforces, so it can exit 0 on a report the matrix rejects as invalid. Fix: share one verdict.

[hqhq1025]

@Astro-Han All three qualification follow-ups are fixed in 3c9a3d93: restart recovery has the required two-attempt budget, rejected attempts remain in the evidence report, and launcher/matrix now share the same verdict contract. The branch was rebased onto current main; scripts, runtime, desktop, lint/typecheck, and latest CI are green. Ready for merge.

[Astro-Han]

Both product changes are correct. ai-sdk-backend.ts preserves the full modelText recovery text with no truncation and only throws on outputs already classified as tool errors, so normal flow is unchanged. The wait/cursor_position ownership exemption in computer-use-real-model-policy.ts matches the runtime's own non-mutating classification (both take an observation lease, not an action lease), so no mutating action loses its owned-observation requirement. The consolidation keeps validateRealReport running on the sanitized report, and the provenance and screenshot-lineage axes come out stronger than before (the sanitizer strips a screenshot's resultObservationId at cu-report-sanitize.mjs:112, so a screenshot cannot set the latest observation lineage).

A cluster of non-blocking robustness and coverage notes:

• Matrix dispatch-trace check is fail-open on extras (cu-provider-matrix.mjs:336-359). It rejects zero dispatch and prevents one trace serving two mutations, but does not reject a duplicate/extra trace or a dispatch trace on a success:false action. The exact-count assertion survives only in the AX harness runtime, not the matrix path.
• target_missing invalidation is enforced but untested (cu-provider-matrix.mjs:401-404). The reset-on-target-loss branch has no unit exercising it; removing those lines would fail nothing. Behavioral coverage rests on the real-runtime restart scenario.
• Declared restart failure is not required to occur (cu-provider-matrix.mjs:372-378). expectedFailures is an allowlist, so a report with only successful mutations can satisfy the type sequence, budget, and final state without ever observing the set_value/target_missing the scenario is meant to force.
• Coordinate-attempt coverage weakened. The deleted real-e2e harness actively attempted a coordinate action and asserted zero dispatch and zero side effects; the consolidated default relies on the model following "never use coordinates" rather than deterministically attempting one. The compatibilityInputBlocked guard itself is still unit-tested in the backend, so this is reduced e2e coverage rather than a lost guard.
• Monitor baseline accepts malformed numeric fields (cu-real-ax-model-e2e-launcher.mjs:176-191). validateMonitorBaseline does not require frontmostPID to be a positive integer or the pointer x/y to be finite, so a NaN PID or pointer passes the READY check the test names "validates every READY field."
• Minor sanitizer inconsistency (cu-report-sanitize.mjs:11). toolCallId is allowlisted and passed verbatim in the trace path while sanitizeCuActionRecord runs it through safeId; tool-call IDs are SDK-generated so the risk is negligible, but the two paths could match. (The unbounded string actual in sanitizeAssertionResult predates this PR.)

[Astro-Han]

Approving. The core behaviors are verified correct; the notes posted are non-blocking (P2/P3 follow-ups).

[hqhq1025]

Rebased onto current main and addressed the additional adversarial qualification findings. Latest head 74fcca88 now records every rejected attempt in the canonical ledger, recomputes counts from that ledger, requires restart target_missing → fresh observation → successful AX retry, binds reports to run/commit/timestamp/generator lineage, preserves the report outside the temp directory, and supports relocated fixtures plus candidate-driver qualification only when path + SHA-256 + server version are all supplied. Validation: scripts 108/108; Runtime 1941 pass, 7 skip, 0 fail. New CI is running.