Skip to content

feat(cu): establish foundation contract and approval boundary - #857

Merged
Astro-Han merged 1 commit into
apache:mainfrom
hqhq1025:codex/cu-foundation-core
Jul 13, 2026
Merged

feat(cu): establish foundation contract and approval boundary#857
Astro-Han merged 1 commit into
apache:mainfrom
hqhq1025:codex/cu-foundation-core

Conversation

@hqhq1025

@hqhq1025hqhq1025 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

First slice of the rebuilt Computer Use stack. This PR establishes the provider-neutral contract and the approval/privacy boundary before any runtime frame state or native executor is added.

  • adds shared action, observation, window/page, bound-action, outcome, and lifecycle error vocabulary;
  • removes foreground input as a valid dispatch tier;
  • separates metadata, screenshot, pointer, keyboard, and semantic approval scopes;
  • prevents unbound mutations from being remembered for the turn;
  • snapshots tool and permission arguments before the first async boundary;
  • persists only an allowlisted Computer Use summary, excluding text, coordinates, and raw UI content;
  • fixes the Pi adapter to accept the canonical computer_use category;
  • adds the accepted foundation contract and current validation matrix.

Reverse-engineering evidence

The contract is aligned with the local Codex Computer Use reproduction lab:

  • docs/08-wrapper-policy-and-toctou.md: canonical app approval, pre-await parameter snapshot, approval is capability rather than action freshness;
  • docs/13-policy-error-state-machine.md: policy -> approval -> fresh observation -> action, plus intervention/lock/blocked-URL states;
  • docs/16-service-process-lifecycle-and-retention.md: exact executable ownership and fail-closed connection cleanup;
  • docs/19-electron-presentation-and-mcp-event-contract.md: presentation is separate from native action transport.

Maka intentionally uses a stricter short-lease policy than Codex's persistent app approval: target and action class changes require a new approval.

Scope

Owned by this PR:

  • core contract and error taxonomy;
  • permission category, scoped leases, and UI;
  • pre-await argument snapshot;
  • Computer Use persistence/telemetry argument redaction.

Not owned by this PR:

  • frame/session state implementation;
  • cua-driver backend or binary packaging;
  • Desktop wiring;
  • cursor presentation;
  • provider qualification or real-model E2E.

Verification

Validated again after rebasing onto maka-agent/main@9750ff51:

  • full repository build passed;
  • full repository test passed: Core 846/846, Runtime 1452 passed with 7 platform skips, UI 125/125, Desktop 2409/2409;
  • full repository typecheck passed, including Storybook;
  • upstream CI passed: typecheck, test, and e2e;
  • git diff --check passed.

Stack

This is PR A. Planned follow-ups:

  1. Runtime frame/action/session leases.
  2. cua-driver service and exact executor.
  3. binary provenance and packaging.
  4. Desktop production wiring.
  5. presentation lifecycle.
  6. cumulative deterministic E2E.

Tracks #836.

@hqhq1025

Copy link
Copy Markdown
ContributorAuthor

@astrohan The rebuilt Computer Use stack is now fully published and ready for review. Merge order: #857 -> #892 -> #893 -> #894 -> #895 -> #896 -> #897 -> #898. Each follow-up currently targets main, so its Files tab is cumulative until predecessors merge; every PR links the fork-local exact net diff. All branches were rebased onto the latest stack and verified, including isolated real-machine E2E and concurrent-user fail-closed E2E.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving #857 as the foundation PR. I do not see a P1 in this PR by itself.

P2

  1. ToolRuntime snapshots tool arguments, but the dynamic sandbox resolver still receives the original args. It should receive executionArgs so permission and execution use the same input.
  2. The approval contract accepts targetless observe and screenshot requests. #857 has no executor, so I am treating this as P2 here, but the contract should either require a target or define the target resolution step before approval.
  3. Real mutation requests carry an observation_id but no app or window. The permission prompt therefore cannot show the canonical target required by the contract. The new Story uses app and window fields that the runtime schema will reject for these actions.
  4. bounded() collapses whitespace and truncates app selectors before building the remembered scope. Distinct exact selectors can receive the same scope key. Authorization identity should remain lossless; display normalization should be separate.
  5. The persisted Computer Use summary still accepts raw app and observationId values without secret redaction. Since the driver accepts an exact window title as app, window titles and secret-shaped identifiers can enter session logs, telemetry, and artifact metadata.
  6. PiAgentBackend writes raw Computer Use arguments through ensureToolCall() before permission projection. Typed text and coordinates can be stored even though the emitted permission request is redacted. The tool_start path has the same gap.
  7. Unknown, malformed, and unbound requests can receive reusable scopes. Examples include an unbound type request with only app, different unknown actions collapsing into the same unknown scope, and screenshot being downgraded by include_screenshot:false. Invalid shapes should fail before permission evaluation, or at minimum be ineligible for turn memory.
  8. The CLI prompt says only allow, but every accepted request is sent with rememberForTurn:true. It needs separate one-time and turn-scoped choices when rememberForTurnAllowed is true.

P3

  1. The Desktop note says a new observation will prompt again, but a remembered observe request for the same target is allowed without another prompt. The copy and lease behavior should agree.
  2. The accepted contract links four docs/*.md evidence files that are not present in the repository. Please add stable references or remove those links.

Scope note: once #892 and #893 are included, item 2 becomes a reachable P1 because targetless observe selects a window after approval and targetless screenshot captures the desktop. That should be fixed before those PRs merge.

@hqhq1025
hqhq1025force-pushed the codex/cu-foundation-core branch from 79923b4 to b15b1e6CompareJuly 13, 2026 17:36
@hqhq1025

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han Thanks, I went through all ten points and applied the ones that hold under the current architecture.

Addressed in #857:

  1. Dynamic sandbox resolution now receives the frozen executionArgs.
  2. Computer Use remembered-scope identity now uses the lossless exact action/app/window/observation tuple; display normalization/redaction is separate.
  3. Permission/persistence summaries redact secret-shaped app selectors and observation IDs, while text and coordinates remain excluded.
  4. Pi tool_start, ensureToolCall, and permission-request storage now project Computer Use args before emitting or persisting them. Raw wire args are retained only in the in-memory permission evaluator so exact scope identity is not collapsed by redaction.
  5. Unknown, malformed, targetless, and unbound requests are not eligible for turn memory, and a forged remembered key cannot bypass that rule. screenshot can no longer downgrade itself with include_screenshot:false.
  6. CLI y/Enter now means allow once; a means allow for the turn and is offered only when rememberForTurnAllowed === true.
  7. Desktop copy now matches the lease behavior: a new observation alone does not re-prompt; target/action/class changes do.
  8. The contract now labels the Codex-lab files as external evidence references instead of repository-local links.

Addressed in #892 because these require Runtime-owned observation state:

  1. observe and screenshot now require an explicit app or window before approval. Targeted screenshot uses observeApp for that exact target instead of desktop capture.
  2. Mutation permission args are synchronously enriched from the active Runtime observation with the trusted app/window. A mismatched or stale observation ID receives no injected target and remains ineligible for turn memory. feat(cu): add exact cua-driver executor service #893 still performs the fresh PID/window/page validation before dispatch.

I intentionally did not make Core invent or trust a canonical target before Runtime has observed one. That would move executor state into the foundation and weaken the boundary rather than fix it.

Validation after rebasing onto current main: #857 full build/test/typecheck passed (Desktop 2415/2415); #892 Runtime 1462 passed with 7 platform skips and full typecheck; #893 Computer Use 103/103 and full typecheck; downstream stack was rebased and its CI is running.

@Astro-Han
Astro-Han merged commit e2b40c4 into apache:mainJul 13, 2026
3 checks passed
Astro-Han added a commit that referenced this pull request Jul 16, 2026
* docs: retire tracked notes into the archive
notes/ regrew tracked files after the archive README retired it as an
undocumented parallel authority. Move the simplification baseline map
(#871-#887) and the shipped composer-mentions v1 spec (#979) into
docs/archive/, register them under Former repository notes, and point
the citing source comments at the archived path.
* docs: archive computer-use chronicles and research records
The #857-#985 computer-use chain left one root document per PR:
follow-up fix records, an incident investigation, and run logs that
duplicate the kept contracts. Archive the seven chronicles, mark the
foundation contract validation matrix as the #857 split-gate baseline
(current state lives in source and contract tests), and register each
archived file in the archive README with a pointer to its current
authority.
Also archive the WorkBuddy/QoderWork reverse-engineering records (the
shipped design contract stays at docs/expert-team-runtime.md) and the
synthesis-cache benchmark log (progress belongs in issues #481/#578),
and fix the two surviving links to the moved paths.
* docs: map current contracts in the documentation README
Seventeen root documents existed outside the authority map with zero
incoming links. After archiving the chronicles, list the surviving
computer-use contracts and the expert-teams runtime contract, and
write down the maintenance rule that PR follow-up records, incident
investigations, and run logs belong in pull requests or the archive —
with notes/ and docs/local/ as untracked local scratch.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@hqhq1025@Astro-Han