Uh oh!
There was an error while loading. Please reload this page.
feat(chat): preview agent media on web and mobile - #5047
feat(chat): preview agent media on web and mobile#5047gabrielelpidio wants to merge 225 commits into
Conversation
Co-authored-by: codex <codex@users.noreply.github.com>
- Initialize provider as unchecked in a pending state - Update initial probe message to reflect session-local status
- Type the runtime effect with `Scope` - Build the ACP session runtime without wrapping it in `Effect.scoped`
- Use strict TurnId and ProviderItemId parsing in Codex session routing - Decode in-memory stdio chunks in streaming mode to avoid split UTF-8 corruption
- Transfer session-owned scopes into adapter state - Ensure runtime scopes close on stop and startup failure - Add regression coverage for scoped lifecycle cleanup
- Close the managed native event logger when the adapter layer tears down - Make session runtime close idempotent with an atomic closed flag - Add coverage for flushing thread native logs on shutdown
- Use codex app-server snapshots for auth, models, and skills - Remove legacy CLI/config discovery paths and related helpers - Update tests for the new provider status flow
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
- Document the target orchestration graph, IDs, lifecycles, and capability model - Add Codex app-server probe fixtures and update the probe test harness
- Introduce orchestration v2 service interfaces and error types - Add replay runtime, fixtures, and integration coverage - Update shared contracts and probe transcripts Co-authored-by: codex <codex@users.noreply.github.com>
- Add Codex adapter and replay harness wiring - Introduce in-memory orchestration projections and provider registry - Expand orchestration contracts for turn and runtime events
Co-authored-by: codex <codex@users.noreply.github.com>
- Add context transfer IDs, schemas, and projections - Support cheap fork creation and Codex native fork rollback - Cover fork idempotency and replay behavior in tests
- Track remaining projection, context transfer, rollback, capability, and subagent work - Clarify current V2 baseline and debugger-only follow-ups
- Map fork and merge-back turns into stored handoffs and transfer resolutions - Add shell snapshot projection support plus coverage tests - Update replay fixtures and web contracts for the new turn flow
Co-authored-by: codex <codex@users.noreply.github.com>
- Move Codex replay recording into `apps/server` - Add Claude Agent SDK replay fixtures and test harness - Update orchestration-v2 fixture scenarios and docs
- Move Claude provider runtime logic into its own module - Share the SDK query runner between live and replay paths - Add replay driver error wrapping for unexpected failures
Port orchestration V2 provider adapter wiring to the provider-instance driver registry. Co-authored-by: codex <codex@users.noreply.github.com>
- persist the selected model on run records - surface run model selection in the debug UI - update replay fixtures and contracts for the new field
- Record Claude SDK transcripts across multiple prompts and restart/query modes - Add approval and tool-call replay coverage for new orchestration fixtures - Update Claude adapter testkit to model open/prompt/permission frames
- Derive Claude SDK query options from runtime policy - Add read-only replay fixture and policy mapping tests - Reuse shared approval-policy fixtures across orchestrator tests Co-authored-by: codex <codex@users.noreply.github.com>
- add active steering and interrupt-restart replay fixtures - update Claude adapter/orchestrator turn handling for steering - refresh replay and integration test coverage
- add interrupt and mid-tool replay fixtures for Claude and Codex - log Claude Agent SDK protocol frames to native event traces - project Codex commandExecution start events into orchestration updates
- Map Cursor SDK agents and runs to V2 thread and turn lifecycles - Update MCP capability, tool, and testing guidance for SDK-based injection
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 084f6c7. Configure here.
Uh oh!
There was an error while loading. Please reload this page.
ApprovabilityVerdict: Needs human review This PR introduces a new feature for previewing agent media in chat, including new turn item types, asset resources, and security-sensitive file handling code (symlink protection, path traversal prevention). The scope includes orchestration changes and new UI components across web and mobile—changes that warrant human review. You can customize Macroscope's approvability policy. Learn more. |
a543fd4 to
378615bCompare22bd872 to
a27c1ccCompare519c42a to
4c55679Compareandybergon
commented
Aug 26, 2026
Fresh reproduction after #6433 merged:
This confirms that the first-class agent image-output path covered by this PR is still needed after #6433. |
ceea97b to
d2f1f51Compare
Problem
Agent-produced images and saved browser evidence are currently shown as paths instead of useful previews. The previous implementation in #4872 depended on the V1 orchestration layer removed by #2829.
What changed
image_viewturn item and project CodeximageView/ savedimageGenerationevents into itpreview_snapshotevidence either to a unique server artifact (save: true) or a validated workspace-relative PNG (savePath)Codex has a native image event and now emits the first-class V2 item. Other providers still get provider-independent markdown media rendering until their adapters expose an equivalent native event.
Important
This PR is intentionally stacked on #2829. TODO: retarget/rebase it to
mainafter #2829 merges.Verification
image_viewitem and rendered the same projection in web and iOSThe server typecheck currently reaches the existing #2829 error in untouched
apps/server/src/mcp/toolkits/worktree/registration.test.ts(ServerConfig | WorkspacePathsmissing from the expected test context). #2829's ownCheckandTestjobs are already failing at this base head.Screenshots
Web — inline previews
Web — expanded preview
Mobile — iOS
Reimplements the media-preview work from #4872 against orchestration V2.
Model: GPT-5.6-Sol
Harness: T3 Code (Codex)
Note
High Risk
Touches signed asset serving, symlink-safe file I/O, and new asset claim types—security-sensitive paths that must stay fail-closed; broad surface across server, contracts, web, and mobile.
Overview
Adds inline image and video previews in web and mobile chat instead of raw paths, reimplemented on orchestration V2 after the V1 layer removal.
Orchestration & feed: Introduces a durable V2
image_viewturn item (CodeximageView/ savedimageGeneration). Completed items surface as dedicatedimage-outputtimeline/feed rows; failed ones stay in the work log.Markdown media: New
MarkdownMedia(web + mobile) resolves markdownimg/videoviaresolveMarkdownMediaSource—direct URLs, thread workspace files,browser-artifact, orthread-image—and loads them through signed asset URLs with loading/error UI and image expand.Assets & server: Extends
AssetResourcewithbrowser-artifactandthread-image; video workspace previews use exact-file claims.resolveAssetcan return streamedopen-fileresponses.noFollowFilehardens reads/writes and screenshot persistence against traversal/symlinks.Preview MCP:
preview_snapshotcansave: true(server artifact +savedScreenshotPath) orsavePath(validated workspace PNG); tool hints updated accordingly.Mobile markdown:
renderImage/ context lets native selectable markdown delegate image nodes to the same media pipeline.Reviewed by Cursor Bugbot for commit 0387ef2. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add inline media preview for agent images and videos in web and mobile chat
image_viewas a new turn item type in the orchestration V2 contract, projected from Codex image/imageGeneration items and emitted asimage-outputtimeline/feed entries in both web and mobile.image-outputrows as clickable thumbnail previews viaImageOutputTimelineRow, and markdownimg/videotags are now rendered through a newMarkdownMediacomponent that resolves thread-scoped asset URLs.MarkdownMediaandThreadImageOutputcomponents with loading, unavailable, and tap-to-expand states; custom image rendering is injected into markdown via a newrenderImageprop and React context.preview_snapshotMCP tool now accepts optionalsave(boolean) orsavePathparameters to persist screenshots either to the server'sbrowser-artifactsstore or a workspace-relative path, with symlink traversal protection.AssetResourcevariants (browser-artifact,thread-image) are added to contracts, with correspondingresolveAssetandissueAssetUrllogic backed by no-follow file opening on Linux, macOS, and other platforms.resolveAssetnow returns anopen-filevariant with aReadStreamin addition to the existingfilevariant; callers that exhaustively switch onResolvedAssetmust handle the new case.Macroscope summarized 0387ef2.