refactor(media): unify file and media previews across clients

[juliusmarminge]

What changed

• Moved markdown file-link parsing, media-source resolution, signed URL state, and media-action availability into client-runtime so web and mobile use the same rules.
• Routed expanded web videos through the shared streaming player, including URL refresh and retry.
• Routed every iOS video through AVKit and every iOS image, PDF, and HTML preview through Quick Look.
• Moved media actions to a long-press on the thumbnail and removed the redundant in-player controls.

Native long-press menus on markdown file chips are in #9258.

Why

The web and mobile clients had separate path parsers, media resolvers, URL lifecycle code, and viewer paths. The copies had already drifted. Some videos downloaded before playback, expired URLs left expanded viewers stuck, and iOS media sometimes opened in React Native viewers instead of AVKit or Quick Look.

This puts the shared decisions in one package and leaves each client responsible only for presentation. On iOS, native viewers own media playback and document preview.

UI changes

Before: clipped initial video controls	After: AVKit controls fill the thumbnail

Pre-fix tap and long-press race reproduced on device: screen recording

Testing

• 40 focused mobile file, markdown, and media tests
• Mobile TypeScript typecheck
• TypeScript lint and formatting checks for changed files
• Native Swift and Kotlin static checks
• Physical iPhone 16 Pro pass against a disposable LAN environment, covering workspace and external files, images, videos, HTML, PDFs, attachments, composer drafts, markdown embeds, thumbnail long-press menus, AVKit, and Quick Look

Checklist

• This PR is focused on one media-preview refactor (net negative after splitting out feat(mobile): long-press file references for path and open actions #9258)
• I explained what changed and why
• I included before/after screenshots for UI changes
• I included a recording for interaction changes

Created with gpt-5.6-sol using the Codex harness in T3 Code.

Note

Unify file and media preview parsing and playback across web and mobile

• Adds shared markdown file-link and media-source resolvers in client-runtime (parseMarkdownFileLink, resolveMediaSource) and replaces local parsing implementations in web, mobile, and the t3-markdown module with these shared helpers
• Reworks video previews on both platforms to use the shared MediaVideoPlayer; mobile removes the Expo-based VideoPlayback and MediaVideoPreviewModal, and web replaces raw <video> rendering in ExpandedImageDialog, MessagesTimeline, and ChatMarkdown with the shared player
• Moves AssetUrlState and assetUrlStateFromResult into client-runtime; web and mobile asset URL hooks now derive loading/failure/success through the shared mapper instead of local inline checks
• Unifies media action menus via a shared MediaActionId union and MediaActionsMenu/useMediaActions; both platforms now expose save/share/open-file actions through the same identifiers and long-press surfaces
• Risk: resolveViewedImageAsset and ViewedImageAsset now only return media-file resources, removing the attachment-resource variant — callers expecting attachment resources from viewed workspace images will break; iOS FilePreview always uses NativeFilePreview and no longer renders MediaImagePreview for actionable images; MediaVideoPlayer no longer renders an expand/maximize control or accepts an onExpand callback

^{Macroscope summarized 452f18a.}

---

Note

Medium Risk
Broad cross-platform media and preview behavior changes (iOS native viewers, inline vs modal video, shared URL lifecycle) touch high-traffic chat and file flows, though logic is consolidated and covered by focused tests.

Overview
Centralizes markdown file-link parsing, media-source classification, signed asset URL state, and media action IDs in client-runtime, then rewires web and mobile to call those helpers instead of duplicated local heuristics.

Mobile drops bespoke link/path code in favor of parseMarkdownFileLink and resolveMediaSource. Video and attachment UX converges on useMediaActions / MediaActionsMenu (long-press on thumbnails); VideoAttachmentMenu, MediaVideoPreviewModal, and the workspace image prefetch atom are removed. iOS routes all full-screen file previews through Quick Look and full-screen videos through AVKit (with URL re-mint on failure); inline/workspace video stays on Expo Video. Composer and thread attachments share attachmentVideoPreviewSource and attachment-scoped asset resources.

Web plays timeline and markdown videos inline via the shared MediaVideoPlayer (no separate “open video” flow or expand control); expanded dialogs stream through the same player with asset refresh. File attachment clicks no longer auto-open a modal for videos. Context menus align action IDs (save, copy-full-path, etc.) and expose save for videos as well as images.

Docs are updated to describe streaming, long-press save/share, and platform-specific players.

^{Reviewed by Cursor Bugbot for commit 452f18a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.5 KiB	+152 B (+1.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+6 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+146 B (+2.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	57.0 KiB	+1.4 KiB (+2.6%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	+27 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−6 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+33 B (+0.5%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.4 KiB	+44 B (+0.1%)	66.4 KiB	✅
Claude	Live turn messages	9	10	+1 (+11.1%)	21	✅

Baseline: 7751299 · PR result: 452f18a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a cross-client media-preview redesign that changes playback, asset URL minting, sharing, file-link parsing, and action menus across shared runtime, web, and mobile production paths. The new behavior and infrastructure span many existing surfaces, exceeding the scope of an auto-approvable refactor.

Not approved because:

• Monthly spending limit reached (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 05b0284. Configure here.}

[macroscopeapp]

🟡 Mediumsrc/terminal-links.ts:251

Activating a POSIX path ending in :0, such as logs/output:0, resolves it to <cwd>/logs/output instead of the actual file <cwd>/logs/output:0. splitFilePathPosition drops the non-positive line suffix and formatFilePathPosition does not restore it; preserve the suffix as part of the path when no valid line is parsed.

- const { path } = position;+ const path =+ position.line === undefined && rawPath.endsWith(":0") ? rawPath : position.path;

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/terminal-links.ts around line 251:
Activating a POSIX path ending in `:0`, such as `logs/output:0`, resolves it to `<cwd>/logs/output` instead of the actual file `<cwd>/logs/output:0`. `splitFilePathPosition` drops the non-positive line suffix and `formatFilePathPosition` does not restore it; preserve the suffix as part of the path when no valid line is parsed.

[macroscopeapp]

🟠 Highcomponents/VideoPreviewModal.tsx:94

Save/share can fail after the modal closes because useLocalPlayback exposes only the resolved uri, so useMediaActions takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft attachment through the share path (or otherwise retain it for the operation), as AttachmentPreviewFile.share does.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/mobile/src/components/VideoPreviewModal.tsx around line 94:
Save/share can fail after the modal closes because `useLocalPlayback` exposes only the resolved `uri`, so `useMediaActions` takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft `attachment` through the share path (or otherwise retain it for the operation), as `AttachmentPreviewFile.share` does.

[macroscopeapp]

🟡 Mediumcomponents/ChatMarkdown.tsx:2621

Workspace-backed images rendered through ChatMarkdownAssetImage no longer open the full-size preview when clicked or activated by keyboard. The onImageExpand={imageExpand} prop was removed from ChatMarkdownVideo, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

 style={authoredSizeStyle}
+ onImageExpand={imageExpand}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/components/ChatMarkdown.tsx around line 2621:
Workspace-backed images rendered through `ChatMarkdownAssetImage` no longer open the full-size preview when clicked or activated by keyboard. The `onImageExpand={imageExpand}` prop was removed from `ChatMarkdownVideo`, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

[macroscopeapp]

Dropping onImageExpand here (and MediaVideoPlayer's onExpand) removes the only route from a markdown video embed to the expanded dialog, while the sibling image embed still expands on click (expandableMarkdownImageProps + cursor-zoom-in, same component). Inline embeds are capped by CHAT_MARKDOWN_MEDIA_MAX_WIDTH_CLASS_NAME, so a video's only remaining enlargement is the browser's own fullscreen button — and it also loses the dialog-level media actions that images keep. docs/user/composer.md (line 100, in a paragraph that covers web, desktop, and mobile) still states "video embeds show a player with controls and an option to expand", and both platforms lost that control in this PR.

If the removal is intentional, updating that doc sentence is the smallest fix; otherwise consider keeping the expand handler for video embeds so images and videos behave the same inline.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Attachment videos now mint their URL inside this dialog, but sourceFailed only reflects Failure. While the environment is reconnecting (connecting/backoff, or connected with no session yet), assetEnvironment.createUrl resolves to Effect.never (client-runtime state/runtime.ts), so assetUrl stays Loading and the dialog paints the black stateClassName placeholder with no message, no retry, and no end state until the connection comes back. Before this change the same click went through downloadFileAttachment, which pre-checked readPreparedConnection and toasted "The environment is not connected.", and the mobile player added in this PR (useMediaPlayback) folds connection._tag === "None" into unavailable for exactly this case — so web is now the only surface with no disconnected end state.

Smallest fix is to treat a missing prepared connection as a failure here (needs usePreparedConnection from ../../state/session), which also gives the panel its Retry button:

constconnection=usePreparedConnection(asset?.environmentId??null);// ...sourceFailed={assetUrl._tag==="Failure"||(asset!==undefined&&connection._tag==="None")}

Keeping the connection pre-check in openFileAttachment before setExpandedImage would work too.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

🟢 Lowinternals/mobile-navigation.md:95

The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, OpenVideoPreviewModal renders MediaActionsMenu with inModal in the header and exposes a Save or share video button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

-Media actions (copy path, open in file viewer, save or share) belong to the surface that-opened the video, a long-press on the thumbnail, so the player carries no-menu. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.+On iOS, media actions (copy path, open in file viewer, save or share) belong to the+surface that opened the video, a long-press on the thumbnail, so the player carries no+menu. Android exposes `MediaActionsMenu` and a `Save or share video` button in the modal. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @docs/internals/mobile-navigation.md around lines 95-97:
The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, `OpenVideoPreviewModal` renders `MediaActionsMenu` with `inModal` in the header and exposes a `Save or share video` button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

[macroscopeapp]

This tile is labelled Play ${file.name}, but on this branch of the handler it no longer plays. onFileOpen is ChatView's openFileAttachment, which routes only html/pdf to the panel and otherwise calls downloadFileAttachment — and this PR removed that function's video branch (the one that used to setExpandedImage({ images: [{ src: url, name, type: "video" }] })). So for a rehydrated draft (file.file === null with an uploadedAttachmentId, the normal state after a reload) clicking the play glyph now silently saves the file to disk instead of opening the player. The removal of the aria-busy/"Loading…" state on these lines assumed the click still ends in the expanded dialog, and timeline videos moved to MediaVideoPlayer, so this is the only surface left on the old play path.

Smallest fix is to open the dialog here instead of downloading: build the item with src: null and actionsSource: { kind: "video", name: file.name, src: null, asset: buildAttachmentVideoAsset(file.uploadEnvironmentId ?? environmentId, { ...file, id: file.uploadedAttachmentId }) } and pass it to onExpandImage — ExpandedVideo already mints and plays that shape. If downloading is the intended behavior for this state, the control should say so rather than "Play".

^{Posted via Macroscope — UI Consistency}

[juliusmarminge]

What changed

• Moved markdown file-link parsing, media-source resolution, signed URL state, and media-action availability into client-runtime so web and mobile use the same rules.
• Routed expanded web videos through the shared streaming player, including URL refresh and retry.
• Routed every iOS video through AVKit and every iOS image, PDF, and HTML preview through Quick Look.
• Moved media actions to a long-press on the thumbnail and removed the redundant in-player controls.

Native long-press menus on markdown file chips are in #9258.

Why

The web and mobile clients had separate path parsers, media resolvers, URL lifecycle code, and viewer paths. The copies had already drifted. Some videos downloaded before playback, expired URLs left expanded viewers stuck, and iOS media sometimes opened in React Native viewers instead of AVKit or Quick Look.

This puts the shared decisions in one package and leaves each client responsible only for presentation. On iOS, native viewers own media playback and document preview.

UI changes

Before: clipped initial video controls	After: AVKit controls fill the thumbnail

Pre-fix tap and long-press race reproduced on device: screen recording

Testing

• 40 focused mobile file, markdown, and media tests
• Mobile TypeScript typecheck
• TypeScript lint and formatting checks for changed files
• Native Swift and Kotlin static checks
• Physical iPhone 16 Pro pass against a disposable LAN environment, covering workspace and external files, images, videos, HTML, PDFs, attachments, composer drafts, markdown embeds, thumbnail long-press menus, AVKit, and Quick Look

Checklist

• This PR is focused on one media-preview refactor (net negative after splitting out feat(mobile): long-press file references for path and open actions #9258)
• I explained what changed and why
• I included before/after screenshots for UI changes
• I included a recording for interaction changes

Created with gpt-5.6-sol using the Codex harness in T3 Code.

Note

Unify file and media preview parsing and playback across web and mobile

• Adds shared markdown file-link and media-source resolvers in client-runtime (parseMarkdownFileLink, resolveMediaSource) and replaces local parsing implementations in web, mobile, and the t3-markdown module with these shared helpers
• Reworks video previews on both platforms to use the shared MediaVideoPlayer; mobile removes the Expo-based VideoPlayback and MediaVideoPreviewModal, and web replaces raw <video> rendering in ExpandedImageDialog, MessagesTimeline, and ChatMarkdown with the shared player
• Moves AssetUrlState and assetUrlStateFromResult into client-runtime; web and mobile asset URL hooks now derive loading/failure/success through the shared mapper instead of local inline checks
• Unifies media action menus via a shared MediaActionId union and MediaActionsMenu/useMediaActions; both platforms now expose save/share/open-file actions through the same identifiers and long-press surfaces
• Risk: resolveViewedImageAsset and ViewedImageAsset now only return media-file resources, removing the attachment-resource variant — callers expecting attachment resources from viewed workspace images will break; iOS FilePreview always uses NativeFilePreview and no longer renders MediaImagePreview for actionable images; MediaVideoPlayer no longer renders an expand/maximize control or accepts an onExpand callback

^{Macroscope summarized 452f18a.}

---

Note

Medium Risk
Broad cross-platform media and preview behavior changes (iOS native viewers, inline vs modal video, shared URL lifecycle) touch high-traffic chat and file flows, though logic is consolidated and covered by focused tests.

Overview
Centralizes markdown file-link parsing, media-source classification, signed asset URL state, and media action IDs in client-runtime, then rewires web and mobile to call those helpers instead of duplicated local heuristics.

Mobile drops bespoke link/path code in favor of parseMarkdownFileLink and resolveMediaSource. Video and attachment UX converges on useMediaActions / MediaActionsMenu (long-press on thumbnails); VideoAttachmentMenu, MediaVideoPreviewModal, and the workspace image prefetch atom are removed. iOS routes all full-screen file previews through Quick Look and full-screen videos through AVKit (with URL re-mint on failure); inline/workspace video stays on Expo Video. Composer and thread attachments share attachmentVideoPreviewSource and attachment-scoped asset resources.

Web plays timeline and markdown videos inline via the shared MediaVideoPlayer (no separate “open video” flow or expand control); expanded dialogs stream through the same player with asset refresh. File attachment clicks no longer auto-open a modal for videos. Context menus align action IDs (save, copy-full-path, etc.) and expose save for videos as well as images.

Docs are updated to describe streaming, long-press save/share, and platform-specific players.

^{Reviewed by Cursor Bugbot for commit 452f18a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.5 KiB	+152 B (+1.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+6 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+146 B (+2.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	57.0 KiB	+1.4 KiB (+2.6%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	+27 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−6 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+33 B (+0.5%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.4 KiB	+44 B (+0.1%)	66.4 KiB	✅
Claude	Live turn messages	9	10	+1 (+11.1%)	21	✅

Baseline: 7751299 · PR result: 452f18a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a cross-client media-preview redesign that changes playback, asset URL minting, sharing, file-link parsing, and action menus across shared runtime, web, and mobile production paths. The new behavior and infrastructure span many existing surfaces, exceeding the scope of an auto-approvable refactor.

Not approved because:

• Monthly spending limit reached (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 05b0284. Configure here.}

[macroscopeapp]

🟡 Mediumsrc/terminal-links.ts:251

Activating a POSIX path ending in :0, such as logs/output:0, resolves it to <cwd>/logs/output instead of the actual file <cwd>/logs/output:0. splitFilePathPosition drops the non-positive line suffix and formatFilePathPosition does not restore it; preserve the suffix as part of the path when no valid line is parsed.

- const { path } = position;+ const path =+ position.line === undefined && rawPath.endsWith(":0") ? rawPath : position.path;

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/terminal-links.ts around line 251:
Activating a POSIX path ending in `:0`, such as `logs/output:0`, resolves it to `<cwd>/logs/output` instead of the actual file `<cwd>/logs/output:0`. `splitFilePathPosition` drops the non-positive line suffix and `formatFilePathPosition` does not restore it; preserve the suffix as part of the path when no valid line is parsed.

[macroscopeapp]

🟠 Highcomponents/VideoPreviewModal.tsx:94

Save/share can fail after the modal closes because useLocalPlayback exposes only the resolved uri, so useMediaActions takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft attachment through the share path (or otherwise retain it for the operation), as AttachmentPreviewFile.share does.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/mobile/src/components/VideoPreviewModal.tsx around line 94:
Save/share can fail after the modal closes because `useLocalPlayback` exposes only the resolved `uri`, so `useMediaActions` takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft `attachment` through the share path (or otherwise retain it for the operation), as `AttachmentPreviewFile.share` does.

[macroscopeapp]

🟡 Mediumcomponents/ChatMarkdown.tsx:2621

Workspace-backed images rendered through ChatMarkdownAssetImage no longer open the full-size preview when clicked or activated by keyboard. The onImageExpand={imageExpand} prop was removed from ChatMarkdownVideo, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

 style={authoredSizeStyle}
+ onImageExpand={imageExpand}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/components/ChatMarkdown.tsx around line 2621:
Workspace-backed images rendered through `ChatMarkdownAssetImage` no longer open the full-size preview when clicked or activated by keyboard. The `onImageExpand={imageExpand}` prop was removed from `ChatMarkdownVideo`, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

[macroscopeapp]

Dropping onImageExpand here (and MediaVideoPlayer's onExpand) removes the only route from a markdown video embed to the expanded dialog, while the sibling image embed still expands on click (expandableMarkdownImageProps + cursor-zoom-in, same component). Inline embeds are capped by CHAT_MARKDOWN_MEDIA_MAX_WIDTH_CLASS_NAME, so a video's only remaining enlargement is the browser's own fullscreen button — and it also loses the dialog-level media actions that images keep. docs/user/composer.md (line 100, in a paragraph that covers web, desktop, and mobile) still states "video embeds show a player with controls and an option to expand", and both platforms lost that control in this PR.

If the removal is intentional, updating that doc sentence is the smallest fix; otherwise consider keeping the expand handler for video embeds so images and videos behave the same inline.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Attachment videos now mint their URL inside this dialog, but sourceFailed only reflects Failure. While the environment is reconnecting (connecting/backoff, or connected with no session yet), assetEnvironment.createUrl resolves to Effect.never (client-runtime state/runtime.ts), so assetUrl stays Loading and the dialog paints the black stateClassName placeholder with no message, no retry, and no end state until the connection comes back. Before this change the same click went through downloadFileAttachment, which pre-checked readPreparedConnection and toasted "The environment is not connected.", and the mobile player added in this PR (useMediaPlayback) folds connection._tag === "None" into unavailable for exactly this case — so web is now the only surface with no disconnected end state.

Smallest fix is to treat a missing prepared connection as a failure here (needs usePreparedConnection from ../../state/session), which also gives the panel its Retry button:

constconnection=usePreparedConnection(asset?.environmentId??null);// ...sourceFailed={assetUrl._tag==="Failure"||(asset!==undefined&&connection._tag==="None")}

Keeping the connection pre-check in openFileAttachment before setExpandedImage would work too.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

🟢 Lowinternals/mobile-navigation.md:95

The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, OpenVideoPreviewModal renders MediaActionsMenu with inModal in the header and exposes a Save or share video button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

-Media actions (copy path, open in file viewer, save or share) belong to the surface that-opened the video, a long-press on the thumbnail, so the player carries no-menu. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.+On iOS, media actions (copy path, open in file viewer, save or share) belong to the+surface that opened the video, a long-press on the thumbnail, so the player carries no+menu. Android exposes `MediaActionsMenu` and a `Save or share video` button in the modal. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @docs/internals/mobile-navigation.md around lines 95-97:
The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, `OpenVideoPreviewModal` renders `MediaActionsMenu` with `inModal` in the header and exposes a `Save or share video` button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

[macroscopeapp]

This tile is labelled Play ${file.name}, but on this branch of the handler it no longer plays. onFileOpen is ChatView's openFileAttachment, which routes only html/pdf to the panel and otherwise calls downloadFileAttachment — and this PR removed that function's video branch (the one that used to setExpandedImage({ images: [{ src: url, name, type: "video" }] })). So for a rehydrated draft (file.file === null with an uploadedAttachmentId, the normal state after a reload) clicking the play glyph now silently saves the file to disk instead of opening the player. The removal of the aria-busy/"Loading…" state on these lines assumed the click still ends in the expanded dialog, and timeline videos moved to MediaVideoPlayer, so this is the only surface left on the old play path.

Smallest fix is to open the dialog here instead of downloading: build the item with src: null and actionsSource: { kind: "video", name: file.name, src: null, asset: buildAttachmentVideoAsset(file.uploadEnvironmentId ?? environmentId, { ...file, id: file.uploadedAttachmentId }) } and pass it to onExpandImage — ExpandedVideo already mints and plays that shape. If downloading is the intended behavior for this state, the control should say so rather than "Play".

^{Posted via Macroscope — UI Consistency}

[juliusmarminge]

What changed

• Moved markdown file-link parsing, media-source resolution, signed URL state, and media-action availability into client-runtime so web and mobile use the same rules.
• Routed expanded web videos through the shared streaming player, including URL refresh and retry.
• Routed every iOS video through AVKit and every iOS image, PDF, and HTML preview through Quick Look.
• Moved media actions to a long-press on the thumbnail and removed the redundant in-player controls.

Native long-press menus on markdown file chips are in #9258.

Why

The web and mobile clients had separate path parsers, media resolvers, URL lifecycle code, and viewer paths. The copies had already drifted. Some videos downloaded before playback, expired URLs left expanded viewers stuck, and iOS media sometimes opened in React Native viewers instead of AVKit or Quick Look.

This puts the shared decisions in one package and leaves each client responsible only for presentation. On iOS, native viewers own media playback and document preview.

UI changes

Before: clipped initial video controls	After: AVKit controls fill the thumbnail

Pre-fix tap and long-press race reproduced on device: screen recording

Testing

• 40 focused mobile file, markdown, and media tests
• Mobile TypeScript typecheck
• TypeScript lint and formatting checks for changed files
• Native Swift and Kotlin static checks
• Physical iPhone 16 Pro pass against a disposable LAN environment, covering workspace and external files, images, videos, HTML, PDFs, attachments, composer drafts, markdown embeds, thumbnail long-press menus, AVKit, and Quick Look

Checklist

• This PR is focused on one media-preview refactor (net negative after splitting out feat(mobile): long-press file references for path and open actions #9258)
• I explained what changed and why
• I included before/after screenshots for UI changes
• I included a recording for interaction changes

Created with gpt-5.6-sol using the Codex harness in T3 Code.

Note

Unify file and media preview parsing and playback across web and mobile

• Adds shared markdown file-link and media-source resolvers in client-runtime (parseMarkdownFileLink, resolveMediaSource) and replaces local parsing implementations in web, mobile, and the t3-markdown module with these shared helpers
• Reworks video previews on both platforms to use the shared MediaVideoPlayer; mobile removes the Expo-based VideoPlayback and MediaVideoPreviewModal, and web replaces raw <video> rendering in ExpandedImageDialog, MessagesTimeline, and ChatMarkdown with the shared player
• Moves AssetUrlState and assetUrlStateFromResult into client-runtime; web and mobile asset URL hooks now derive loading/failure/success through the shared mapper instead of local inline checks
• Unifies media action menus via a shared MediaActionId union and MediaActionsMenu/useMediaActions; both platforms now expose save/share/open-file actions through the same identifiers and long-press surfaces
• Risk: resolveViewedImageAsset and ViewedImageAsset now only return media-file resources, removing the attachment-resource variant — callers expecting attachment resources from viewed workspace images will break; iOS FilePreview always uses NativeFilePreview and no longer renders MediaImagePreview for actionable images; MediaVideoPlayer no longer renders an expand/maximize control or accepts an onExpand callback

^{Macroscope summarized 452f18a.}

---

Note

Medium Risk
Broad cross-platform media and preview behavior changes (iOS native viewers, inline vs modal video, shared URL lifecycle) touch high-traffic chat and file flows, though logic is consolidated and covered by focused tests.

Overview
Centralizes markdown file-link parsing, media-source classification, signed asset URL state, and media action IDs in client-runtime, then rewires web and mobile to call those helpers instead of duplicated local heuristics.

Mobile drops bespoke link/path code in favor of parseMarkdownFileLink and resolveMediaSource. Video and attachment UX converges on useMediaActions / MediaActionsMenu (long-press on thumbnails); VideoAttachmentMenu, MediaVideoPreviewModal, and the workspace image prefetch atom are removed. iOS routes all full-screen file previews through Quick Look and full-screen videos through AVKit (with URL re-mint on failure); inline/workspace video stays on Expo Video. Composer and thread attachments share attachmentVideoPreviewSource and attachment-scoped asset resources.

Web plays timeline and markdown videos inline via the shared MediaVideoPlayer (no separate “open video” flow or expand control); expanded dialogs stream through the same player with asset refresh. File attachment clicks no longer auto-open a modal for videos. Context menus align action IDs (save, copy-full-path, etc.) and expose save for videos as well as images.

Docs are updated to describe streaming, long-press save/share, and platform-specific players.

^{Reviewed by Cursor Bugbot for commit 452f18a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.5 KiB	+152 B (+1.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+6 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+146 B (+2.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	57.0 KiB	+1.4 KiB (+2.6%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	+27 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−6 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+33 B (+0.5%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.4 KiB	+44 B (+0.1%)	66.4 KiB	✅
Claude	Live turn messages	9	10	+1 (+11.1%)	21	✅

Baseline: 7751299 · PR result: 452f18a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a cross-client media-preview redesign that changes playback, asset URL minting, sharing, file-link parsing, and action menus across shared runtime, web, and mobile production paths. The new behavior and infrastructure span many existing surfaces, exceeding the scope of an auto-approvable refactor.

Not approved because:

• Monthly spending limit reached (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 05b0284. Configure here.}

[macroscopeapp]

🟡 Mediumsrc/terminal-links.ts:251

Activating a POSIX path ending in :0, such as logs/output:0, resolves it to <cwd>/logs/output instead of the actual file <cwd>/logs/output:0. splitFilePathPosition drops the non-positive line suffix and formatFilePathPosition does not restore it; preserve the suffix as part of the path when no valid line is parsed.

- const { path } = position;+ const path =+ position.line === undefined && rawPath.endsWith(":0") ? rawPath : position.path;

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/terminal-links.ts around line 251:
Activating a POSIX path ending in `:0`, such as `logs/output:0`, resolves it to `<cwd>/logs/output` instead of the actual file `<cwd>/logs/output:0`. `splitFilePathPosition` drops the non-positive line suffix and `formatFilePathPosition` does not restore it; preserve the suffix as part of the path when no valid line is parsed.

[macroscopeapp]

🟠 Highcomponents/VideoPreviewModal.tsx:94

Save/share can fail after the modal closes because useLocalPlayback exposes only the resolved uri, so useMediaActions takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft attachment through the share path (or otherwise retain it for the operation), as AttachmentPreviewFile.share does.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/mobile/src/components/VideoPreviewModal.tsx around line 94:
Save/share can fail after the modal closes because `useLocalPlayback` exposes only the resolved `uri`, so `useMediaActions` takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft `attachment` through the share path (or otherwise retain it for the operation), as `AttachmentPreviewFile.share` does.

[macroscopeapp]

🟡 Mediumcomponents/ChatMarkdown.tsx:2621

Workspace-backed images rendered through ChatMarkdownAssetImage no longer open the full-size preview when clicked or activated by keyboard. The onImageExpand={imageExpand} prop was removed from ChatMarkdownVideo, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

 style={authoredSizeStyle}
+ onImageExpand={imageExpand}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/components/ChatMarkdown.tsx around line 2621:
Workspace-backed images rendered through `ChatMarkdownAssetImage` no longer open the full-size preview when clicked or activated by keyboard. The `onImageExpand={imageExpand}` prop was removed from `ChatMarkdownVideo`, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

[macroscopeapp]

Dropping onImageExpand here (and MediaVideoPlayer's onExpand) removes the only route from a markdown video embed to the expanded dialog, while the sibling image embed still expands on click (expandableMarkdownImageProps + cursor-zoom-in, same component). Inline embeds are capped by CHAT_MARKDOWN_MEDIA_MAX_WIDTH_CLASS_NAME, so a video's only remaining enlargement is the browser's own fullscreen button — and it also loses the dialog-level media actions that images keep. docs/user/composer.md (line 100, in a paragraph that covers web, desktop, and mobile) still states "video embeds show a player with controls and an option to expand", and both platforms lost that control in this PR.

If the removal is intentional, updating that doc sentence is the smallest fix; otherwise consider keeping the expand handler for video embeds so images and videos behave the same inline.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Attachment videos now mint their URL inside this dialog, but sourceFailed only reflects Failure. While the environment is reconnecting (connecting/backoff, or connected with no session yet), assetEnvironment.createUrl resolves to Effect.never (client-runtime state/runtime.ts), so assetUrl stays Loading and the dialog paints the black stateClassName placeholder with no message, no retry, and no end state until the connection comes back. Before this change the same click went through downloadFileAttachment, which pre-checked readPreparedConnection and toasted "The environment is not connected.", and the mobile player added in this PR (useMediaPlayback) folds connection._tag === "None" into unavailable for exactly this case — so web is now the only surface with no disconnected end state.

Smallest fix is to treat a missing prepared connection as a failure here (needs usePreparedConnection from ../../state/session), which also gives the panel its Retry button:

constconnection=usePreparedConnection(asset?.environmentId??null);// ...sourceFailed={assetUrl._tag==="Failure"||(asset!==undefined&&connection._tag==="None")}

Keeping the connection pre-check in openFileAttachment before setExpandedImage would work too.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

🟢 Lowinternals/mobile-navigation.md:95

The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, OpenVideoPreviewModal renders MediaActionsMenu with inModal in the header and exposes a Save or share video button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

-Media actions (copy path, open in file viewer, save or share) belong to the surface that-opened the video, a long-press on the thumbnail, so the player carries no-menu. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.+On iOS, media actions (copy path, open in file viewer, save or share) belong to the+surface that opened the video, a long-press on the thumbnail, so the player carries no+menu. Android exposes `MediaActionsMenu` and a `Save or share video` button in the modal. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @docs/internals/mobile-navigation.md around lines 95-97:
The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, `OpenVideoPreviewModal` renders `MediaActionsMenu` with `inModal` in the header and exposes a `Save or share video` button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

[macroscopeapp]

This tile is labelled Play ${file.name}, but on this branch of the handler it no longer plays. onFileOpen is ChatView's openFileAttachment, which routes only html/pdf to the panel and otherwise calls downloadFileAttachment — and this PR removed that function's video branch (the one that used to setExpandedImage({ images: [{ src: url, name, type: "video" }] })). So for a rehydrated draft (file.file === null with an uploadedAttachmentId, the normal state after a reload) clicking the play glyph now silently saves the file to disk instead of opening the player. The removal of the aria-busy/"Loading…" state on these lines assumed the click still ends in the expanded dialog, and timeline videos moved to MediaVideoPlayer, so this is the only surface left on the old play path.

Smallest fix is to open the dialog here instead of downloading: build the item with src: null and actionsSource: { kind: "video", name: file.name, src: null, asset: buildAttachmentVideoAsset(file.uploadEnvironmentId ?? environmentId, { ...file, id: file.uploadedAttachmentId }) } and pass it to onExpandImage — ExpandedVideo already mints and plays that shape. If downloading is the intended behavior for this state, the control should say so rather than "Play".

^{Posted via Macroscope — UI Consistency}

[juliusmarminge]

What changed

• Moved markdown file-link parsing, media-source resolution, signed URL state, and media-action availability into client-runtime so web and mobile use the same rules.
• Routed expanded web videos through the shared streaming player, including URL refresh and retry.
• Routed every iOS video through AVKit and every iOS image, PDF, and HTML preview through Quick Look.
• Moved media actions to a long-press on the thumbnail and removed the redundant in-player controls.

Native long-press menus on markdown file chips are in #9258.

Why

The web and mobile clients had separate path parsers, media resolvers, URL lifecycle code, and viewer paths. The copies had already drifted. Some videos downloaded before playback, expired URLs left expanded viewers stuck, and iOS media sometimes opened in React Native viewers instead of AVKit or Quick Look.

This puts the shared decisions in one package and leaves each client responsible only for presentation. On iOS, native viewers own media playback and document preview.

UI changes

Before: clipped initial video controls	After: AVKit controls fill the thumbnail

Pre-fix tap and long-press race reproduced on device: screen recording

Testing

• 40 focused mobile file, markdown, and media tests
• Mobile TypeScript typecheck
• TypeScript lint and formatting checks for changed files
• Native Swift and Kotlin static checks
• Physical iPhone 16 Pro pass against a disposable LAN environment, covering workspace and external files, images, videos, HTML, PDFs, attachments, composer drafts, markdown embeds, thumbnail long-press menus, AVKit, and Quick Look

Checklist

• This PR is focused on one media-preview refactor (net negative after splitting out feat(mobile): long-press file references for path and open actions #9258)
• I explained what changed and why
• I included before/after screenshots for UI changes
• I included a recording for interaction changes

Created with gpt-5.6-sol using the Codex harness in T3 Code.

Note

Unify file and media preview parsing and playback across web and mobile

• Adds shared markdown file-link and media-source resolvers in client-runtime (parseMarkdownFileLink, resolveMediaSource) and replaces local parsing implementations in web, mobile, and the t3-markdown module with these shared helpers
• Reworks video previews on both platforms to use the shared MediaVideoPlayer; mobile removes the Expo-based VideoPlayback and MediaVideoPreviewModal, and web replaces raw <video> rendering in ExpandedImageDialog, MessagesTimeline, and ChatMarkdown with the shared player
• Moves AssetUrlState and assetUrlStateFromResult into client-runtime; web and mobile asset URL hooks now derive loading/failure/success through the shared mapper instead of local inline checks
• Unifies media action menus via a shared MediaActionId union and MediaActionsMenu/useMediaActions; both platforms now expose save/share/open-file actions through the same identifiers and long-press surfaces
• Risk: resolveViewedImageAsset and ViewedImageAsset now only return media-file resources, removing the attachment-resource variant — callers expecting attachment resources from viewed workspace images will break; iOS FilePreview always uses NativeFilePreview and no longer renders MediaImagePreview for actionable images; MediaVideoPlayer no longer renders an expand/maximize control or accepts an onExpand callback

^{Macroscope summarized 452f18a.}

---

Note

Medium Risk
Broad cross-platform media and preview behavior changes (iOS native viewers, inline vs modal video, shared URL lifecycle) touch high-traffic chat and file flows, though logic is consolidated and covered by focused tests.

Overview
Centralizes markdown file-link parsing, media-source classification, signed asset URL state, and media action IDs in client-runtime, then rewires web and mobile to call those helpers instead of duplicated local heuristics.

Mobile drops bespoke link/path code in favor of parseMarkdownFileLink and resolveMediaSource. Video and attachment UX converges on useMediaActions / MediaActionsMenu (long-press on thumbnails); VideoAttachmentMenu, MediaVideoPreviewModal, and the workspace image prefetch atom are removed. iOS routes all full-screen file previews through Quick Look and full-screen videos through AVKit (with URL re-mint on failure); inline/workspace video stays on Expo Video. Composer and thread attachments share attachmentVideoPreviewSource and attachment-scoped asset resources.

Web plays timeline and markdown videos inline via the shared MediaVideoPlayer (no separate “open video” flow or expand control); expanded dialogs stream through the same player with asset refresh. File attachment clicks no longer auto-open a modal for videos. Context menus align action IDs (save, copy-full-path, etc.) and expose save for videos as well as images.

Docs are updated to describe streaming, long-press save/share, and platform-specific players.

^{Reviewed by Cursor Bugbot for commit 452f18a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.5 KiB	+152 B (+1.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+6 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+146 B (+2.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	57.0 KiB	+1.4 KiB (+2.6%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	+27 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−6 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+33 B (+0.5%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.4 KiB	+44 B (+0.1%)	66.4 KiB	✅
Claude	Live turn messages	9	10	+1 (+11.1%)	21	✅

Baseline: 7751299 · PR result: 452f18a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a cross-client media-preview redesign that changes playback, asset URL minting, sharing, file-link parsing, and action menus across shared runtime, web, and mobile production paths. The new behavior and infrastructure span many existing surfaces, exceeding the scope of an auto-approvable refactor.

Not approved because:

• Monthly spending limit reached (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 05b0284. Configure here.}

[macroscopeapp]

🟡 Mediumsrc/terminal-links.ts:251

Activating a POSIX path ending in :0, such as logs/output:0, resolves it to <cwd>/logs/output instead of the actual file <cwd>/logs/output:0. splitFilePathPosition drops the non-positive line suffix and formatFilePathPosition does not restore it; preserve the suffix as part of the path when no valid line is parsed.

- const { path } = position;+ const path =+ position.line === undefined && rawPath.endsWith(":0") ? rawPath : position.path;

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/terminal-links.ts around line 251:
Activating a POSIX path ending in `:0`, such as `logs/output:0`, resolves it to `<cwd>/logs/output` instead of the actual file `<cwd>/logs/output:0`. `splitFilePathPosition` drops the non-positive line suffix and `formatFilePathPosition` does not restore it; preserve the suffix as part of the path when no valid line is parsed.

[macroscopeapp]

🟠 Highcomponents/VideoPreviewModal.tsx:94

Save/share can fail after the modal closes because useLocalPlayback exposes only the resolved uri, so useMediaActions takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft attachment through the share path (or otherwise retain it for the operation), as AttachmentPreviewFile.share does.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/mobile/src/components/VideoPreviewModal.tsx around line 94:
Save/share can fail after the modal closes because `useLocalPlayback` exposes only the resolved `uri`, so `useMediaActions` takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft `attachment` through the share path (or otherwise retain it for the operation), as `AttachmentPreviewFile.share` does.

[macroscopeapp]

🟡 Mediumcomponents/ChatMarkdown.tsx:2621

Workspace-backed images rendered through ChatMarkdownAssetImage no longer open the full-size preview when clicked or activated by keyboard. The onImageExpand={imageExpand} prop was removed from ChatMarkdownVideo, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

 style={authoredSizeStyle}
+ onImageExpand={imageExpand}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/components/ChatMarkdown.tsx around line 2621:
Workspace-backed images rendered through `ChatMarkdownAssetImage` no longer open the full-size preview when clicked or activated by keyboard. The `onImageExpand={imageExpand}` prop was removed from `ChatMarkdownVideo`, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

[macroscopeapp]

Dropping onImageExpand here (and MediaVideoPlayer's onExpand) removes the only route from a markdown video embed to the expanded dialog, while the sibling image embed still expands on click (expandableMarkdownImageProps + cursor-zoom-in, same component). Inline embeds are capped by CHAT_MARKDOWN_MEDIA_MAX_WIDTH_CLASS_NAME, so a video's only remaining enlargement is the browser's own fullscreen button — and it also loses the dialog-level media actions that images keep. docs/user/composer.md (line 100, in a paragraph that covers web, desktop, and mobile) still states "video embeds show a player with controls and an option to expand", and both platforms lost that control in this PR.

If the removal is intentional, updating that doc sentence is the smallest fix; otherwise consider keeping the expand handler for video embeds so images and videos behave the same inline.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Attachment videos now mint their URL inside this dialog, but sourceFailed only reflects Failure. While the environment is reconnecting (connecting/backoff, or connected with no session yet), assetEnvironment.createUrl resolves to Effect.never (client-runtime state/runtime.ts), so assetUrl stays Loading and the dialog paints the black stateClassName placeholder with no message, no retry, and no end state until the connection comes back. Before this change the same click went through downloadFileAttachment, which pre-checked readPreparedConnection and toasted "The environment is not connected.", and the mobile player added in this PR (useMediaPlayback) folds connection._tag === "None" into unavailable for exactly this case — so web is now the only surface with no disconnected end state.

Smallest fix is to treat a missing prepared connection as a failure here (needs usePreparedConnection from ../../state/session), which also gives the panel its Retry button:

constconnection=usePreparedConnection(asset?.environmentId??null);// ...sourceFailed={assetUrl._tag==="Failure"||(asset!==undefined&&connection._tag==="None")}

Keeping the connection pre-check in openFileAttachment before setExpandedImage would work too.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

🟢 Lowinternals/mobile-navigation.md:95

The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, OpenVideoPreviewModal renders MediaActionsMenu with inModal in the header and exposes a Save or share video button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

-Media actions (copy path, open in file viewer, save or share) belong to the surface that-opened the video, a long-press on the thumbnail, so the player carries no-menu. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.+On iOS, media actions (copy path, open in file viewer, save or share) belong to the+surface that opened the video, a long-press on the thumbnail, so the player carries no+menu. Android exposes `MediaActionsMenu` and a `Save or share video` button in the modal. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @docs/internals/mobile-navigation.md around lines 95-97:
The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, `OpenVideoPreviewModal` renders `MediaActionsMenu` with `inModal` in the header and exposes a `Save or share video` button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

[macroscopeapp]

This tile is labelled Play ${file.name}, but on this branch of the handler it no longer plays. onFileOpen is ChatView's openFileAttachment, which routes only html/pdf to the panel and otherwise calls downloadFileAttachment — and this PR removed that function's video branch (the one that used to setExpandedImage({ images: [{ src: url, name, type: "video" }] })). So for a rehydrated draft (file.file === null with an uploadedAttachmentId, the normal state after a reload) clicking the play glyph now silently saves the file to disk instead of opening the player. The removal of the aria-busy/"Loading…" state on these lines assumed the click still ends in the expanded dialog, and timeline videos moved to MediaVideoPlayer, so this is the only surface left on the old play path.

Smallest fix is to open the dialog here instead of downloading: build the item with src: null and actionsSource: { kind: "video", name: file.name, src: null, asset: buildAttachmentVideoAsset(file.uploadEnvironmentId ?? environmentId, { ...file, id: file.uploadedAttachmentId }) } and pass it to onExpandImage — ExpandedVideo already mints and plays that shape. If downloading is the intended behavior for this state, the control should say so rather than "Play".

^{Posted via Macroscope — UI Consistency}

[juliusmarminge]

What changed

• Moved markdown file-link parsing, media-source resolution, signed URL state, and media-action availability into client-runtime so web and mobile use the same rules.
• Routed expanded web videos through the shared streaming player, including URL refresh and retry.
• Routed every iOS video through AVKit and every iOS image, PDF, and HTML preview through Quick Look.
• Moved media actions to a long-press on the thumbnail and removed the redundant in-player controls.

Native long-press menus on markdown file chips are in #9258.

Why

The web and mobile clients had separate path parsers, media resolvers, URL lifecycle code, and viewer paths. The copies had already drifted. Some videos downloaded before playback, expired URLs left expanded viewers stuck, and iOS media sometimes opened in React Native viewers instead of AVKit or Quick Look.

This puts the shared decisions in one package and leaves each client responsible only for presentation. On iOS, native viewers own media playback and document preview.

UI changes

Before: clipped initial video controls	After: AVKit controls fill the thumbnail

Pre-fix tap and long-press race reproduced on device: screen recording

Testing

• 40 focused mobile file, markdown, and media tests
• Mobile TypeScript typecheck
• TypeScript lint and formatting checks for changed files
• Native Swift and Kotlin static checks
• Physical iPhone 16 Pro pass against a disposable LAN environment, covering workspace and external files, images, videos, HTML, PDFs, attachments, composer drafts, markdown embeds, thumbnail long-press menus, AVKit, and Quick Look

Checklist

• This PR is focused on one media-preview refactor (net negative after splitting out feat(mobile): long-press file references for path and open actions #9258)
• I explained what changed and why
• I included before/after screenshots for UI changes
• I included a recording for interaction changes

Created with gpt-5.6-sol using the Codex harness in T3 Code.

Note

Unify file and media preview parsing and playback across web and mobile

• Adds shared markdown file-link and media-source resolvers in client-runtime (parseMarkdownFileLink, resolveMediaSource) and replaces local parsing implementations in web, mobile, and the t3-markdown module with these shared helpers
• Reworks video previews on both platforms to use the shared MediaVideoPlayer; mobile removes the Expo-based VideoPlayback and MediaVideoPreviewModal, and web replaces raw <video> rendering in ExpandedImageDialog, MessagesTimeline, and ChatMarkdown with the shared player
• Moves AssetUrlState and assetUrlStateFromResult into client-runtime; web and mobile asset URL hooks now derive loading/failure/success through the shared mapper instead of local inline checks
• Unifies media action menus via a shared MediaActionId union and MediaActionsMenu/useMediaActions; both platforms now expose save/share/open-file actions through the same identifiers and long-press surfaces
• Risk: resolveViewedImageAsset and ViewedImageAsset now only return media-file resources, removing the attachment-resource variant — callers expecting attachment resources from viewed workspace images will break; iOS FilePreview always uses NativeFilePreview and no longer renders MediaImagePreview for actionable images; MediaVideoPlayer no longer renders an expand/maximize control or accepts an onExpand callback

^{Macroscope summarized 452f18a.}

---

Note

Medium Risk
Broad cross-platform media and preview behavior changes (iOS native viewers, inline vs modal video, shared URL lifecycle) touch high-traffic chat and file flows, though logic is consolidated and covered by focused tests.

Overview
Centralizes markdown file-link parsing, media-source classification, signed asset URL state, and media action IDs in client-runtime, then rewires web and mobile to call those helpers instead of duplicated local heuristics.

Mobile drops bespoke link/path code in favor of parseMarkdownFileLink and resolveMediaSource. Video and attachment UX converges on useMediaActions / MediaActionsMenu (long-press on thumbnails); VideoAttachmentMenu, MediaVideoPreviewModal, and the workspace image prefetch atom are removed. iOS routes all full-screen file previews through Quick Look and full-screen videos through AVKit (with URL re-mint on failure); inline/workspace video stays on Expo Video. Composer and thread attachments share attachmentVideoPreviewSource and attachment-scoped asset resources.

Web plays timeline and markdown videos inline via the shared MediaVideoPlayer (no separate “open video” flow or expand control); expanded dialogs stream through the same player with asset refresh. File attachment clicks no longer auto-open a modal for videos. Context menus align action IDs (save, copy-full-path, etc.) and expose save for videos as well as images.

Docs are updated to describe streaming, long-press save/share, and platform-specific players.

^{Reviewed by Cursor Bugbot for commit 452f18a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.5 KiB	+152 B (+1.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+6 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+146 B (+2.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	57.0 KiB	+1.4 KiB (+2.6%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	+27 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−6 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+33 B (+0.5%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.4 KiB	+44 B (+0.1%)	66.4 KiB	✅
Claude	Live turn messages	9	10	+1 (+11.1%)	21	✅

Baseline: 7751299 · PR result: 452f18a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a cross-client media-preview redesign that changes playback, asset URL minting, sharing, file-link parsing, and action menus across shared runtime, web, and mobile production paths. The new behavior and infrastructure span many existing surfaces, exceeding the scope of an auto-approvable refactor.

Not approved because:

• Monthly spending limit reached (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 05b0284. Configure here.}

[macroscopeapp]

🟡 Mediumsrc/terminal-links.ts:251

Activating a POSIX path ending in :0, such as logs/output:0, resolves it to <cwd>/logs/output instead of the actual file <cwd>/logs/output:0. splitFilePathPosition drops the non-positive line suffix and formatFilePathPosition does not restore it; preserve the suffix as part of the path when no valid line is parsed.

- const { path } = position;+ const path =+ position.line === undefined && rawPath.endsWith(":0") ? rawPath : position.path;

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/terminal-links.ts around line 251:
Activating a POSIX path ending in `:0`, such as `logs/output:0`, resolves it to `<cwd>/logs/output` instead of the actual file `<cwd>/logs/output:0`. `splitFilePathPosition` drops the non-positive line suffix and `formatFilePathPosition` does not restore it; preserve the suffix as part of the path when no valid line is parsed.

[macroscopeapp]

🟠 Highcomponents/VideoPreviewModal.tsx:94

Save/share can fail after the modal closes because useLocalPlayback exposes only the resolved uri, so useMediaActions takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft attachment through the share path (or otherwise retain it for the operation), as AttachmentPreviewFile.share does.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/mobile/src/components/VideoPreviewModal.tsx around line 94:
Save/share can fail after the modal closes because `useLocalPlayback` exposes only the resolved `uri`, so `useMediaActions` takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft `attachment` through the share path (or otherwise retain it for the operation), as `AttachmentPreviewFile.share` does.

[macroscopeapp]

🟡 Mediumcomponents/ChatMarkdown.tsx:2621

Workspace-backed images rendered through ChatMarkdownAssetImage no longer open the full-size preview when clicked or activated by keyboard. The onImageExpand={imageExpand} prop was removed from ChatMarkdownVideo, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

 style={authoredSizeStyle}
+ onImageExpand={imageExpand}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/components/ChatMarkdown.tsx around line 2621:
Workspace-backed images rendered through `ChatMarkdownAssetImage` no longer open the full-size preview when clicked or activated by keyboard. The `onImageExpand={imageExpand}` prop was removed from `ChatMarkdownVideo`, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

[macroscopeapp]

Dropping onImageExpand here (and MediaVideoPlayer's onExpand) removes the only route from a markdown video embed to the expanded dialog, while the sibling image embed still expands on click (expandableMarkdownImageProps + cursor-zoom-in, same component). Inline embeds are capped by CHAT_MARKDOWN_MEDIA_MAX_WIDTH_CLASS_NAME, so a video's only remaining enlargement is the browser's own fullscreen button — and it also loses the dialog-level media actions that images keep. docs/user/composer.md (line 100, in a paragraph that covers web, desktop, and mobile) still states "video embeds show a player with controls and an option to expand", and both platforms lost that control in this PR.

If the removal is intentional, updating that doc sentence is the smallest fix; otherwise consider keeping the expand handler for video embeds so images and videos behave the same inline.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Attachment videos now mint their URL inside this dialog, but sourceFailed only reflects Failure. While the environment is reconnecting (connecting/backoff, or connected with no session yet), assetEnvironment.createUrl resolves to Effect.never (client-runtime state/runtime.ts), so assetUrl stays Loading and the dialog paints the black stateClassName placeholder with no message, no retry, and no end state until the connection comes back. Before this change the same click went through downloadFileAttachment, which pre-checked readPreparedConnection and toasted "The environment is not connected.", and the mobile player added in this PR (useMediaPlayback) folds connection._tag === "None" into unavailable for exactly this case — so web is now the only surface with no disconnected end state.

Smallest fix is to treat a missing prepared connection as a failure here (needs usePreparedConnection from ../../state/session), which also gives the panel its Retry button:

constconnection=usePreparedConnection(asset?.environmentId??null);// ...sourceFailed={assetUrl._tag==="Failure"||(asset!==undefined&&connection._tag==="None")}

Keeping the connection pre-check in openFileAttachment before setExpandedImage would work too.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

🟢 Lowinternals/mobile-navigation.md:95

The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, OpenVideoPreviewModal renders MediaActionsMenu with inModal in the header and exposes a Save or share video button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

-Media actions (copy path, open in file viewer, save or share) belong to the surface that-opened the video, a long-press on the thumbnail, so the player carries no-menu. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.+On iOS, media actions (copy path, open in file viewer, save or share) belong to the+surface that opened the video, a long-press on the thumbnail, so the player carries no+menu. Android exposes `MediaActionsMenu` and a `Save or share video` button in the modal. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @docs/internals/mobile-navigation.md around lines 95-97:
The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, `OpenVideoPreviewModal` renders `MediaActionsMenu` with `inModal` in the header and exposes a `Save or share video` button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

[macroscopeapp]

This tile is labelled Play ${file.name}, but on this branch of the handler it no longer plays. onFileOpen is ChatView's openFileAttachment, which routes only html/pdf to the panel and otherwise calls downloadFileAttachment — and this PR removed that function's video branch (the one that used to setExpandedImage({ images: [{ src: url, name, type: "video" }] })). So for a rehydrated draft (file.file === null with an uploadedAttachmentId, the normal state after a reload) clicking the play glyph now silently saves the file to disk instead of opening the player. The removal of the aria-busy/"Loading…" state on these lines assumed the click still ends in the expanded dialog, and timeline videos moved to MediaVideoPlayer, so this is the only surface left on the old play path.

Smallest fix is to open the dialog here instead of downloading: build the item with src: null and actionsSource: { kind: "video", name: file.name, src: null, asset: buildAttachmentVideoAsset(file.uploadEnvironmentId ?? environmentId, { ...file, id: file.uploadedAttachmentId }) } and pass it to onExpandImage — ExpandedVideo already mints and plays that shape. If downloading is the intended behavior for this state, the control should say so rather than "Play".

^{Posted via Macroscope — UI Consistency}

[juliusmarminge]

What changed

• Moved markdown file-link parsing, media-source resolution, signed URL state, and media-action availability into client-runtime so web and mobile use the same rules.
• Routed expanded web videos through the shared streaming player, including URL refresh and retry.
• Routed every iOS video through AVKit and every iOS image, PDF, and HTML preview through Quick Look.
• Moved media actions to a long-press on the thumbnail and removed the redundant in-player controls.

Native long-press menus on markdown file chips are in #9258.

Why

The web and mobile clients had separate path parsers, media resolvers, URL lifecycle code, and viewer paths. The copies had already drifted. Some videos downloaded before playback, expired URLs left expanded viewers stuck, and iOS media sometimes opened in React Native viewers instead of AVKit or Quick Look.

This puts the shared decisions in one package and leaves each client responsible only for presentation. On iOS, native viewers own media playback and document preview.

UI changes

Before: clipped initial video controls	After: AVKit controls fill the thumbnail

Pre-fix tap and long-press race reproduced on device: screen recording

Testing

• 40 focused mobile file, markdown, and media tests
• Mobile TypeScript typecheck
• TypeScript lint and formatting checks for changed files
• Native Swift and Kotlin static checks
• Physical iPhone 16 Pro pass against a disposable LAN environment, covering workspace and external files, images, videos, HTML, PDFs, attachments, composer drafts, markdown embeds, thumbnail long-press menus, AVKit, and Quick Look

Checklist

• This PR is focused on one media-preview refactor (net negative after splitting out feat(mobile): long-press file references for path and open actions #9258)
• I explained what changed and why
• I included before/after screenshots for UI changes
• I included a recording for interaction changes

Created with gpt-5.6-sol using the Codex harness in T3 Code.

Note

Unify file and media preview parsing and playback across web and mobile

• Adds shared markdown file-link and media-source resolvers in client-runtime (parseMarkdownFileLink, resolveMediaSource) and replaces local parsing implementations in web, mobile, and the t3-markdown module with these shared helpers
• Reworks video previews on both platforms to use the shared MediaVideoPlayer; mobile removes the Expo-based VideoPlayback and MediaVideoPreviewModal, and web replaces raw <video> rendering in ExpandedImageDialog, MessagesTimeline, and ChatMarkdown with the shared player
• Moves AssetUrlState and assetUrlStateFromResult into client-runtime; web and mobile asset URL hooks now derive loading/failure/success through the shared mapper instead of local inline checks
• Unifies media action menus via a shared MediaActionId union and MediaActionsMenu/useMediaActions; both platforms now expose save/share/open-file actions through the same identifiers and long-press surfaces
• Risk: resolveViewedImageAsset and ViewedImageAsset now only return media-file resources, removing the attachment-resource variant — callers expecting attachment resources from viewed workspace images will break; iOS FilePreview always uses NativeFilePreview and no longer renders MediaImagePreview for actionable images; MediaVideoPlayer no longer renders an expand/maximize control or accepts an onExpand callback

^{Macroscope summarized 452f18a.}

---

Note

Medium Risk
Broad cross-platform media and preview behavior changes (iOS native viewers, inline vs modal video, shared URL lifecycle) touch high-traffic chat and file flows, though logic is consolidated and covered by focused tests.

Overview
Centralizes markdown file-link parsing, media-source classification, signed asset URL state, and media action IDs in client-runtime, then rewires web and mobile to call those helpers instead of duplicated local heuristics.

Mobile drops bespoke link/path code in favor of parseMarkdownFileLink and resolveMediaSource. Video and attachment UX converges on useMediaActions / MediaActionsMenu (long-press on thumbnails); VideoAttachmentMenu, MediaVideoPreviewModal, and the workspace image prefetch atom are removed. iOS routes all full-screen file previews through Quick Look and full-screen videos through AVKit (with URL re-mint on failure); inline/workspace video stays on Expo Video. Composer and thread attachments share attachmentVideoPreviewSource and attachment-scoped asset resources.

Web plays timeline and markdown videos inline via the shared MediaVideoPlayer (no separate “open video” flow or expand control); expanded dialogs stream through the same player with asset refresh. File attachment clicks no longer auto-open a modal for videos. Context menus align action IDs (save, copy-full-path, etc.) and expose save for videos as well as images.

Docs are updated to describe streaming, long-press save/share, and platform-specific players.

^{Reviewed by Cursor Bugbot for commit 452f18a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.5 KiB	+152 B (+1.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+6 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+146 B (+2.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	57.0 KiB	+1.4 KiB (+2.6%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	+27 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−6 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+33 B (+0.5%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.4 KiB	+44 B (+0.1%)	66.4 KiB	✅
Claude	Live turn messages	9	10	+1 (+11.1%)	21	✅

Baseline: 7751299 · PR result: 452f18a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a cross-client media-preview redesign that changes playback, asset URL minting, sharing, file-link parsing, and action menus across shared runtime, web, and mobile production paths. The new behavior and infrastructure span many existing surfaces, exceeding the scope of an auto-approvable refactor.

Not approved because:

• Monthly spending limit reached (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 05b0284. Configure here.}

[macroscopeapp]

🟡 Mediumsrc/terminal-links.ts:251

Activating a POSIX path ending in :0, such as logs/output:0, resolves it to <cwd>/logs/output instead of the actual file <cwd>/logs/output:0. splitFilePathPosition drops the non-positive line suffix and formatFilePathPosition does not restore it; preserve the suffix as part of the path when no valid line is parsed.

- const { path } = position;+ const path =+ position.line === undefined && rawPath.endsWith(":0") ? rawPath : position.path;

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/terminal-links.ts around line 251:
Activating a POSIX path ending in `:0`, such as `logs/output:0`, resolves it to `<cwd>/logs/output` instead of the actual file `<cwd>/logs/output:0`. `splitFilePathPosition` drops the non-positive line suffix and `formatFilePathPosition` does not restore it; preserve the suffix as part of the path when no valid line is parsed.

[macroscopeapp]

🟠 Highcomponents/VideoPreviewModal.tsx:94

Save/share can fail after the modal closes because useLocalPlayback exposes only the resolved uri, so useMediaActions takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft attachment through the share path (or otherwise retain it for the operation), as AttachmentPreviewFile.share does.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/mobile/src/components/VideoPreviewModal.tsx around line 94:
Save/share can fail after the modal closes because `useLocalPlayback` exposes only the resolved `uri`, so `useMediaActions` takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft `attachment` through the share path (or otherwise retain it for the operation), as `AttachmentPreviewFile.share` does.

[macroscopeapp]

🟡 Mediumcomponents/ChatMarkdown.tsx:2621

Workspace-backed images rendered through ChatMarkdownAssetImage no longer open the full-size preview when clicked or activated by keyboard. The onImageExpand={imageExpand} prop was removed from ChatMarkdownVideo, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

 style={authoredSizeStyle}
+ onImageExpand={imageExpand}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/components/ChatMarkdown.tsx around line 2621:
Workspace-backed images rendered through `ChatMarkdownAssetImage` no longer open the full-size preview when clicked or activated by keyboard. The `onImageExpand={imageExpand}` prop was removed from `ChatMarkdownVideo`, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

[macroscopeapp]

Dropping onImageExpand here (and MediaVideoPlayer's onExpand) removes the only route from a markdown video embed to the expanded dialog, while the sibling image embed still expands on click (expandableMarkdownImageProps + cursor-zoom-in, same component). Inline embeds are capped by CHAT_MARKDOWN_MEDIA_MAX_WIDTH_CLASS_NAME, so a video's only remaining enlargement is the browser's own fullscreen button — and it also loses the dialog-level media actions that images keep. docs/user/composer.md (line 100, in a paragraph that covers web, desktop, and mobile) still states "video embeds show a player with controls and an option to expand", and both platforms lost that control in this PR.

If the removal is intentional, updating that doc sentence is the smallest fix; otherwise consider keeping the expand handler for video embeds so images and videos behave the same inline.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Attachment videos now mint their URL inside this dialog, but sourceFailed only reflects Failure. While the environment is reconnecting (connecting/backoff, or connected with no session yet), assetEnvironment.createUrl resolves to Effect.never (client-runtime state/runtime.ts), so assetUrl stays Loading and the dialog paints the black stateClassName placeholder with no message, no retry, and no end state until the connection comes back. Before this change the same click went through downloadFileAttachment, which pre-checked readPreparedConnection and toasted "The environment is not connected.", and the mobile player added in this PR (useMediaPlayback) folds connection._tag === "None" into unavailable for exactly this case — so web is now the only surface with no disconnected end state.

Smallest fix is to treat a missing prepared connection as a failure here (needs usePreparedConnection from ../../state/session), which also gives the panel its Retry button:

constconnection=usePreparedConnection(asset?.environmentId??null);// ...sourceFailed={assetUrl._tag==="Failure"||(asset!==undefined&&connection._tag==="None")}

Keeping the connection pre-check in openFileAttachment before setExpandedImage would work too.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

🟢 Lowinternals/mobile-navigation.md:95

The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, OpenVideoPreviewModal renders MediaActionsMenu with inModal in the header and exposes a Save or share video button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

-Media actions (copy path, open in file viewer, save or share) belong to the surface that-opened the video, a long-press on the thumbnail, so the player carries no-menu. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.+On iOS, media actions (copy path, open in file viewer, save or share) belong to the+surface that opened the video, a long-press on the thumbnail, so the player carries no+menu. Android exposes `MediaActionsMenu` and a `Save or share video` button in the modal. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @docs/internals/mobile-navigation.md around lines 95-97:
The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, `OpenVideoPreviewModal` renders `MediaActionsMenu` with `inModal` in the header and exposes a `Save or share video` button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

[macroscopeapp]

This tile is labelled Play ${file.name}, but on this branch of the handler it no longer plays. onFileOpen is ChatView's openFileAttachment, which routes only html/pdf to the panel and otherwise calls downloadFileAttachment — and this PR removed that function's video branch (the one that used to setExpandedImage({ images: [{ src: url, name, type: "video" }] })). So for a rehydrated draft (file.file === null with an uploadedAttachmentId, the normal state after a reload) clicking the play glyph now silently saves the file to disk instead of opening the player. The removal of the aria-busy/"Loading…" state on these lines assumed the click still ends in the expanded dialog, and timeline videos moved to MediaVideoPlayer, so this is the only surface left on the old play path.

Smallest fix is to open the dialog here instead of downloading: build the item with src: null and actionsSource: { kind: "video", name: file.name, src: null, asset: buildAttachmentVideoAsset(file.uploadEnvironmentId ?? environmentId, { ...file, id: file.uploadedAttachmentId }) } and pass it to onExpandImage — ExpandedVideo already mints and plays that shape. If downloading is the intended behavior for this state, the control should say so rather than "Play".

^{Posted via Macroscope — UI Consistency}

[juliusmarminge]

What changed

• Moved markdown file-link parsing, media-source resolution, signed URL state, and media-action availability into client-runtime so web and mobile use the same rules.
• Routed expanded web videos through the shared streaming player, including URL refresh and retry.
• Routed every iOS video through AVKit and every iOS image, PDF, and HTML preview through Quick Look.
• Moved media actions to a long-press on the thumbnail and removed the redundant in-player controls.

Native long-press menus on markdown file chips are in #9258.

Why

The web and mobile clients had separate path parsers, media resolvers, URL lifecycle code, and viewer paths. The copies had already drifted. Some videos downloaded before playback, expired URLs left expanded viewers stuck, and iOS media sometimes opened in React Native viewers instead of AVKit or Quick Look.

This puts the shared decisions in one package and leaves each client responsible only for presentation. On iOS, native viewers own media playback and document preview.

UI changes

Before: clipped initial video controls	After: AVKit controls fill the thumbnail

Pre-fix tap and long-press race reproduced on device: screen recording

Testing

• 40 focused mobile file, markdown, and media tests
• Mobile TypeScript typecheck
• TypeScript lint and formatting checks for changed files
• Native Swift and Kotlin static checks
• Physical iPhone 16 Pro pass against a disposable LAN environment, covering workspace and external files, images, videos, HTML, PDFs, attachments, composer drafts, markdown embeds, thumbnail long-press menus, AVKit, and Quick Look

Checklist

• This PR is focused on one media-preview refactor (net negative after splitting out feat(mobile): long-press file references for path and open actions #9258)
• I explained what changed and why
• I included before/after screenshots for UI changes
• I included a recording for interaction changes

Created with gpt-5.6-sol using the Codex harness in T3 Code.

Note

Unify file and media preview parsing and playback across web and mobile

• Adds shared markdown file-link and media-source resolvers in client-runtime (parseMarkdownFileLink, resolveMediaSource) and replaces local parsing implementations in web, mobile, and the t3-markdown module with these shared helpers
• Reworks video previews on both platforms to use the shared MediaVideoPlayer; mobile removes the Expo-based VideoPlayback and MediaVideoPreviewModal, and web replaces raw <video> rendering in ExpandedImageDialog, MessagesTimeline, and ChatMarkdown with the shared player
• Moves AssetUrlState and assetUrlStateFromResult into client-runtime; web and mobile asset URL hooks now derive loading/failure/success through the shared mapper instead of local inline checks
• Unifies media action menus via a shared MediaActionId union and MediaActionsMenu/useMediaActions; both platforms now expose save/share/open-file actions through the same identifiers and long-press surfaces
• Risk: resolveViewedImageAsset and ViewedImageAsset now only return media-file resources, removing the attachment-resource variant — callers expecting attachment resources from viewed workspace images will break; iOS FilePreview always uses NativeFilePreview and no longer renders MediaImagePreview for actionable images; MediaVideoPlayer no longer renders an expand/maximize control or accepts an onExpand callback

^{Macroscope summarized 452f18a.}

---

Note

Medium Risk
Broad cross-platform media and preview behavior changes (iOS native viewers, inline vs modal video, shared URL lifecycle) touch high-traffic chat and file flows, though logic is consolidated and covered by focused tests.

Overview
Centralizes markdown file-link parsing, media-source classification, signed asset URL state, and media action IDs in client-runtime, then rewires web and mobile to call those helpers instead of duplicated local heuristics.

Mobile drops bespoke link/path code in favor of parseMarkdownFileLink and resolveMediaSource. Video and attachment UX converges on useMediaActions / MediaActionsMenu (long-press on thumbnails); VideoAttachmentMenu, MediaVideoPreviewModal, and the workspace image prefetch atom are removed. iOS routes all full-screen file previews through Quick Look and full-screen videos through AVKit (with URL re-mint on failure); inline/workspace video stays on Expo Video. Composer and thread attachments share attachmentVideoPreviewSource and attachment-scoped asset resources.

Web plays timeline and markdown videos inline via the shared MediaVideoPlayer (no separate “open video” flow or expand control); expanded dialogs stream through the same player with asset refresh. File attachment clicks no longer auto-open a modal for videos. Context menus align action IDs (save, copy-full-path, etc.) and expose save for videos as well as images.

Docs are updated to describe streaming, long-press save/share, and platform-specific players.

^{Reviewed by Cursor Bugbot for commit 452f18a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.5 KiB	+152 B (+1.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+6 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+146 B (+2.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	57.0 KiB	+1.4 KiB (+2.6%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	+27 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−6 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+33 B (+0.5%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.4 KiB	+44 B (+0.1%)	66.4 KiB	✅
Claude	Live turn messages	9	10	+1 (+11.1%)	21	✅

Baseline: 7751299 · PR result: 452f18a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a cross-client media-preview redesign that changes playback, asset URL minting, sharing, file-link parsing, and action menus across shared runtime, web, and mobile production paths. The new behavior and infrastructure span many existing surfaces, exceeding the scope of an auto-approvable refactor.

Not approved because:

• Monthly spending limit reached (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 05b0284. Configure here.}

[macroscopeapp]

🟡 Mediumsrc/terminal-links.ts:251

Activating a POSIX path ending in :0, such as logs/output:0, resolves it to <cwd>/logs/output instead of the actual file <cwd>/logs/output:0. splitFilePathPosition drops the non-positive line suffix and formatFilePathPosition does not restore it; preserve the suffix as part of the path when no valid line is parsed.

- const { path } = position;+ const path =+ position.line === undefined && rawPath.endsWith(":0") ? rawPath : position.path;

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/terminal-links.ts around line 251:
Activating a POSIX path ending in `:0`, such as `logs/output:0`, resolves it to `<cwd>/logs/output` instead of the actual file `<cwd>/logs/output:0`. `splitFilePathPosition` drops the non-positive line suffix and `formatFilePathPosition` does not restore it; preserve the suffix as part of the path when no valid line is parsed.

[macroscopeapp]

🟠 Highcomponents/VideoPreviewModal.tsx:94

Save/share can fail after the modal closes because useLocalPlayback exposes only the resolved uri, so useMediaActions takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft attachment through the share path (or otherwise retain it for the operation), as AttachmentPreviewFile.share does.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/mobile/src/components/VideoPreviewModal.tsx around line 94:
Save/share can fail after the modal closes because `useLocalPlayback` exposes only the resolved `uri`, so `useMediaActions` takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft `attachment` through the share path (or otherwise retain it for the operation), as `AttachmentPreviewFile.share` does.

[macroscopeapp]

🟡 Mediumcomponents/ChatMarkdown.tsx:2621

Workspace-backed images rendered through ChatMarkdownAssetImage no longer open the full-size preview when clicked or activated by keyboard. The onImageExpand={imageExpand} prop was removed from ChatMarkdownVideo, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

 style={authoredSizeStyle}
+ onImageExpand={imageExpand}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/components/ChatMarkdown.tsx around line 2621:
Workspace-backed images rendered through `ChatMarkdownAssetImage` no longer open the full-size preview when clicked or activated by keyboard. The `onImageExpand={imageExpand}` prop was removed from `ChatMarkdownVideo`, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

[macroscopeapp]

Dropping onImageExpand here (and MediaVideoPlayer's onExpand) removes the only route from a markdown video embed to the expanded dialog, while the sibling image embed still expands on click (expandableMarkdownImageProps + cursor-zoom-in, same component). Inline embeds are capped by CHAT_MARKDOWN_MEDIA_MAX_WIDTH_CLASS_NAME, so a video's only remaining enlargement is the browser's own fullscreen button — and it also loses the dialog-level media actions that images keep. docs/user/composer.md (line 100, in a paragraph that covers web, desktop, and mobile) still states "video embeds show a player with controls and an option to expand", and both platforms lost that control in this PR.

If the removal is intentional, updating that doc sentence is the smallest fix; otherwise consider keeping the expand handler for video embeds so images and videos behave the same inline.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Attachment videos now mint their URL inside this dialog, but sourceFailed only reflects Failure. While the environment is reconnecting (connecting/backoff, or connected with no session yet), assetEnvironment.createUrl resolves to Effect.never (client-runtime state/runtime.ts), so assetUrl stays Loading and the dialog paints the black stateClassName placeholder with no message, no retry, and no end state until the connection comes back. Before this change the same click went through downloadFileAttachment, which pre-checked readPreparedConnection and toasted "The environment is not connected.", and the mobile player added in this PR (useMediaPlayback) folds connection._tag === "None" into unavailable for exactly this case — so web is now the only surface with no disconnected end state.

Smallest fix is to treat a missing prepared connection as a failure here (needs usePreparedConnection from ../../state/session), which also gives the panel its Retry button:

constconnection=usePreparedConnection(asset?.environmentId??null);// ...sourceFailed={assetUrl._tag==="Failure"||(asset!==undefined&&connection._tag==="None")}

Keeping the connection pre-check in openFileAttachment before setExpandedImage would work too.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

🟢 Lowinternals/mobile-navigation.md:95

The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, OpenVideoPreviewModal renders MediaActionsMenu with inModal in the header and exposes a Save or share video button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

-Media actions (copy path, open in file viewer, save or share) belong to the surface that-opened the video, a long-press on the thumbnail, so the player carries no-menu. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.+On iOS, media actions (copy path, open in file viewer, save or share) belong to the+surface that opened the video, a long-press on the thumbnail, so the player carries no+menu. Android exposes `MediaActionsMenu` and a `Save or share video` button in the modal. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @docs/internals/mobile-navigation.md around lines 95-97:
The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, `OpenVideoPreviewModal` renders `MediaActionsMenu` with `inModal` in the header and exposes a `Save or share video` button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

[macroscopeapp]

This tile is labelled Play ${file.name}, but on this branch of the handler it no longer plays. onFileOpen is ChatView's openFileAttachment, which routes only html/pdf to the panel and otherwise calls downloadFileAttachment — and this PR removed that function's video branch (the one that used to setExpandedImage({ images: [{ src: url, name, type: "video" }] })). So for a rehydrated draft (file.file === null with an uploadedAttachmentId, the normal state after a reload) clicking the play glyph now silently saves the file to disk instead of opening the player. The removal of the aria-busy/"Loading…" state on these lines assumed the click still ends in the expanded dialog, and timeline videos moved to MediaVideoPlayer, so this is the only surface left on the old play path.

Smallest fix is to open the dialog here instead of downloading: build the item with src: null and actionsSource: { kind: "video", name: file.name, src: null, asset: buildAttachmentVideoAsset(file.uploadEnvironmentId ?? environmentId, { ...file, id: file.uploadedAttachmentId }) } and pass it to onExpandImage — ExpandedVideo already mints and plays that shape. If downloading is the intended behavior for this state, the control should say so rather than "Play".

^{Posted via Macroscope — UI Consistency}

[juliusmarminge]

What changed

• Moved markdown file-link parsing, media-source resolution, signed URL state, and media-action availability into client-runtime so web and mobile use the same rules.
• Routed expanded web videos through the shared streaming player, including URL refresh and retry.
• Routed every iOS video through AVKit and every iOS image, PDF, and HTML preview through Quick Look.
• Moved media actions to a long-press on the thumbnail and removed the redundant in-player controls.

Native long-press menus on markdown file chips are in #9258.

Why

The web and mobile clients had separate path parsers, media resolvers, URL lifecycle code, and viewer paths. The copies had already drifted. Some videos downloaded before playback, expired URLs left expanded viewers stuck, and iOS media sometimes opened in React Native viewers instead of AVKit or Quick Look.

This puts the shared decisions in one package and leaves each client responsible only for presentation. On iOS, native viewers own media playback and document preview.

UI changes

Before: clipped initial video controls	After: AVKit controls fill the thumbnail

Pre-fix tap and long-press race reproduced on device: screen recording

Testing

• 40 focused mobile file, markdown, and media tests
• Mobile TypeScript typecheck
• TypeScript lint and formatting checks for changed files
• Native Swift and Kotlin static checks
• Physical iPhone 16 Pro pass against a disposable LAN environment, covering workspace and external files, images, videos, HTML, PDFs, attachments, composer drafts, markdown embeds, thumbnail long-press menus, AVKit, and Quick Look

Checklist

• This PR is focused on one media-preview refactor (net negative after splitting out feat(mobile): long-press file references for path and open actions #9258)
• I explained what changed and why
• I included before/after screenshots for UI changes
• I included a recording for interaction changes

Created with gpt-5.6-sol using the Codex harness in T3 Code.

Note

Unify file and media preview parsing and playback across web and mobile

• Adds shared markdown file-link and media-source resolvers in client-runtime (parseMarkdownFileLink, resolveMediaSource) and replaces local parsing implementations in web, mobile, and the t3-markdown module with these shared helpers
• Reworks video previews on both platforms to use the shared MediaVideoPlayer; mobile removes the Expo-based VideoPlayback and MediaVideoPreviewModal, and web replaces raw <video> rendering in ExpandedImageDialog, MessagesTimeline, and ChatMarkdown with the shared player
• Moves AssetUrlState and assetUrlStateFromResult into client-runtime; web and mobile asset URL hooks now derive loading/failure/success through the shared mapper instead of local inline checks
• Unifies media action menus via a shared MediaActionId union and MediaActionsMenu/useMediaActions; both platforms now expose save/share/open-file actions through the same identifiers and long-press surfaces
• Risk: resolveViewedImageAsset and ViewedImageAsset now only return media-file resources, removing the attachment-resource variant — callers expecting attachment resources from viewed workspace images will break; iOS FilePreview always uses NativeFilePreview and no longer renders MediaImagePreview for actionable images; MediaVideoPlayer no longer renders an expand/maximize control or accepts an onExpand callback

^{Macroscope summarized 452f18a.}

---

Note

Medium Risk
Broad cross-platform media and preview behavior changes (iOS native viewers, inline vs modal video, shared URL lifecycle) touch high-traffic chat and file flows, though logic is consolidated and covered by focused tests.

Overview
Centralizes markdown file-link parsing, media-source classification, signed asset URL state, and media action IDs in client-runtime, then rewires web and mobile to call those helpers instead of duplicated local heuristics.

Mobile drops bespoke link/path code in favor of parseMarkdownFileLink and resolveMediaSource. Video and attachment UX converges on useMediaActions / MediaActionsMenu (long-press on thumbnails); VideoAttachmentMenu, MediaVideoPreviewModal, and the workspace image prefetch atom are removed. iOS routes all full-screen file previews through Quick Look and full-screen videos through AVKit (with URL re-mint on failure); inline/workspace video stays on Expo Video. Composer and thread attachments share attachmentVideoPreviewSource and attachment-scoped asset resources.

Web plays timeline and markdown videos inline via the shared MediaVideoPlayer (no separate “open video” flow or expand control); expanded dialogs stream through the same player with asset refresh. File attachment clicks no longer auto-open a modal for videos. Context menus align action IDs (save, copy-full-path, etc.) and expose save for videos as well as images.

Docs are updated to describe streaming, long-press save/share, and platform-specific players.

^{Reviewed by Cursor Bugbot for commit 452f18a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.5 KiB	+152 B (+1.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+6 B (+0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+146 B (+2.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	57.0 KiB	+1.4 KiB (+2.6%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	+27 B (+0.2%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−6 B (−0.1%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+33 B (+0.5%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.4 KiB	+44 B (+0.1%)	66.4 KiB	✅
Claude	Live turn messages	9	10	+1 (+11.1%)	21	✅

Baseline: 7751299 · PR result: 452f18a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a cross-client media-preview redesign that changes playback, asset URL minting, sharing, file-link parsing, and action menus across shared runtime, web, and mobile production paths. The new behavior and infrastructure span many existing surfaces, exceeding the scope of an auto-approvable refactor.

Not approved because:

• Monthly spending limit reached (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 05b0284. Configure here.}

[macroscopeapp]

🟡 Mediumsrc/terminal-links.ts:251

Activating a POSIX path ending in :0, such as logs/output:0, resolves it to <cwd>/logs/output instead of the actual file <cwd>/logs/output:0. splitFilePathPosition drops the non-positive line suffix and formatFilePathPosition does not restore it; preserve the suffix as part of the path when no valid line is parsed.

- const { path } = position;+ const path =+ position.line === undefined && rawPath.endsWith(":0") ? rawPath : position.path;

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/terminal-links.ts around line 251:
Activating a POSIX path ending in `:0`, such as `logs/output:0`, resolves it to `<cwd>/logs/output` instead of the actual file `<cwd>/logs/output:0`. `splitFilePathPosition` drops the non-positive line suffix and `formatFilePathPosition` does not restore it; preserve the suffix as part of the path when no valid line is parsed.

[macroscopeapp]

🟠 Highcomponents/VideoPreviewModal.tsx:94

Save/share can fail after the modal closes because useLocalPlayback exposes only the resolved uri, so useMediaActions takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft attachment through the share path (or otherwise retain it for the operation), as AttachmentPreviewFile.share does.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/mobile/src/components/VideoPreviewModal.tsx around line 94:
Save/share can fail after the modal closes because `useLocalPlayback` exposes only the resolved `uri`, so `useMediaActions` takes the URI share path without acquiring a composer-file lease. Cleanup then releases the preview lease while the asynchronous copy is still reading the source, allowing it to be deleted; pass the draft `attachment` through the share path (or otherwise retain it for the operation), as `AttachmentPreviewFile.share` does.

[macroscopeapp]

🟡 Mediumcomponents/ChatMarkdown.tsx:2621

Workspace-backed images rendered through ChatMarkdownAssetImage no longer open the full-size preview when clicked or activated by keyboard. The onImageExpand={imageExpand} prop was removed from ChatMarkdownVideo, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

 style={authoredSizeStyle}
+ onImageExpand={imageExpand}

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @apps/web/src/components/ChatMarkdown.tsx around line 2621:
Workspace-backed images rendered through `ChatMarkdownAssetImage` no longer open the full-size preview when clicked or activated by keyboard. The `onImageExpand={imageExpand}` prop was removed from `ChatMarkdownVideo`, so the asset renderer receives no expansion callback, while direct images still retain theirs; restore that prop for workspace media.

[macroscopeapp]

Dropping onImageExpand here (and MediaVideoPlayer's onExpand) removes the only route from a markdown video embed to the expanded dialog, while the sibling image embed still expands on click (expandableMarkdownImageProps + cursor-zoom-in, same component). Inline embeds are capped by CHAT_MARKDOWN_MEDIA_MAX_WIDTH_CLASS_NAME, so a video's only remaining enlargement is the browser's own fullscreen button — and it also loses the dialog-level media actions that images keep. docs/user/composer.md (line 100, in a paragraph that covers web, desktop, and mobile) still states "video embeds show a player with controls and an option to expand", and both platforms lost that control in this PR.

If the removal is intentional, updating that doc sentence is the smallest fix; otherwise consider keeping the expand handler for video embeds so images and videos behave the same inline.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Attachment videos now mint their URL inside this dialog, but sourceFailed only reflects Failure. While the environment is reconnecting (connecting/backoff, or connected with no session yet), assetEnvironment.createUrl resolves to Effect.never (client-runtime state/runtime.ts), so assetUrl stays Loading and the dialog paints the black stateClassName placeholder with no message, no retry, and no end state until the connection comes back. Before this change the same click went through downloadFileAttachment, which pre-checked readPreparedConnection and toasted "The environment is not connected.", and the mobile player added in this PR (useMediaPlayback) folds connection._tag === "None" into unavailable for exactly this case — so web is now the only surface with no disconnected end state.

Smallest fix is to treat a missing prepared connection as a failure here (needs usePreparedConnection from ../../state/session), which also gives the panel its Retry button:

constconnection=usePreparedConnection(asset?.environmentId??null);// ...sourceFailed={assetUrl._tag==="Failure"||(asset!==undefined&&connection._tag==="None")}

Keeping the connection pre-check in openFileAttachment before setExpandedImage would work too.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

🟢 Lowinternals/mobile-navigation.md:95

The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, OpenVideoPreviewModal renders MediaActionsMenu with inModal in the header and exposes a Save or share video button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

-Media actions (copy path, open in file viewer, save or share) belong to the surface that-opened the video, a long-press on the thumbnail, so the player carries no-menu. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.+On iOS, media actions (copy path, open in file viewer, save or share) belong to the+surface that opened the video, a long-press on the thumbnail, so the player carries no+menu. Android exposes `MediaActionsMenu` and a `Save or share video` button in the modal. AVKit has no retry; a failure alerts and closes, re-mints the URL in the background, and the next open plays from the fresh one.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @docs/internals/mobile-navigation.md around lines 95-97:
The documentation incorrectly says the player carries no media-action menu and that actions are available only via a long-press on the thumbnail. On Android, `OpenVideoPreviewModal` renders `MediaActionsMenu` with `inModal` in the header and exposes a `Save or share video` button, so users can access preview actions directly in the modal; please scope the no-menu/long-press behavior to iOS.

[macroscopeapp]

This tile is labelled Play ${file.name}, but on this branch of the handler it no longer plays. onFileOpen is ChatView's openFileAttachment, which routes only html/pdf to the panel and otherwise calls downloadFileAttachment — and this PR removed that function's video branch (the one that used to setExpandedImage({ images: [{ src: url, name, type: "video" }] })). So for a rehydrated draft (file.file === null with an uploadedAttachmentId, the normal state after a reload) clicking the play glyph now silently saves the file to disk instead of opening the player. The removal of the aria-busy/"Loading…" state on these lines assumed the click still ends in the expanded dialog, and timeline videos moved to MediaVideoPlayer, so this is the only surface left on the old play path.

Smallest fix is to open the dialog here instead of downloading: build the item with src: null and actionsSource: { kind: "video", name: file.name, src: null, asset: buildAttachmentVideoAsset(file.uploadEnvironmentId ?? environmentId, { ...file, id: file.uploadedAttachmentId }) } and pass it to onExpandImage — ExpandedVideo already mints and plays that shape. If downloading is the intended behavior for this state, the control should say so rather than "Play".

^{Posted via Macroscope — UI Consistency}