feat: render saved media in chat across clients - #4872

Closed
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews
Closed

feat: render saved media in chat across clients#4872
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews

Conversation

@gabrielelpidio

@gabrielelpidiogabrielelpidio commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add secure signed asset access for saved screenshots, recordings, workspace files, and browser artifacts.
  • Let preview snapshots save to workspace paths or browser artifact storage.
  • Render saved media directly in web and desktop chat markdown, including expanded images and settled-turn galleries.
  • Preserve image-view paths in projected provider activity.
  • Add the corresponding mobile behavior: signed media resolution, inline images, the existing full-screen image viewer, and user-initiated recording controls.
  • Share markdown media path classification across web and mobile.

This combines the work from #4321 with mobile support in one PR targeting main.

Follow-up

Testing

  • The original Render saved screenshots and recordings in chat markdown #4321 focused suite passed 277 tests before the mobile split.
  • The mobile addition passed 44 focused shared/mobile markdown tests.
  • Shared, web, and mobile typechecks passed.
  • Targeted lint and formatting passed.
  • Mobile native static analysis passed; optional local SwiftLint, ktlint, and detekt binaries were unavailable and skipped.
  • Web production build passed.
  • Verified signed workspace images inline and expanded in the real web client.
  • Verified user and assistant images, expanded images, and MP4 previews on an iPhone 17 Pro simulator running iOS 26.5.
  • Re-ran the focused tests, package typechecks, targeted lint and formatting, and mobile native static analysis after rebasing onto the current main.

Visual evidence

  • Web: inline user/assistant images and the expanded-image dialog.
  • Mobile before: workspace-relative image nodes reserve space but remain blank.
  • Mobile after: images render in both message roles and open full-screen.
  • Mobile recordings: inline controls render in both message roles and playback remains user initiated.

Model: GPT-5.6-sol
Harness: T3 Code (Codex)

Note

Render saved media (images and videos) in chat across web and mobile clients

  • Adds end-to-end support for displaying image_view and imageGeneration media outputs in chat timelines on web and mobile, resolving asset URLs via thread-aware asset resolution.
  • Introduces thread-image and browser-artifact asset resource types with strict symlink-safe file access using no-follow semantics on all platforms (noFollowFile.ts).
  • Extends the messages timeline on web to emit image-output rows and inline assistantImageOutputs, with thumbnail galleries, loading/failure states, and click-to-expand previews (MessagesTimeline.logic.ts, MessagesTimeline.tsx).
  • Adds a MarkdownMedia component on both web (MarkdownMedia.tsx) and mobile (MarkdownMedia.tsx) that resolves workspace-relative and direct media sources and renders images or videos with unavailable fallback states.
  • Extends the preview_snapshot MCP tool to accept save/savePath parameters and persist screenshots to the workspace or a dedicated artifacts directory (handlers.ts).
  • Risk: asset resolution for thread-image re-canonicalizes paths on every request; post-issuance symlink swaps are detected and return null, not an error, so callers must handle silent resolution failures.

Macroscope summarized 275e7d9.


Note

High Risk
Touches signed asset serving, symlink-safe file I/O, and workspace path validation—security-sensitive paths with a large cross-stack surface (server, web, mobile, contracts).

Overview
Chat on web, mobile, and native iOS markdown can now show images and videos from markdown, workspace paths, browser artifacts, and tool image outputs—not just plain links or blank placeholders.

Signed asset access adds browser-artifact and thread-image resources, video preview extensions, and stricter workspace containment (e.g. ..name dirs vs real .. escapes). Sensitive reads/writes use no-follow open helpers on Linux/Darwin/fallback platforms so symlink swaps after token issuance don’t redirect serving or snapshot saves.

Preview preview_snapshot can persist PNGs via save (artifact dir) or savePath (workspace) with containment checks; tool docs steer agents to embed returned paths in replies.

Timeline derives deduplicated image galleries on terminal assistant rows (and trailing rows when needed), withholding in-flight turn images until the turn settles.

Provider projection keeps image-view/generation paths while still pruning huge base64 payloads from activity data.

Reviewed by Cursor Bugbot for commit 275e7d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitaiBot commented Jul 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27353ca3-545c-4bd2-9346-bed42f19dd44

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 29, 2026
@gabrielelpidiogabrielelpidio changed the title feat(mobile): render chat media previewsfeat: render saved media in chat across clientsJul 30, 2026
@gabrielelpidio
gabrielelpidio changed the base branch from t3code/render-chat-media to mainJuly 30, 2026 01:57
juliusmarmingeand others added 10 commits July 29, 2026 21:58
- Add secure asset access for workspace media and browser artifacts
- Support saving preview screenshots and embedding playable recordings
- Preserve error causes in screenshot save failures
- Reject save+savePath combinations and describe both schema fields
- Verify canonical (symlink-resolved) containment before workspace writes
- Add collision-resistant suffix to artifact screenshot names
- Fix Windows drive paths, protocol-relative URLs, query strings, and
extensionless video sources in markdown media resolution
- Only treat absolute browser-artifacts paths as server artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verify the deepest existing ancestor canonicalizes inside the workspace
before mkdir, and reject existing destination symlinks that resolve
outside the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Re-resolve browser-artifact and thread-image claims on every request so
post-issuance symlink swaps cannot redirect a signed token
- Verify canonical containment and media extension after realPath
- Render unscoped image outputs when no turn is unsettled
- Keep the image activity id paired with its path across collapsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drop readonly/idempotent hints from preview_snapshot now that it writes
- Treat only real .. segments as escapes when resolving workspace assets
- Append image galleries when turns share a target row
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload pruning (#4622) stripped every item field outside the command
allowlist, so image view activities reached clients without the path and
the chat gallery rendered nothing. Preserve the item type and path while
still dropping the base64 image bytes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gabrielelpidio
gabrielelpidioforce-pushed the t3code/mobile-chat-media-previews branch from 5b2f955 to 5445615CompareJuly 30, 2026 01:59
@github-actionsgithub-actionsBot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 30, 2026
Comment threadapps/server/src/mcp/toolkits/preview/handlers.ts Outdated
Comment threadapps/server/src/assets/AssetAccess.ts
Comment threadapps/server/src/mcp/McpHttpServer.ts
Comment threadapps/web/src/components/chat/MessagesTimeline.tsx Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5445615. Configure here.

Comment threadapps/web/src/components/chat/MessagesTimeline.tsx
@macroscopeapp

macroscopeappBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a new feature for rendering saved media (images/videos) in chat, adding new asset types, security-sensitive path validation logic, and MCP tool capabilities for saving screenshots. The scope and security implications warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Comment threadapps/server/src/noFollowFile.ts Outdated
Comment threadapps/server/src/noFollowFile.ts
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gabrielelpidio@juliusmarminge@Noojuno
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat: render saved media in chat across clients - #4872

Closed
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews
Closed

feat: render saved media in chat across clients#4872
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews

Conversation

@gabrielelpidio

@gabrielelpidiogabrielelpidio commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add secure signed asset access for saved screenshots, recordings, workspace files, and browser artifacts.
  • Let preview snapshots save to workspace paths or browser artifact storage.
  • Render saved media directly in web and desktop chat markdown, including expanded images and settled-turn galleries.
  • Preserve image-view paths in projected provider activity.
  • Add the corresponding mobile behavior: signed media resolution, inline images, the existing full-screen image viewer, and user-initiated recording controls.
  • Share markdown media path classification across web and mobile.

This combines the work from #4321 with mobile support in one PR targeting main.

Follow-up

Testing

  • The original Render saved screenshots and recordings in chat markdown #4321 focused suite passed 277 tests before the mobile split.
  • The mobile addition passed 44 focused shared/mobile markdown tests.
  • Shared, web, and mobile typechecks passed.
  • Targeted lint and formatting passed.
  • Mobile native static analysis passed; optional local SwiftLint, ktlint, and detekt binaries were unavailable and skipped.
  • Web production build passed.
  • Verified signed workspace images inline and expanded in the real web client.
  • Verified user and assistant images, expanded images, and MP4 previews on an iPhone 17 Pro simulator running iOS 26.5.
  • Re-ran the focused tests, package typechecks, targeted lint and formatting, and mobile native static analysis after rebasing onto the current main.

Visual evidence

  • Web: inline user/assistant images and the expanded-image dialog.
  • Mobile before: workspace-relative image nodes reserve space but remain blank.
  • Mobile after: images render in both message roles and open full-screen.
  • Mobile recordings: inline controls render in both message roles and playback remains user initiated.

Model: GPT-5.6-sol
Harness: T3 Code (Codex)

Note

Render saved media (images and videos) in chat across web and mobile clients

  • Adds end-to-end support for displaying image_view and imageGeneration media outputs in chat timelines on web and mobile, resolving asset URLs via thread-aware asset resolution.
  • Introduces thread-image and browser-artifact asset resource types with strict symlink-safe file access using no-follow semantics on all platforms (noFollowFile.ts).
  • Extends the messages timeline on web to emit image-output rows and inline assistantImageOutputs, with thumbnail galleries, loading/failure states, and click-to-expand previews (MessagesTimeline.logic.ts, MessagesTimeline.tsx).
  • Adds a MarkdownMedia component on both web (MarkdownMedia.tsx) and mobile (MarkdownMedia.tsx) that resolves workspace-relative and direct media sources and renders images or videos with unavailable fallback states.
  • Extends the preview_snapshot MCP tool to accept save/savePath parameters and persist screenshots to the workspace or a dedicated artifacts directory (handlers.ts).
  • Risk: asset resolution for thread-image re-canonicalizes paths on every request; post-issuance symlink swaps are detected and return null, not an error, so callers must handle silent resolution failures.

Macroscope summarized 275e7d9.


Note

High Risk
Touches signed asset serving, symlink-safe file I/O, and workspace path validation—security-sensitive paths with a large cross-stack surface (server, web, mobile, contracts).

Overview
Chat on web, mobile, and native iOS markdown can now show images and videos from markdown, workspace paths, browser artifacts, and tool image outputs—not just plain links or blank placeholders.

Signed asset access adds browser-artifact and thread-image resources, video preview extensions, and stricter workspace containment (e.g. ..name dirs vs real .. escapes). Sensitive reads/writes use no-follow open helpers on Linux/Darwin/fallback platforms so symlink swaps after token issuance don’t redirect serving or snapshot saves.

Preview preview_snapshot can persist PNGs via save (artifact dir) or savePath (workspace) with containment checks; tool docs steer agents to embed returned paths in replies.

Timeline derives deduplicated image galleries on terminal assistant rows (and trailing rows when needed), withholding in-flight turn images until the turn settles.

Provider projection keeps image-view/generation paths while still pruning huge base64 payloads from activity data.

Reviewed by Cursor Bugbot for commit 275e7d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitaiBot commented Jul 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27353ca3-545c-4bd2-9346-bed42f19dd44

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 29, 2026
@gabrielelpidiogabrielelpidio changed the title feat(mobile): render chat media previewsfeat: render saved media in chat across clientsJul 30, 2026
@gabrielelpidio
gabrielelpidio changed the base branch from t3code/render-chat-media to mainJuly 30, 2026 01:57
juliusmarmingeand others added 10 commits July 29, 2026 21:58
- Add secure asset access for workspace media and browser artifacts
- Support saving preview screenshots and embedding playable recordings
- Preserve error causes in screenshot save failures
- Reject save+savePath combinations and describe both schema fields
- Verify canonical (symlink-resolved) containment before workspace writes
- Add collision-resistant suffix to artifact screenshot names
- Fix Windows drive paths, protocol-relative URLs, query strings, and
extensionless video sources in markdown media resolution
- Only treat absolute browser-artifacts paths as server artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verify the deepest existing ancestor canonicalizes inside the workspace
before mkdir, and reject existing destination symlinks that resolve
outside the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Re-resolve browser-artifact and thread-image claims on every request so
post-issuance symlink swaps cannot redirect a signed token
- Verify canonical containment and media extension after realPath
- Render unscoped image outputs when no turn is unsettled
- Keep the image activity id paired with its path across collapsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drop readonly/idempotent hints from preview_snapshot now that it writes
- Treat only real .. segments as escapes when resolving workspace assets
- Append image galleries when turns share a target row
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload pruning (#4622) stripped every item field outside the command
allowlist, so image view activities reached clients without the path and
the chat gallery rendered nothing. Preserve the item type and path while
still dropping the base64 image bytes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gabrielelpidio
gabrielelpidioforce-pushed the t3code/mobile-chat-media-previews branch from 5b2f955 to 5445615CompareJuly 30, 2026 01:59
@github-actionsgithub-actionsBot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 30, 2026
Comment threadapps/server/src/mcp/toolkits/preview/handlers.ts Outdated
Comment threadapps/server/src/assets/AssetAccess.ts
Comment threadapps/server/src/mcp/McpHttpServer.ts
Comment threadapps/web/src/components/chat/MessagesTimeline.tsx Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5445615. Configure here.

Comment threadapps/web/src/components/chat/MessagesTimeline.tsx
@macroscopeapp

macroscopeappBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a new feature for rendering saved media (images/videos) in chat, adding new asset types, security-sensitive path validation logic, and MCP tool capabilities for saving screenshots. The scope and security implications warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Comment threadapps/server/src/noFollowFile.ts Outdated
Comment threadapps/server/src/noFollowFile.ts
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gabrielelpidio@juliusmarminge@Noojuno
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: render saved media in chat across clients - #4872

Closed
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews
Closed

feat: render saved media in chat across clients#4872
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews

Conversation

@gabrielelpidio

@gabrielelpidiogabrielelpidio commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add secure signed asset access for saved screenshots, recordings, workspace files, and browser artifacts.
  • Let preview snapshots save to workspace paths or browser artifact storage.
  • Render saved media directly in web and desktop chat markdown, including expanded images and settled-turn galleries.
  • Preserve image-view paths in projected provider activity.
  • Add the corresponding mobile behavior: signed media resolution, inline images, the existing full-screen image viewer, and user-initiated recording controls.
  • Share markdown media path classification across web and mobile.

This combines the work from #4321 with mobile support in one PR targeting main.

Follow-up

Testing

  • The original Render saved screenshots and recordings in chat markdown #4321 focused suite passed 277 tests before the mobile split.
  • The mobile addition passed 44 focused shared/mobile markdown tests.
  • Shared, web, and mobile typechecks passed.
  • Targeted lint and formatting passed.
  • Mobile native static analysis passed; optional local SwiftLint, ktlint, and detekt binaries were unavailable and skipped.
  • Web production build passed.
  • Verified signed workspace images inline and expanded in the real web client.
  • Verified user and assistant images, expanded images, and MP4 previews on an iPhone 17 Pro simulator running iOS 26.5.
  • Re-ran the focused tests, package typechecks, targeted lint and formatting, and mobile native static analysis after rebasing onto the current main.

Visual evidence

  • Web: inline user/assistant images and the expanded-image dialog.
  • Mobile before: workspace-relative image nodes reserve space but remain blank.
  • Mobile after: images render in both message roles and open full-screen.
  • Mobile recordings: inline controls render in both message roles and playback remains user initiated.

Model: GPT-5.6-sol
Harness: T3 Code (Codex)

Note

Render saved media (images and videos) in chat across web and mobile clients

  • Adds end-to-end support for displaying image_view and imageGeneration media outputs in chat timelines on web and mobile, resolving asset URLs via thread-aware asset resolution.
  • Introduces thread-image and browser-artifact asset resource types with strict symlink-safe file access using no-follow semantics on all platforms (noFollowFile.ts).
  • Extends the messages timeline on web to emit image-output rows and inline assistantImageOutputs, with thumbnail galleries, loading/failure states, and click-to-expand previews (MessagesTimeline.logic.ts, MessagesTimeline.tsx).
  • Adds a MarkdownMedia component on both web (MarkdownMedia.tsx) and mobile (MarkdownMedia.tsx) that resolves workspace-relative and direct media sources and renders images or videos with unavailable fallback states.
  • Extends the preview_snapshot MCP tool to accept save/savePath parameters and persist screenshots to the workspace or a dedicated artifacts directory (handlers.ts).
  • Risk: asset resolution for thread-image re-canonicalizes paths on every request; post-issuance symlink swaps are detected and return null, not an error, so callers must handle silent resolution failures.

Macroscope summarized 275e7d9.


Note

High Risk
Touches signed asset serving, symlink-safe file I/O, and workspace path validation—security-sensitive paths with a large cross-stack surface (server, web, mobile, contracts).

Overview
Chat on web, mobile, and native iOS markdown can now show images and videos from markdown, workspace paths, browser artifacts, and tool image outputs—not just plain links or blank placeholders.

Signed asset access adds browser-artifact and thread-image resources, video preview extensions, and stricter workspace containment (e.g. ..name dirs vs real .. escapes). Sensitive reads/writes use no-follow open helpers on Linux/Darwin/fallback platforms so symlink swaps after token issuance don’t redirect serving or snapshot saves.

Preview preview_snapshot can persist PNGs via save (artifact dir) or savePath (workspace) with containment checks; tool docs steer agents to embed returned paths in replies.

Timeline derives deduplicated image galleries on terminal assistant rows (and trailing rows when needed), withholding in-flight turn images until the turn settles.

Provider projection keeps image-view/generation paths while still pruning huge base64 payloads from activity data.

Reviewed by Cursor Bugbot for commit 275e7d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitaiBot commented Jul 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27353ca3-545c-4bd2-9346-bed42f19dd44

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 29, 2026
@gabrielelpidiogabrielelpidio changed the title feat(mobile): render chat media previewsfeat: render saved media in chat across clientsJul 30, 2026
@gabrielelpidio
gabrielelpidio changed the base branch from t3code/render-chat-media to mainJuly 30, 2026 01:57
juliusmarmingeand others added 10 commits July 29, 2026 21:58
- Add secure asset access for workspace media and browser artifacts
- Support saving preview screenshots and embedding playable recordings
- Preserve error causes in screenshot save failures
- Reject save+savePath combinations and describe both schema fields
- Verify canonical (symlink-resolved) containment before workspace writes
- Add collision-resistant suffix to artifact screenshot names
- Fix Windows drive paths, protocol-relative URLs, query strings, and
extensionless video sources in markdown media resolution
- Only treat absolute browser-artifacts paths as server artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verify the deepest existing ancestor canonicalizes inside the workspace
before mkdir, and reject existing destination symlinks that resolve
outside the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Re-resolve browser-artifact and thread-image claims on every request so
post-issuance symlink swaps cannot redirect a signed token
- Verify canonical containment and media extension after realPath
- Render unscoped image outputs when no turn is unsettled
- Keep the image activity id paired with its path across collapsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drop readonly/idempotent hints from preview_snapshot now that it writes
- Treat only real .. segments as escapes when resolving workspace assets
- Append image galleries when turns share a target row
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload pruning (#4622) stripped every item field outside the command
allowlist, so image view activities reached clients without the path and
the chat gallery rendered nothing. Preserve the item type and path while
still dropping the base64 image bytes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gabrielelpidio
gabrielelpidioforce-pushed the t3code/mobile-chat-media-previews branch from 5b2f955 to 5445615CompareJuly 30, 2026 01:59
@github-actionsgithub-actionsBot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 30, 2026
Comment threadapps/server/src/mcp/toolkits/preview/handlers.ts Outdated
Comment threadapps/server/src/assets/AssetAccess.ts
Comment threadapps/server/src/mcp/McpHttpServer.ts
Comment threadapps/web/src/components/chat/MessagesTimeline.tsx Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5445615. Configure here.

Comment threadapps/web/src/components/chat/MessagesTimeline.tsx
@macroscopeapp

macroscopeappBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a new feature for rendering saved media (images/videos) in chat, adding new asset types, security-sensitive path validation logic, and MCP tool capabilities for saving screenshots. The scope and security implications warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Comment threadapps/server/src/noFollowFile.ts Outdated
Comment threadapps/server/src/noFollowFile.ts
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gabrielelpidio@juliusmarminge@Noojuno
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: render saved media in chat across clients - #4872

Closed
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews
Closed

feat: render saved media in chat across clients#4872
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews

Conversation

@gabrielelpidio

@gabrielelpidiogabrielelpidio commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add secure signed asset access for saved screenshots, recordings, workspace files, and browser artifacts.
  • Let preview snapshots save to workspace paths or browser artifact storage.
  • Render saved media directly in web and desktop chat markdown, including expanded images and settled-turn galleries.
  • Preserve image-view paths in projected provider activity.
  • Add the corresponding mobile behavior: signed media resolution, inline images, the existing full-screen image viewer, and user-initiated recording controls.
  • Share markdown media path classification across web and mobile.

This combines the work from #4321 with mobile support in one PR targeting main.

Follow-up

Testing

  • The original Render saved screenshots and recordings in chat markdown #4321 focused suite passed 277 tests before the mobile split.
  • The mobile addition passed 44 focused shared/mobile markdown tests.
  • Shared, web, and mobile typechecks passed.
  • Targeted lint and formatting passed.
  • Mobile native static analysis passed; optional local SwiftLint, ktlint, and detekt binaries were unavailable and skipped.
  • Web production build passed.
  • Verified signed workspace images inline and expanded in the real web client.
  • Verified user and assistant images, expanded images, and MP4 previews on an iPhone 17 Pro simulator running iOS 26.5.
  • Re-ran the focused tests, package typechecks, targeted lint and formatting, and mobile native static analysis after rebasing onto the current main.

Visual evidence

  • Web: inline user/assistant images and the expanded-image dialog.
  • Mobile before: workspace-relative image nodes reserve space but remain blank.
  • Mobile after: images render in both message roles and open full-screen.
  • Mobile recordings: inline controls render in both message roles and playback remains user initiated.

Model: GPT-5.6-sol
Harness: T3 Code (Codex)

Note

Render saved media (images and videos) in chat across web and mobile clients

  • Adds end-to-end support for displaying image_view and imageGeneration media outputs in chat timelines on web and mobile, resolving asset URLs via thread-aware asset resolution.
  • Introduces thread-image and browser-artifact asset resource types with strict symlink-safe file access using no-follow semantics on all platforms (noFollowFile.ts).
  • Extends the messages timeline on web to emit image-output rows and inline assistantImageOutputs, with thumbnail galleries, loading/failure states, and click-to-expand previews (MessagesTimeline.logic.ts, MessagesTimeline.tsx).
  • Adds a MarkdownMedia component on both web (MarkdownMedia.tsx) and mobile (MarkdownMedia.tsx) that resolves workspace-relative and direct media sources and renders images or videos with unavailable fallback states.
  • Extends the preview_snapshot MCP tool to accept save/savePath parameters and persist screenshots to the workspace or a dedicated artifacts directory (handlers.ts).
  • Risk: asset resolution for thread-image re-canonicalizes paths on every request; post-issuance symlink swaps are detected and return null, not an error, so callers must handle silent resolution failures.

Macroscope summarized 275e7d9.


Note

High Risk
Touches signed asset serving, symlink-safe file I/O, and workspace path validation—security-sensitive paths with a large cross-stack surface (server, web, mobile, contracts).

Overview
Chat on web, mobile, and native iOS markdown can now show images and videos from markdown, workspace paths, browser artifacts, and tool image outputs—not just plain links or blank placeholders.

Signed asset access adds browser-artifact and thread-image resources, video preview extensions, and stricter workspace containment (e.g. ..name dirs vs real .. escapes). Sensitive reads/writes use no-follow open helpers on Linux/Darwin/fallback platforms so symlink swaps after token issuance don’t redirect serving or snapshot saves.

Preview preview_snapshot can persist PNGs via save (artifact dir) or savePath (workspace) with containment checks; tool docs steer agents to embed returned paths in replies.

Timeline derives deduplicated image galleries on terminal assistant rows (and trailing rows when needed), withholding in-flight turn images until the turn settles.

Provider projection keeps image-view/generation paths while still pruning huge base64 payloads from activity data.

Reviewed by Cursor Bugbot for commit 275e7d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitaiBot commented Jul 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27353ca3-545c-4bd2-9346-bed42f19dd44

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 29, 2026
@gabrielelpidiogabrielelpidio changed the title feat(mobile): render chat media previewsfeat: render saved media in chat across clientsJul 30, 2026
@gabrielelpidio
gabrielelpidio changed the base branch from t3code/render-chat-media to mainJuly 30, 2026 01:57
juliusmarmingeand others added 10 commits July 29, 2026 21:58
- Add secure asset access for workspace media and browser artifacts
- Support saving preview screenshots and embedding playable recordings
- Preserve error causes in screenshot save failures
- Reject save+savePath combinations and describe both schema fields
- Verify canonical (symlink-resolved) containment before workspace writes
- Add collision-resistant suffix to artifact screenshot names
- Fix Windows drive paths, protocol-relative URLs, query strings, and
extensionless video sources in markdown media resolution
- Only treat absolute browser-artifacts paths as server artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verify the deepest existing ancestor canonicalizes inside the workspace
before mkdir, and reject existing destination symlinks that resolve
outside the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Re-resolve browser-artifact and thread-image claims on every request so
post-issuance symlink swaps cannot redirect a signed token
- Verify canonical containment and media extension after realPath
- Render unscoped image outputs when no turn is unsettled
- Keep the image activity id paired with its path across collapsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drop readonly/idempotent hints from preview_snapshot now that it writes
- Treat only real .. segments as escapes when resolving workspace assets
- Append image galleries when turns share a target row
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload pruning (#4622) stripped every item field outside the command
allowlist, so image view activities reached clients without the path and
the chat gallery rendered nothing. Preserve the item type and path while
still dropping the base64 image bytes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gabrielelpidio
gabrielelpidioforce-pushed the t3code/mobile-chat-media-previews branch from 5b2f955 to 5445615CompareJuly 30, 2026 01:59
@github-actionsgithub-actionsBot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 30, 2026
Comment threadapps/server/src/mcp/toolkits/preview/handlers.ts Outdated
Comment threadapps/server/src/assets/AssetAccess.ts
Comment threadapps/server/src/mcp/McpHttpServer.ts
Comment threadapps/web/src/components/chat/MessagesTimeline.tsx Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5445615. Configure here.

Comment threadapps/web/src/components/chat/MessagesTimeline.tsx
@macroscopeapp

macroscopeappBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a new feature for rendering saved media (images/videos) in chat, adding new asset types, security-sensitive path validation logic, and MCP tool capabilities for saving screenshots. The scope and security implications warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Comment threadapps/server/src/noFollowFile.ts Outdated
Comment threadapps/server/src/noFollowFile.ts
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gabrielelpidio@juliusmarminge@Noojuno
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat: render saved media in chat across clients - #4872

Closed
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews
Closed

feat: render saved media in chat across clients#4872
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews

Conversation

@gabrielelpidio

@gabrielelpidiogabrielelpidio commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add secure signed asset access for saved screenshots, recordings, workspace files, and browser artifacts.
  • Let preview snapshots save to workspace paths or browser artifact storage.
  • Render saved media directly in web and desktop chat markdown, including expanded images and settled-turn galleries.
  • Preserve image-view paths in projected provider activity.
  • Add the corresponding mobile behavior: signed media resolution, inline images, the existing full-screen image viewer, and user-initiated recording controls.
  • Share markdown media path classification across web and mobile.

This combines the work from #4321 with mobile support in one PR targeting main.

Follow-up

Testing

  • The original Render saved screenshots and recordings in chat markdown #4321 focused suite passed 277 tests before the mobile split.
  • The mobile addition passed 44 focused shared/mobile markdown tests.
  • Shared, web, and mobile typechecks passed.
  • Targeted lint and formatting passed.
  • Mobile native static analysis passed; optional local SwiftLint, ktlint, and detekt binaries were unavailable and skipped.
  • Web production build passed.
  • Verified signed workspace images inline and expanded in the real web client.
  • Verified user and assistant images, expanded images, and MP4 previews on an iPhone 17 Pro simulator running iOS 26.5.
  • Re-ran the focused tests, package typechecks, targeted lint and formatting, and mobile native static analysis after rebasing onto the current main.

Visual evidence

  • Web: inline user/assistant images and the expanded-image dialog.
  • Mobile before: workspace-relative image nodes reserve space but remain blank.
  • Mobile after: images render in both message roles and open full-screen.
  • Mobile recordings: inline controls render in both message roles and playback remains user initiated.

Model: GPT-5.6-sol
Harness: T3 Code (Codex)

Note

Render saved media (images and videos) in chat across web and mobile clients

  • Adds end-to-end support for displaying image_view and imageGeneration media outputs in chat timelines on web and mobile, resolving asset URLs via thread-aware asset resolution.
  • Introduces thread-image and browser-artifact asset resource types with strict symlink-safe file access using no-follow semantics on all platforms (noFollowFile.ts).
  • Extends the messages timeline on web to emit image-output rows and inline assistantImageOutputs, with thumbnail galleries, loading/failure states, and click-to-expand previews (MessagesTimeline.logic.ts, MessagesTimeline.tsx).
  • Adds a MarkdownMedia component on both web (MarkdownMedia.tsx) and mobile (MarkdownMedia.tsx) that resolves workspace-relative and direct media sources and renders images or videos with unavailable fallback states.
  • Extends the preview_snapshot MCP tool to accept save/savePath parameters and persist screenshots to the workspace or a dedicated artifacts directory (handlers.ts).
  • Risk: asset resolution for thread-image re-canonicalizes paths on every request; post-issuance symlink swaps are detected and return null, not an error, so callers must handle silent resolution failures.

Macroscope summarized 275e7d9.


Note

High Risk
Touches signed asset serving, symlink-safe file I/O, and workspace path validation—security-sensitive paths with a large cross-stack surface (server, web, mobile, contracts).

Overview
Chat on web, mobile, and native iOS markdown can now show images and videos from markdown, workspace paths, browser artifacts, and tool image outputs—not just plain links or blank placeholders.

Signed asset access adds browser-artifact and thread-image resources, video preview extensions, and stricter workspace containment (e.g. ..name dirs vs real .. escapes). Sensitive reads/writes use no-follow open helpers on Linux/Darwin/fallback platforms so symlink swaps after token issuance don’t redirect serving or snapshot saves.

Preview preview_snapshot can persist PNGs via save (artifact dir) or savePath (workspace) with containment checks; tool docs steer agents to embed returned paths in replies.

Timeline derives deduplicated image galleries on terminal assistant rows (and trailing rows when needed), withholding in-flight turn images until the turn settles.

Provider projection keeps image-view/generation paths while still pruning huge base64 payloads from activity data.

Reviewed by Cursor Bugbot for commit 275e7d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitaiBot commented Jul 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27353ca3-545c-4bd2-9346-bed42f19dd44

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 29, 2026
@gabrielelpidiogabrielelpidio changed the title feat(mobile): render chat media previewsfeat: render saved media in chat across clientsJul 30, 2026
@gabrielelpidio
gabrielelpidio changed the base branch from t3code/render-chat-media to mainJuly 30, 2026 01:57
juliusmarmingeand others added 10 commits July 29, 2026 21:58
- Add secure asset access for workspace media and browser artifacts
- Support saving preview screenshots and embedding playable recordings
- Preserve error causes in screenshot save failures
- Reject save+savePath combinations and describe both schema fields
- Verify canonical (symlink-resolved) containment before workspace writes
- Add collision-resistant suffix to artifact screenshot names
- Fix Windows drive paths, protocol-relative URLs, query strings, and
extensionless video sources in markdown media resolution
- Only treat absolute browser-artifacts paths as server artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verify the deepest existing ancestor canonicalizes inside the workspace
before mkdir, and reject existing destination symlinks that resolve
outside the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Re-resolve browser-artifact and thread-image claims on every request so
post-issuance symlink swaps cannot redirect a signed token
- Verify canonical containment and media extension after realPath
- Render unscoped image outputs when no turn is unsettled
- Keep the image activity id paired with its path across collapsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drop readonly/idempotent hints from preview_snapshot now that it writes
- Treat only real .. segments as escapes when resolving workspace assets
- Append image galleries when turns share a target row
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload pruning (#4622) stripped every item field outside the command
allowlist, so image view activities reached clients without the path and
the chat gallery rendered nothing. Preserve the item type and path while
still dropping the base64 image bytes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gabrielelpidio
gabrielelpidioforce-pushed the t3code/mobile-chat-media-previews branch from 5b2f955 to 5445615CompareJuly 30, 2026 01:59
@github-actionsgithub-actionsBot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 30, 2026
Comment threadapps/server/src/mcp/toolkits/preview/handlers.ts Outdated
Comment threadapps/server/src/assets/AssetAccess.ts
Comment threadapps/server/src/mcp/McpHttpServer.ts
Comment threadapps/web/src/components/chat/MessagesTimeline.tsx Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5445615. Configure here.

Comment threadapps/web/src/components/chat/MessagesTimeline.tsx
@macroscopeapp

macroscopeappBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a new feature for rendering saved media (images/videos) in chat, adding new asset types, security-sensitive path validation logic, and MCP tool capabilities for saving screenshots. The scope and security implications warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Comment threadapps/server/src/noFollowFile.ts Outdated
Comment threadapps/server/src/noFollowFile.ts
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gabrielelpidio@juliusmarminge@Noojuno
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: render saved media in chat across clients - #4872

Closed
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews
Closed

feat: render saved media in chat across clients#4872
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews

Conversation

@gabrielelpidio

@gabrielelpidiogabrielelpidio commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add secure signed asset access for saved screenshots, recordings, workspace files, and browser artifacts.
  • Let preview snapshots save to workspace paths or browser artifact storage.
  • Render saved media directly in web and desktop chat markdown, including expanded images and settled-turn galleries.
  • Preserve image-view paths in projected provider activity.
  • Add the corresponding mobile behavior: signed media resolution, inline images, the existing full-screen image viewer, and user-initiated recording controls.
  • Share markdown media path classification across web and mobile.

This combines the work from #4321 with mobile support in one PR targeting main.

Follow-up

Testing

  • The original Render saved screenshots and recordings in chat markdown #4321 focused suite passed 277 tests before the mobile split.
  • The mobile addition passed 44 focused shared/mobile markdown tests.
  • Shared, web, and mobile typechecks passed.
  • Targeted lint and formatting passed.
  • Mobile native static analysis passed; optional local SwiftLint, ktlint, and detekt binaries were unavailable and skipped.
  • Web production build passed.
  • Verified signed workspace images inline and expanded in the real web client.
  • Verified user and assistant images, expanded images, and MP4 previews on an iPhone 17 Pro simulator running iOS 26.5.
  • Re-ran the focused tests, package typechecks, targeted lint and formatting, and mobile native static analysis after rebasing onto the current main.

Visual evidence

  • Web: inline user/assistant images and the expanded-image dialog.
  • Mobile before: workspace-relative image nodes reserve space but remain blank.
  • Mobile after: images render in both message roles and open full-screen.
  • Mobile recordings: inline controls render in both message roles and playback remains user initiated.

Model: GPT-5.6-sol
Harness: T3 Code (Codex)

Note

Render saved media (images and videos) in chat across web and mobile clients

  • Adds end-to-end support for displaying image_view and imageGeneration media outputs in chat timelines on web and mobile, resolving asset URLs via thread-aware asset resolution.
  • Introduces thread-image and browser-artifact asset resource types with strict symlink-safe file access using no-follow semantics on all platforms (noFollowFile.ts).
  • Extends the messages timeline on web to emit image-output rows and inline assistantImageOutputs, with thumbnail galleries, loading/failure states, and click-to-expand previews (MessagesTimeline.logic.ts, MessagesTimeline.tsx).
  • Adds a MarkdownMedia component on both web (MarkdownMedia.tsx) and mobile (MarkdownMedia.tsx) that resolves workspace-relative and direct media sources and renders images or videos with unavailable fallback states.
  • Extends the preview_snapshot MCP tool to accept save/savePath parameters and persist screenshots to the workspace or a dedicated artifacts directory (handlers.ts).
  • Risk: asset resolution for thread-image re-canonicalizes paths on every request; post-issuance symlink swaps are detected and return null, not an error, so callers must handle silent resolution failures.

Macroscope summarized 275e7d9.


Note

High Risk
Touches signed asset serving, symlink-safe file I/O, and workspace path validation—security-sensitive paths with a large cross-stack surface (server, web, mobile, contracts).

Overview
Chat on web, mobile, and native iOS markdown can now show images and videos from markdown, workspace paths, browser artifacts, and tool image outputs—not just plain links or blank placeholders.

Signed asset access adds browser-artifact and thread-image resources, video preview extensions, and stricter workspace containment (e.g. ..name dirs vs real .. escapes). Sensitive reads/writes use no-follow open helpers on Linux/Darwin/fallback platforms so symlink swaps after token issuance don’t redirect serving or snapshot saves.

Preview preview_snapshot can persist PNGs via save (artifact dir) or savePath (workspace) with containment checks; tool docs steer agents to embed returned paths in replies.

Timeline derives deduplicated image galleries on terminal assistant rows (and trailing rows when needed), withholding in-flight turn images until the turn settles.

Provider projection keeps image-view/generation paths while still pruning huge base64 payloads from activity data.

Reviewed by Cursor Bugbot for commit 275e7d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitaiBot commented Jul 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27353ca3-545c-4bd2-9346-bed42f19dd44

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 29, 2026
@gabrielelpidiogabrielelpidio changed the title feat(mobile): render chat media previewsfeat: render saved media in chat across clientsJul 30, 2026
@gabrielelpidio
gabrielelpidio changed the base branch from t3code/render-chat-media to mainJuly 30, 2026 01:57
juliusmarmingeand others added 10 commits July 29, 2026 21:58
- Add secure asset access for workspace media and browser artifacts
- Support saving preview screenshots and embedding playable recordings
- Preserve error causes in screenshot save failures
- Reject save+savePath combinations and describe both schema fields
- Verify canonical (symlink-resolved) containment before workspace writes
- Add collision-resistant suffix to artifact screenshot names
- Fix Windows drive paths, protocol-relative URLs, query strings, and
extensionless video sources in markdown media resolution
- Only treat absolute browser-artifacts paths as server artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verify the deepest existing ancestor canonicalizes inside the workspace
before mkdir, and reject existing destination symlinks that resolve
outside the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Re-resolve browser-artifact and thread-image claims on every request so
post-issuance symlink swaps cannot redirect a signed token
- Verify canonical containment and media extension after realPath
- Render unscoped image outputs when no turn is unsettled
- Keep the image activity id paired with its path across collapsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drop readonly/idempotent hints from preview_snapshot now that it writes
- Treat only real .. segments as escapes when resolving workspace assets
- Append image galleries when turns share a target row
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload pruning (#4622) stripped every item field outside the command
allowlist, so image view activities reached clients without the path and
the chat gallery rendered nothing. Preserve the item type and path while
still dropping the base64 image bytes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gabrielelpidio
gabrielelpidioforce-pushed the t3code/mobile-chat-media-previews branch from 5b2f955 to 5445615CompareJuly 30, 2026 01:59
@github-actionsgithub-actionsBot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 30, 2026
Comment threadapps/server/src/mcp/toolkits/preview/handlers.ts Outdated
Comment threadapps/server/src/assets/AssetAccess.ts
Comment threadapps/server/src/mcp/McpHttpServer.ts
Comment threadapps/web/src/components/chat/MessagesTimeline.tsx Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5445615. Configure here.

Comment threadapps/web/src/components/chat/MessagesTimeline.tsx
@macroscopeapp

macroscopeappBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a new feature for rendering saved media (images/videos) in chat, adding new asset types, security-sensitive path validation logic, and MCP tool capabilities for saving screenshots. The scope and security implications warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Comment threadapps/server/src/noFollowFile.ts Outdated
Comment threadapps/server/src/noFollowFile.ts
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gabrielelpidio@juliusmarminge@Noojuno
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: render saved media in chat across clients - #4872

Closed
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews
Closed

feat: render saved media in chat across clients#4872
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews

Conversation

@gabrielelpidio

@gabrielelpidiogabrielelpidio commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add secure signed asset access for saved screenshots, recordings, workspace files, and browser artifacts.
  • Let preview snapshots save to workspace paths or browser artifact storage.
  • Render saved media directly in web and desktop chat markdown, including expanded images and settled-turn galleries.
  • Preserve image-view paths in projected provider activity.
  • Add the corresponding mobile behavior: signed media resolution, inline images, the existing full-screen image viewer, and user-initiated recording controls.
  • Share markdown media path classification across web and mobile.

This combines the work from #4321 with mobile support in one PR targeting main.

Follow-up

Testing

  • The original Render saved screenshots and recordings in chat markdown #4321 focused suite passed 277 tests before the mobile split.
  • The mobile addition passed 44 focused shared/mobile markdown tests.
  • Shared, web, and mobile typechecks passed.
  • Targeted lint and formatting passed.
  • Mobile native static analysis passed; optional local SwiftLint, ktlint, and detekt binaries were unavailable and skipped.
  • Web production build passed.
  • Verified signed workspace images inline and expanded in the real web client.
  • Verified user and assistant images, expanded images, and MP4 previews on an iPhone 17 Pro simulator running iOS 26.5.
  • Re-ran the focused tests, package typechecks, targeted lint and formatting, and mobile native static analysis after rebasing onto the current main.

Visual evidence

  • Web: inline user/assistant images and the expanded-image dialog.
  • Mobile before: workspace-relative image nodes reserve space but remain blank.
  • Mobile after: images render in both message roles and open full-screen.
  • Mobile recordings: inline controls render in both message roles and playback remains user initiated.

Model: GPT-5.6-sol
Harness: T3 Code (Codex)

Note

Render saved media (images and videos) in chat across web and mobile clients

  • Adds end-to-end support for displaying image_view and imageGeneration media outputs in chat timelines on web and mobile, resolving asset URLs via thread-aware asset resolution.
  • Introduces thread-image and browser-artifact asset resource types with strict symlink-safe file access using no-follow semantics on all platforms (noFollowFile.ts).
  • Extends the messages timeline on web to emit image-output rows and inline assistantImageOutputs, with thumbnail galleries, loading/failure states, and click-to-expand previews (MessagesTimeline.logic.ts, MessagesTimeline.tsx).
  • Adds a MarkdownMedia component on both web (MarkdownMedia.tsx) and mobile (MarkdownMedia.tsx) that resolves workspace-relative and direct media sources and renders images or videos with unavailable fallback states.
  • Extends the preview_snapshot MCP tool to accept save/savePath parameters and persist screenshots to the workspace or a dedicated artifacts directory (handlers.ts).
  • Risk: asset resolution for thread-image re-canonicalizes paths on every request; post-issuance symlink swaps are detected and return null, not an error, so callers must handle silent resolution failures.

Macroscope summarized 275e7d9.


Note

High Risk
Touches signed asset serving, symlink-safe file I/O, and workspace path validation—security-sensitive paths with a large cross-stack surface (server, web, mobile, contracts).

Overview
Chat on web, mobile, and native iOS markdown can now show images and videos from markdown, workspace paths, browser artifacts, and tool image outputs—not just plain links or blank placeholders.

Signed asset access adds browser-artifact and thread-image resources, video preview extensions, and stricter workspace containment (e.g. ..name dirs vs real .. escapes). Sensitive reads/writes use no-follow open helpers on Linux/Darwin/fallback platforms so symlink swaps after token issuance don’t redirect serving or snapshot saves.

Preview preview_snapshot can persist PNGs via save (artifact dir) or savePath (workspace) with containment checks; tool docs steer agents to embed returned paths in replies.

Timeline derives deduplicated image galleries on terminal assistant rows (and trailing rows when needed), withholding in-flight turn images until the turn settles.

Provider projection keeps image-view/generation paths while still pruning huge base64 payloads from activity data.

Reviewed by Cursor Bugbot for commit 275e7d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitaiBot commented Jul 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27353ca3-545c-4bd2-9346-bed42f19dd44

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 29, 2026
@gabrielelpidiogabrielelpidio changed the title feat(mobile): render chat media previewsfeat: render saved media in chat across clientsJul 30, 2026
@gabrielelpidio
gabrielelpidio changed the base branch from t3code/render-chat-media to mainJuly 30, 2026 01:57
juliusmarmingeand others added 10 commits July 29, 2026 21:58
- Add secure asset access for workspace media and browser artifacts
- Support saving preview screenshots and embedding playable recordings
- Preserve error causes in screenshot save failures
- Reject save+savePath combinations and describe both schema fields
- Verify canonical (symlink-resolved) containment before workspace writes
- Add collision-resistant suffix to artifact screenshot names
- Fix Windows drive paths, protocol-relative URLs, query strings, and
extensionless video sources in markdown media resolution
- Only treat absolute browser-artifacts paths as server artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verify the deepest existing ancestor canonicalizes inside the workspace
before mkdir, and reject existing destination symlinks that resolve
outside the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Re-resolve browser-artifact and thread-image claims on every request so
post-issuance symlink swaps cannot redirect a signed token
- Verify canonical containment and media extension after realPath
- Render unscoped image outputs when no turn is unsettled
- Keep the image activity id paired with its path across collapsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drop readonly/idempotent hints from preview_snapshot now that it writes
- Treat only real .. segments as escapes when resolving workspace assets
- Append image galleries when turns share a target row
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload pruning (#4622) stripped every item field outside the command
allowlist, so image view activities reached clients without the path and
the chat gallery rendered nothing. Preserve the item type and path while
still dropping the base64 image bytes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gabrielelpidio
gabrielelpidioforce-pushed the t3code/mobile-chat-media-previews branch from 5b2f955 to 5445615CompareJuly 30, 2026 01:59
@github-actionsgithub-actionsBot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 30, 2026
Comment threadapps/server/src/mcp/toolkits/preview/handlers.ts Outdated
Comment threadapps/server/src/assets/AssetAccess.ts
Comment threadapps/server/src/mcp/McpHttpServer.ts
Comment threadapps/web/src/components/chat/MessagesTimeline.tsx Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5445615. Configure here.

Comment threadapps/web/src/components/chat/MessagesTimeline.tsx
@macroscopeapp

macroscopeappBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a new feature for rendering saved media (images/videos) in chat, adding new asset types, security-sensitive path validation logic, and MCP tool capabilities for saving screenshots. The scope and security implications warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Comment threadapps/server/src/noFollowFile.ts Outdated
Comment threadapps/server/src/noFollowFile.ts
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gabrielelpidio@juliusmarminge@Noojuno
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat: render saved media in chat across clients - #4872

Closed
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews
Closed

feat: render saved media in chat across clients#4872
gabrielelpidio wants to merge 13 commits into
mainfrom
t3code/mobile-chat-media-previews

Conversation

@gabrielelpidio

@gabrielelpidiogabrielelpidio commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add secure signed asset access for saved screenshots, recordings, workspace files, and browser artifacts.
  • Let preview snapshots save to workspace paths or browser artifact storage.
  • Render saved media directly in web and desktop chat markdown, including expanded images and settled-turn galleries.
  • Preserve image-view paths in projected provider activity.
  • Add the corresponding mobile behavior: signed media resolution, inline images, the existing full-screen image viewer, and user-initiated recording controls.
  • Share markdown media path classification across web and mobile.

This combines the work from #4321 with mobile support in one PR targeting main.

Follow-up

Testing

  • The original Render saved screenshots and recordings in chat markdown #4321 focused suite passed 277 tests before the mobile split.
  • The mobile addition passed 44 focused shared/mobile markdown tests.
  • Shared, web, and mobile typechecks passed.
  • Targeted lint and formatting passed.
  • Mobile native static analysis passed; optional local SwiftLint, ktlint, and detekt binaries were unavailable and skipped.
  • Web production build passed.
  • Verified signed workspace images inline and expanded in the real web client.
  • Verified user and assistant images, expanded images, and MP4 previews on an iPhone 17 Pro simulator running iOS 26.5.
  • Re-ran the focused tests, package typechecks, targeted lint and formatting, and mobile native static analysis after rebasing onto the current main.

Visual evidence

  • Web: inline user/assistant images and the expanded-image dialog.
  • Mobile before: workspace-relative image nodes reserve space but remain blank.
  • Mobile after: images render in both message roles and open full-screen.
  • Mobile recordings: inline controls render in both message roles and playback remains user initiated.

Model: GPT-5.6-sol
Harness: T3 Code (Codex)

Note

Render saved media (images and videos) in chat across web and mobile clients

  • Adds end-to-end support for displaying image_view and imageGeneration media outputs in chat timelines on web and mobile, resolving asset URLs via thread-aware asset resolution.
  • Introduces thread-image and browser-artifact asset resource types with strict symlink-safe file access using no-follow semantics on all platforms (noFollowFile.ts).
  • Extends the messages timeline on web to emit image-output rows and inline assistantImageOutputs, with thumbnail galleries, loading/failure states, and click-to-expand previews (MessagesTimeline.logic.ts, MessagesTimeline.tsx).
  • Adds a MarkdownMedia component on both web (MarkdownMedia.tsx) and mobile (MarkdownMedia.tsx) that resolves workspace-relative and direct media sources and renders images or videos with unavailable fallback states.
  • Extends the preview_snapshot MCP tool to accept save/savePath parameters and persist screenshots to the workspace or a dedicated artifacts directory (handlers.ts).
  • Risk: asset resolution for thread-image re-canonicalizes paths on every request; post-issuance symlink swaps are detected and return null, not an error, so callers must handle silent resolution failures.

Macroscope summarized 275e7d9.


Note

High Risk
Touches signed asset serving, symlink-safe file I/O, and workspace path validation—security-sensitive paths with a large cross-stack surface (server, web, mobile, contracts).

Overview
Chat on web, mobile, and native iOS markdown can now show images and videos from markdown, workspace paths, browser artifacts, and tool image outputs—not just plain links or blank placeholders.

Signed asset access adds browser-artifact and thread-image resources, video preview extensions, and stricter workspace containment (e.g. ..name dirs vs real .. escapes). Sensitive reads/writes use no-follow open helpers on Linux/Darwin/fallback platforms so symlink swaps after token issuance don’t redirect serving or snapshot saves.

Preview preview_snapshot can persist PNGs via save (artifact dir) or savePath (workspace) with containment checks; tool docs steer agents to embed returned paths in replies.

Timeline derives deduplicated image galleries on terminal assistant rows (and trailing rows when needed), withholding in-flight turn images until the turn settles.

Provider projection keeps image-view/generation paths while still pruning huge base64 payloads from activity data.

Reviewed by Cursor Bugbot for commit 275e7d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitaiBot commented Jul 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27353ca3-545c-4bd2-9346-bed42f19dd44

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 29, 2026
@gabrielelpidiogabrielelpidio changed the title feat(mobile): render chat media previewsfeat: render saved media in chat across clientsJul 30, 2026
@gabrielelpidio
gabrielelpidio changed the base branch from t3code/render-chat-media to mainJuly 30, 2026 01:57
juliusmarmingeand others added 10 commits July 29, 2026 21:58
- Add secure asset access for workspace media and browser artifacts
- Support saving preview screenshots and embedding playable recordings
- Preserve error causes in screenshot save failures
- Reject save+savePath combinations and describe both schema fields
- Verify canonical (symlink-resolved) containment before workspace writes
- Add collision-resistant suffix to artifact screenshot names
- Fix Windows drive paths, protocol-relative URLs, query strings, and
extensionless video sources in markdown media resolution
- Only treat absolute browser-artifacts paths as server artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verify the deepest existing ancestor canonicalizes inside the workspace
before mkdir, and reject existing destination symlinks that resolve
outside the root.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Re-resolve browser-artifact and thread-image claims on every request so
post-issuance symlink swaps cannot redirect a signed token
- Verify canonical containment and media extension after realPath
- Render unscoped image outputs when no turn is unsettled
- Keep the image activity id paired with its path across collapsing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drop readonly/idempotent hints from preview_snapshot now that it writes
- Treat only real .. segments as escapes when resolving workspace assets
- Append image galleries when turns share a target row
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload pruning (#4622) stripped every item field outside the command
allowlist, so image view activities reached clients without the path and
the chat gallery rendered nothing. Preserve the item type and path while
still dropping the base64 image bytes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gabrielelpidio
gabrielelpidioforce-pushed the t3code/mobile-chat-media-previews branch from 5b2f955 to 5445615CompareJuly 30, 2026 01:59
@github-actionsgithub-actionsBot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 30, 2026
Comment threadapps/server/src/mcp/toolkits/preview/handlers.ts Outdated
Comment threadapps/server/src/assets/AssetAccess.ts
Comment threadapps/server/src/mcp/McpHttpServer.ts
Comment threadapps/web/src/components/chat/MessagesTimeline.tsx Outdated

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5445615. Configure here.

Comment threadapps/web/src/components/chat/MessagesTimeline.tsx
@macroscopeapp

macroscopeappBot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a new feature for rendering saved media (images/videos) in chat, adding new asset types, security-sensitive path validation logic, and MCP tool capabilities for saving screenshots. The scope and security implications warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Comment threadapps/server/src/noFollowFile.ts Outdated
Comment threadapps/server/src/noFollowFile.ts
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 deletes the V1 orchestration layer this PR builds on — apps/server/src/orchestration/**, provider/Layers/*Adapter.ts and provider/Services/** are removed and replaced by apps/server/src/orchestration-v2/**, with the IPC surface renamed to ORCHESTRATION_V2_WS_METHODS. The files this PR touches either no longer exist or are rewritten, so it can't be rebased — it would need reimplementing against the V2 adapters.

This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them.

Once #2829 merges, please rebase onto main, port the change to the V2 equivalent, and reopen (or open a fresh PR). Ping me and I'll prioritise the review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL1,000+ changed lines (additions + deletions).vouch:trustedPR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gabrielelpidio@juliusmarminge@Noojuno