Skip to content

Unify file upload: server-driven clipboard + tmux injection - #660

Merged
selfcontained merged 13 commits into
mainfrom
agt_16399979a72d/dispatch-dev
Jun 12, 2026
Merged

Unify file upload: server-driven clipboard + tmux injection#660
selfcontained merged 13 commits into
mainfrom
agt_16399979a72d/dispatch-dev

Conversation

@niiyeboah

@niiyeboahniiyeboah commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Server-side inject mode: POST /api/v1/agents/:id/media gains an inject=true form field. When set, the server decides the delivery mechanism (native clipboard for images when capable, or [File #N] <path> via tmux send-keys) and executes terminal injection itself — the client no longer manages delivery logic.
  • Client simplification: Replaces two separate upload callbacks (uploadAndInsertFiles for drag-drop, pasteImage for clipboard paste) with a single uploadFiles callback. Removes fileSeqRef, clipboardCapableRef, pasteImageRef, and the useSystemDefaults dependency from the terminal hook. Net -88 lines on the client.
  • Dead endpoint removal: Removes POST /api/v1/clipboard/image (184 lines) — its logic now lives in shared/lib/clipboard-write.ts, called by the media inject flow.
  • Paste extension fix: Clipboard-pasted images now get correct extensions via extensionForMime() — WebP gets .webp, GIF gets .gif (previously all non-PNG/JPEG got .jpg).
  • Bug fixes from review: code != null fix for signal-killed xclip, settled-race guard in clipboard write, PID-reuse TOCTOU fix in xclip reaper.

Test plan

  • pnpm run check — type check passes (server + web + scripts)
  • pnpm run finalize:web — production build succeeds
  • pnpm run test — unit tests pass
  • pnpm run test:e2e — 171 E2E tests pass (including updated drag-drop/paste tests)
  • Manual: drag-drop a text file onto terminal → [File #N] <path> appears in agent prompt
  • Manual: paste an image on macOS → native clipboard delivery (Ctrl+V sent to tmux)
  • Manual: paste an image on Linux without Xvfb → falls back to path-based delivery

🤖 Generated with Claude Code

@niiyeboah
niiyeboahforce-pushed the agt_16399979a72d/dispatch-dev branch from 3b00537 to 267824aCompareJune 9, 2026 17:19
@niiyeboah
niiyeboahforce-pushed the agt_16399979a72d/dispatch-dev branch from c5c1b98 to 8097c6fCompareJune 10, 2026 19:27
@niiyeboah

Copy link
Copy Markdown
CollaboratorAuthor

Update: addressed architecture + infra review (both approved ✅)

Ran author-requested architecture-review and infra-review passes on this branch. Both returned request_changes in round 1 and approve in round 2. All findings are fixed in eed75ca (+ a round-2 follow-up in 4eb8c4a).

Correctness / robustness

  • Replaced the 150ms paste heuristic with a deterministic readiness wait.waitForClipboardSelection() polls xclip -o -t TARGETS until xclip owns the selection and advertises the image mime (2s deadline), rejecting on timeout. A timeout now returns non-2xx so the client's path-based fallback fires — the previously silent, non-recoverable empty-paste race is gone.
  • uploadingFiles has a single owner per phase in pasteImage (cleared on native success and before handing off to the path-based fallback), fixing the premature "Uploading…" hide.

Maintainability

  • Extracted hostClipboardImageCapable() (shared/lib/clipboard-capability.ts) as the single source of truth for /system/defaults and /clipboard/image — closes the prior linux-vs-any-non-darwin divergence; the test asserts the helper.
  • Added a shared useSystemDefaults() React Query hook; migrated both use-terminal and create-agent-dialog off duplicate raw fetches.
  • Corrected the misleading /clipboard/image "not called / remove this route" comments; softened the [File #N] counter comment (per-mount, cosmetic).

Infra hardening

  • Rate limit (30/min) + 8MB size cap (→413) on /clipboard/image.
  • xclip reaper: SIGTERM the previous resident xclip before respawn; orphan-on-restart documented; round-2 PID-reuse TOCTOU closed by clearing the tracked PID on exit (4eb8c4a).
  • macOS temp file moved into a private mkdtemp(0700) dir (no predictable /tmp clobber).
  • Documented the shared-Xvfb cross-agent clipboard limitation at the DISPLAY export (per-session isolation scoped out).

Verification

Server + web type checks clean · 92 server unit tests pass · web production build succeeds.

@selfcontainedselfcontained changed the title Add drag-and-drop / paste file upload to the terminalUnify file upload: server-driven clipboard + tmux injectionJun 12, 2026
niiyeboahand others added 12 commits June 12, 2026 08:26
Drop files onto the agent terminal → upload to the media store, insert
`[File #N] <path>` (per-agent index, increments across drops). Drag-and-drop
is always path-based (works on headless VMs).
Image paste (Cmd/Ctrl+V) is hybrid: when the host can place the image on a
clipboard the agent CLI can read (macOS pasteboard, or Linux+Xvfb), it injects
natively (host clipboard + Ctrl+V → inline [Image #N]); otherwise it falls back
to the path-based upload. Capability is reported by GET /api/v1/system/defaults
(`clipboardImagePaste`); the agent env exports DISPLAY=$DISPATCH_COPY_DISPLAY so
the CLI can read the Xvfb clipboard.
Shiny status-gradient drop overlay + shared-ActivityBars uploading pill
(role=status). Hardened the /api/v1/clipboard/image endpoint (stdin EPIPE guard).
Incorporates frontend-ux / infra / architecture review fixes. Tests: command
builder (DISPLAY), system defaults (capability), e2e (drag→media, paste native
when capable, paste fallback when not).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Architecture review:
- Extract hostClipboardImageCapable() so /system/defaults and
/clipboard/image share one predicate (no drift; Linux branch now
gated on it instead of an open-ended else).
- Fix misleading /clipboard/image header + 150ms comments: the paste
hybrid DOES call this route; drop the "remove this route" guidance.
- pasteImage: single owner for the uploadingFiles flag per phase
(clear before delegating to the path-based fallback).
- Source clipboardImagePaste via a shared useSystemDefaults React Query
hook; migrate create-agent-dialog off its duplicate raw fetch.
- Soften the [File #N] counter comment (per-mount, cosmetic label).
Infra review:
- Replace the 150ms heuristic with a deterministic readiness poll
(xclip owns the selection + advertises the target) and reject on
timeout so the client can fall back — closes the silent, previously
non-recoverable paste-failure race.
- Track and SIGTERM the previous resident xclip before spawning a new
one; document the orphan-on-restart behavior.
- Rate-limit (30/min) and size-cap (8MB) /clipboard/image.
- macOS temp file now written into a private mkdtemp dir (no
predictable /tmp clobber); cleaned via rm.
- Document the shared-Xvfb / cross-agent clipboard limitation on the
DISPLAY export.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clear lastXclipPid in the proc 'exit' handler so a self-exited xclip's
(possibly recycled) PID is never SIGTERM'd on the next paste.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…injection
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…Files callback
Remove the dual uploadAndInsertFiles/pasteImage callbacks and their
supporting refs (fileSeqRef, clipboardCapableRef, useSystemDefaults)
from use-terminal.ts. Replace with a single uploadFiles callback that
delegates delivery entirely to the server via the inject flag. The
client no longer types paths into the terminal, tracks file numbering,
or manages clipboard routing — the server handles all of it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Clipboard-write logic moved to shared/lib/clipboard-write.ts (used by
the media inject flow). The client no longer routes to a separate
clipboard endpoint — all uploads go through the unified media endpoint.
E2E tests updated to match: the two clipboard-endpoint tests are
replaced by a single "pastes images via the media endpoint" test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use pasteText() (bracketed paste, no Enter) instead of sendCommand()
for [File #N] injection. This matches the clipboard path behavior —
both now paste without submitting, letting the user add context or
review before sending.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each upload E2E test now checks the media list API to confirm the DB
entry exists with correct name, source, and size — not just that the
request was sent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract UploadingOverlay into its own component that logs on mount. The
E2E test listens for that console message instead of injecting an 800ms
route delay. Also wait for the upload response (not just the request) so
the DB write has committed before asserting on media entries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@selfcontained
selfcontainedforce-pushed the agt_16399979a72d/dispatch-dev branch from 2372a35 to 7a82956CompareJune 12, 2026 14:32
The server-side inject flow calls hostClipboardImageCapable() directly —
the client no longer needs the capability flag. Drop it from the API
response, client type, and unit test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@selfcontained
selfcontained merged commit ae4e6f3 into mainJun 12, 2026
1 check passed
@selfcontained
selfcontained deleted the agt_16399979a72d/dispatch-dev branch June 12, 2026 15:25
selfcontained added a commit that referenced this pull request Jun 13, 2026
PR #660 unified file upload with server-driven clipboard + tmux injection,
but the in-app Media docs didn't mention drag-and-drop or clipboard paste
onto the terminal. Added an "Uploading files to agents" section covering
both methods and updated the sidebar Share file paragraph to distinguish
non-injected uploads. Added a file-upload ambient tip for feature discovery.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
selfcontained added a commit that referenced this pull request Aug 11, 2026
- Fall back to a generic reason ("This action is currently unavailable.")
in the caption slot when a disabled shortcut has no caption of its own —
the tooltip alone isn't reachable on touch devices, so a caption-less
disabled pin previously went inert with zero visible explanation.
- Swap in a Ban icon for a disabled shortcut, ahead of the destructive
AlertTriangle swap, so it reads distinctly from the other blocked states
(agent-not-running, no-stable-id) that share the same dimmed styling.
- Rename ShortcutPinItem's local `disabled` prop to `agentUnavailable` to
stop it colliding, in reading, with the new `pin.disabled` field.
Addresses review #660 items #1415, #1416, #1417.
selfcontained added a commit that referenced this pull request Aug 11, 2026
* Add a disabled state to shortcut pins
Retiring a shortcut pin's action currently means deleting the pin — losing
the context of what was offered. Add disabled: boolean (shortcut pins only,
alongside icon/variant/confirm) so an agent can grey out a shortcut instead:
the button renders non-interactive (aria-disabled, dimmed, cursor-not-allowed,
no click delivery), and the existing caption field doubles as the reason
shown under it (e.g. "already building — agt_...").
- pin-merge.ts: disabled joins the shortcut-only fields stripped on re-type
- pin-run.ts / mcp-handlers.ts: server-side guard refuses to run a disabled
pin even if the client bypasses the greyed-out button
- pins-panel.tsx: disabled pins get their own tooltip reason, independent of
the existing "agent not running" / "no stable ID" unavailable states
- dispatch_pin tool description + schema document the new field
Motivated by an idea from the shortcut-pins dogfooding thread: an idea's
launch pin has to be deleted once its builder is already running.
* Address frontend-ux-review feedback on disabled pins
- Fall back to a generic reason ("This action is currently unavailable.")
in the caption slot when a disabled shortcut has no caption of its own —
the tooltip alone isn't reachable on touch devices, so a caption-less
disabled pin previously went inert with zero visible explanation.
- Swap in a Ban icon for a disabled shortcut, ahead of the destructive
AlertTriangle swap, so it reads distinctly from the other blocked states
(agent-not-running, no-stable-id) that share the same dimmed styling.
- Rename ShortcutPinItem's local `disabled` prop to `agentUnavailable` to
stop it colliding, in reading, with the new `pin.disabled` field.
Addresses review #660 items #1415, #1416, #1417.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@niiyeboah@selfcontained