Skip to content

Add per-agent whiteboard with Excalidraw thin relay architecture - #752

Merged
selfcontained merged 6 commits into
mainfrom
agt_9e9d7d8b039e/whiteboard-thin-relay-excalidraw
Jul 28, 2026
Merged

Add per-agent whiteboard with Excalidraw thin relay architecture#752
selfcontained merged 6 commits into
mainfrom
agt_9e9d7d8b039e/whiteboard-thin-relay-excalidraw

Conversation

@selfcontained

@selfcontainedselfcontained commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds a per-agent whiteboard powered by Excalidraw with a thin relay architecture — agents construct raw Excalidraw element JSON directly via MCP tools, the server merges by id and persists to Postgres, and the frontend renders an interactive canvas with live bidirectional sync via SSE.
  • Three MCP tools: whiteboard_update (with embedded Excalidraw element format cheat sheet), whiteboard_get, and whiteboard_clear. REST endpoints for user-side scene saves with optimistic locking (version check + 409 on conflict).
  • Frontend: lazy-loaded Excalidraw component in a new "Whiteboard" center pane tab, self-hosted fonts via Vite plugin, debounced save (1s) and snapshot export (4s), pointer-down deferral for remote updates, "agent drew" violet dot indicator.
  • restoreElements hydration fix so the board doesn't go blank after agent updates.

New files

  • apps/server/src/db/migrations/0030_whiteboards.sql — whiteboards table
  • apps/server/src/shared/whiteboard.ts — shared types + simplifyElements()
  • apps/server/src/shared/whiteboard-store.ts — persistence layer
  • apps/server/src/shared/mcp/whiteboard-tools.ts — MCP tool definitions
  • apps/server/src/server/mcp-whiteboard-handlers.ts — thin relay handlers
  • apps/server/src/routes/whiteboard.ts — REST endpoints
  • apps/web/src/hooks/use-whiteboard.ts — React Query hook
  • apps/web/src/components/app/whiteboard-tab.tsx — Excalidraw canvas
  • apps/web/src/components/app/whiteboard-pane.tsx — lazy mount wrapper

Test plan

  • Unit tests for simplifyElements, isValidScene, handler merge/upsert/delete/retry/overflow logic
  • E2E tests: tab navigation, REST PUT/GET persistence, optimistic locking 409, MCP tool round-trip
  • pnpm run check — type checking passes
  • pnpm run finalize:web — production build succeeds

🤖 Generated with Claude Code

@selfcontained
selfcontainedforce-pushed the agt_9e9d7d8b039e/whiteboard-thin-relay-excalidraw branch 2 times, most recently from 3674a51 to 22ec0adCompareJuly 27, 2026 18:53
selfcontainedand others added 6 commits July 27, 2026 18:13
Agents construct raw Excalidraw element JSON directly via MCP tools
(whiteboard_update, whiteboard_get, whiteboard_clear). The server is a
thin relay — it merges elements by id, persists to Postgres with
optimistic locking, and pushes updates via SSE. The frontend renders
an interactive Excalidraw canvas with live sync in both directions.
Key pieces:
- Migration 0030: whiteboards table (agent_id PK, JSONB scene, version)
- MCP tools with embedded Excalidraw element format cheat sheet (~200 lines)
- REST endpoints for user-side scene save with optimistic locking
- Whiteboard tab in center pane with lazy-loaded Excalidraw component
- Self-hosted Excalidraw fonts via Vite plugin (excluding CJK for 12MB savings)
- SSE-driven React Query invalidation with "agent drew" dot indicator
- 31 unit tests + 4 E2E tests covering REST API, MCP tools, and UI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When the whiteboard tab is hidden (display:none) and an agent updates
the scene via MCP, Excalidraw's updateScene() updates internal state
but the canvas can't repaint at zero dimensions. Call refresh() when
the tab becomes visible so the canvas repaints with the latest content.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…dration
Raw server elements lack Excalidraw internal properties (seed, version,
roughness, opacity, strokeWidth, etc.) needed for rendering. Use
restoreElements() to hydrate sparse server elements before updateScene()
so the canvas renders correctly without requiring a page reload.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ct merge, split pane
- Rename migration 0030_whiteboards → 0037_whiteboards to avoid collision with 0030_agent-messages
- Invalidate whiteboard query for all SSE sources, not just agent-originated
- Handle 409 save conflicts with local-first element merge and retry
- Merge remote elements into local scene when user has unsaved edits
- Prevent whiteboard tab from being dragged into split pane mode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When applyRemote merges local + remote elements (dirty path), don't
update sceneVersionRef — the rescheduled save must still detect a diff
to persist the merged scene. Clean path (no pending edits) continues
to update sceneVersionRef normally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…e library/hints/help, add tip
- Remove diffStats from tab bar, relocate diff count badge to agents-view header
- Enable whiteboard tab drag for split pane; add whiteboard render path to CenterPaneSplit
- Remove whiteboard guards from use-split-pane enterSplit/handleTabDrop
- Hide Excalidraw library UI, toolbar hints, and help button via CSS overrides
- Add whiteboard inline + ambient tip (since 0.30.1)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@selfcontained
selfcontainedforce-pushed the agt_9e9d7d8b039e/whiteboard-thin-relay-excalidraw branch from 07ac854 to b29dec7CompareJuly 28, 2026 00:18
@selfcontained
selfcontained merged commit a0f96ed into mainJul 28, 2026
1 check passed
@selfcontained
selfcontained deleted the agt_9e9d7d8b039e/whiteboard-thin-relay-excalidraw branch July 28, 2026 00:26
selfcontained added a commit that referenced this pull request Jul 28, 2026
- docs-pane Agents section: new Whiteboard subsection (live two-way sync,
MCP tools, PNG snapshot caveat, agent-drew dot); split-pane copy now
includes the Whiteboard tab
- docs-pane Repo Tools: add whiteboard_get/update/clear to the built-in
tools list
- docs-pane Reviewers: note reviewers have read-only whiteboard_get (their
own board)
- tips.ts: point the whiteboard tip at the new agents#whiteboard anchor
- README: whiteboard feature bullet, interactive-agent tool table rows,
and corrected persona tool list (was missing pin/media list tools)
- docs/03-api-spec.md: Whiteboard REST section + whiteboard.changed SSE row
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
selfcontained added a commit that referenced this pull request Jul 29, 2026
Covers the previously untested whiteboard-era routing pieces:
- agent-routes.test.ts pins the three agent URL helpers to their literal
paths (the contract shared by the router config, useMatch patterns, and
navigate call sites).
- use-agents-view-routing.test.tsx drives the hook through a real
MemoryRouter: invalid-agent redirect gating (loading / validated id) and
its replace semantics probed via history back-navigation, legacy
feedback/review deep-link collapse, tab match flags, and onTabChange
navigation including its replace behavior and no-agent no-op.
- use-whiteboard.test.tsx covers the query-key shape, the enabled gate on
a null agent id, the versioned endpoint URL, staleTime-Infinity remount
caching, and per-agent cache separation.
A 13-mutant battery against the three source files is fully caught.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
selfcontained added a commit that referenced this pull request Aug 3, 2026
* Split agents-view into header component + layout/expansion hooks
agents-view.tsx regrew to 952 lines after the split-pane (#743/#800) and
whiteboard (#752) features landed. Extract the three growth clusters:
- use-center-pane-layout.ts: split-pane state, tab drag-and-drop plumbing,
and the stable terminal container reparenting portal logic
- agents-view-header.tsx: the h-14 header bar (sidebar toggle, quick
phrases, diff-stats badge, center tab bar, media toggle, reconnect scan)
- use-expanded-agent.ts: persisted expanded-card state and the
selection-follow effect
Pure structural refactor; rendered output and behavior are identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Preserve effect registration order for expanded-agent sync
Review feedback: the persistence and selection-follow effects originally
registered late in AgentsView's effect sequence. Split them into
useExpandedAgentSync, called at the original effect position, so passive
effect ordering is unchanged. useExpandedAgent keeps only the state and
toggle at the original useState position.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@selfcontained