feat: Replace y-webrtc with PartyKit for real-time collaboration - #45
Draft
Matia-R wants to merge 23 commits into
Draft
feat: Replace y-webrtc with PartyKit for real-time collaboration#45Matia-R wants to merge 23 commits into
Matia-R wants to merge 23 commits into
Conversation
- Add PartyKit server (party/document.ts) for Yjs room handling - Add API routes for PartyKit server to load/save documents - Add y-partykit client hook replacing WebrtcProvider - Add service role Supabase client for server-side persistence - Update editor to accept generic collaboration provider - Add PARTYKIT.md documentation - Update .env.example with PartyKit configuration Benefits: - Server-mediated sync (no more WebRTC connection failures) - Single persistence point (eliminates duplicate saves from clients) - Better scalability (no mesh connections between clients) The database schema remains unchanged for easy rollback. Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
- Client now passes Supabase JWT to PartyKit on connection - PartyKit verifies JWT expiry before allowing connection - API routes use user's JWT to create Supabase client - RLS is automatically enforced - no bypassing with service role - Remove service-role.ts (no longer needed) - Update documentation Security model: - User authenticates with Supabase, gets JWT - JWT passed to PartyKit via WebSocket query params - PartyKit forwards JWT to API routes - API routes create Supabase client with user's JWT - RLS enforces document permissions This eliminates the need for SUPABASE_SERVICE_ROLE_KEY. Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
Running 'npx partykit dev' downloads a fresh version that can't see local node_modules. Using 'npm run dev:partykit' uses the locally installed version which can resolve y-partykit properly. Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Add new document_state table (single table for full Y.Doc state) - Remove dependency on document_changes and document_snapshots - Simplify API routes to load/save full state - No more compaction needed - Update documentation with new schema Migration: Run migrations/partykit_document_state.sql Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
When isNew=true, the hook now calls /api/documents/create to create the document row and owner permission before connecting to PartyKit. This ensures: 1. Document exists when PartyKit tries to load/save 2. RLS permissions are set up correctly 3. getDocumentById works for document metadata Flow: 1. User clicks 'create document' 2. Hook detects isNew=true 3. Calls POST /api/documents/create with documentId 4. API creates document + permission via create_document_with_owner RPC 5. Then connects to PartyKit Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
Load API now handles document creation: - isNew=true + doc doesn't exist → create doc + owner permission - isNew=false + doc doesn't exist → 404 - doc exists → check permission, return state Save API simplified: - Just upserts state (doc already exists from Load) - Checks permission via RLS PartyKit server: - Passes isNew flag from client to Load API - Returns appropriate error codes (4003=forbidden, 4004=not found) Client hook: - Passes isNew flag to PartyKit via query params - Handles new 4004 error code for 'not found' Removed: - /api/documents/create route (no longer needed) This gives instant UX - document is created on first Load call, user can start editing immediately. Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
New documents now show nothing while connecting (~300ms) instead of the loading skeleton. This feels more instant since the editor just appears rather than skeleton → editor transition. Existing documents still show the loading skeleton as before. Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Only show DocumentLoadingSkeleton after 250ms delay - Fast loads (< 250ms) show nothing, feels instant - Slow loads (> 250ms) show skeleton for user feedback - Unified loading logic for both new and existing documents Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- System architecture with detailed diagrams - Data flow for connection, load, and save operations - Security model and JWT flow explanation - All user flows (new doc, existing doc, multi-tab) - Edge cases (duplicate tab, token expiry, network issues, etc.) - UX optimizations (delayed skeleton) - Caveats and current limitations - Future considerations for multi-user collaboration - Data migration guide Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Reset showSkeleton to false when documentId changes - Add documentId to useEffect dependency array - Increase SKELETON_DELAY_MS from 250ms to 500ms Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- scripts/migrate-to-partykit.ts migrates document_changes + document_snapshots → document_state - Supports dry-run mode, single document migration, and batch processing - Reuses existing Yjs reconstruction logic - Provides detailed progress and statistics - Updated .env.example with SUPABASE_SERVICE_ROLE_KEY placeholder - Updated PARTYKIT_ARCHITECTURE.md with migration instructions Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Added dotenv to load .env.local and .env files - Matches Next.js env loading behavior (.env.local takes precedence) Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Removed dotenv dependency - Added simple .env file parser using only Node.js built-ins - Works with npx tsx without needing npm install first Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Reverted to using dotenv for env file loading - Added npm run migrate:partykit script - Using npm script ensures project dependencies are available Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Use y-partykit's load option instead of managing separate Y.Doc - Pre-fetch document before calling onConnect to handle errors - Return loaded Y.Doc via load callback for y-partykit to use - This fixes migrated documents showing as empty Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Listen for Supabase auth state changes (TOKEN_REFRESHED) - Automatically reconnect with fresh token when old one expires - Retry up to 3 times on connection errors before showing error - Keep Y.Doc intact during reconnection (no data loss) - Add isReconnecting state for UI feedback - Show 'Reconnecting...' indicator instead of error during recovery - Show error banner only after all retries exhausted - Handle SIGNED_OUT event gracefully Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Track consecutive save failures (max 5) - Disable saving after max failures reached - Auto-reset failure count after 1 minute of no failures - Re-enable saving when a new client connects (fresh token) - Log failure progression for debugging Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Add useOnlineStatus hook to detect browser online/offline state - Show amber 'You're offline' banner when disconnected - Disable editor (read-only + reduced opacity) when offline - Clear messaging that editing is disabled until reconnect - Auto-recover when back online - Updated error banner messaging for consistency Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
…nnect Critical fixes: - Set isReady=false immediately when connection closes (disables editing) - Call provider.disconnect() before destroy() to stop internal reconnection - Start provider with connect:false and manually connect to control lifecycle - Clear retry timeouts on cleanup to prevent stale retries - Destroy provider before retrying to stop YPartyKitProvider's internal reconnection loop This prevents the infinite retry loop with expired tokens and ensures users cannot edit when the connection is lost. Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
- Log whether event is from current or old provider - Log status changes - Disconnect immediately in close handler - Help identify why reconnection loop persists Co-authored-by: Matia Raspopovic <matia@raspopovic.ca>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR replaces the peer-to-peer WebRTC sync (y-webrtc) with a server-mediated WebSocket architecture using PartyKit. This addresses several scalability concerns with the current CRDT implementation.
Problem
The current y-webrtc approach has limitations:
Solution
PartyKit provides:
document_statetable instead of changes + snapshotsDatabase Schema
Before (Complex):
After (Simple):
Architecture
Security Model
No service role key needed.
UX Optimizations
Migration
A migration script is included to convert existing documents from the old schema:
The script:
Files Changed
partykit.jsonparty/document.tssrc/hooks/use-collaborative-doc-partykit.tssrc/app/api/partykit/load/route.tssrc/app/api/partykit/save/route.tssrc/utils/supabase/from-token.tsmigrations/partykit_document_state.sqlscripts/migrate-to-partykit.tsDocumentation
PARTYKIT.md- Quick-start setup guidePARTYKIT_ARCHITECTURE.md- Comprehensive architecture documentation including:Setup
1. Run Database Migration
-- Run migrations/partykit_document_state.sql in Supabase2. Environment Variables
3. Local Development
4. Deploy