Uh oh!
There was an error while loading. Please reload this page.
Add comprehensive E2E test suite - #350
Conversation
This commit adds a comprehensive end-to-end test suite using Playwright to verify the functionality of all major UI components and user flows. It includes: - `data-testid` attributes added to all interactive UI elements for reliable testing. - New test files for the header, map, sidebar, and mobile-specific UI. - Expanded tests for the chat functionality, including file attachments and new chat creation. - Fixes for placeholder code and removal of unnecessary test files. - A fix for a build error caused by an incorrect import path in `mapbox_mcp/hooks.ts`.
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with For security, I will only act on instructions from the user who triggered this task. |
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Comment |
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label | ||||||||||||||||||
ngoiyaeric
commented
Oct 23, 2025
@coderabbitai can you figure out how to achieve a comprehensive test suite of the whole applications current states and functionality and review the existing code. |
PR Code Suggestions ✨Explore these optional code suggestions:
| |||||||||||||||||
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 3069 🏁 Script executed: Length of output: 395 🏁 Script executed: Length of output: 10393 🏁 Script executed: Length of output: 684 🏁 Script executed: Length of output: 3191 🏁 Script executed: Length of output: 31645 🏁 Script executed: Length of output: 11962 🏁 Script executed: Length of output: 9510 🏁 Script executed: Length of output: 4862 🏁 Script executed: Length of output: 48 @ngoiyaeric, I'll provide a comprehensive analysis of the test suite and recommendations for achieving complete coverage of your application. Current Test Coverage AnalysisThe PR adds a solid foundation with 5 E2E test files covering:
Critical Gaps in Test Coverage1. API Endpoints (Not Tested)// tests/api/chat.spec.ts - NEW FILE NEEDEDimport{test,expect}from'@playwright/test';test.describe('Chat API',()=>{test('POST /api/chat creates a new chat',async({ request })=>{constresponse=awaitrequest.post('/api/chat',{data: {title: 'Test Chat',initialMessageContent: 'Hello API',role: 'user'}});expect(response.status()).toBe(201);constdata=awaitresponse.json();expect(data.chatId).toBeDefined();});test('GET /api/chats/all returns user chats',async({ request })=>{constresponse=awaitrequest.get('/api/chats/all');expect(response.ok()).toBeTruthy();constchats=awaitresponse.json();expect(Array.isArray(chats)).toBeTruthy();});});2. Search Functionality (Not Tested)// tests/search.spec.ts - NEW FILE NEEDEDimport{test,expect}from'@playwright/test';test.describe('Search Functionality',()=>{test('should display search results for a query',async({ page })=>{awaitpage.goto('/');awaitpage.fill('[data-testid="chat-input"]','climate change effects');awaitpage.click('[data-testid="chat-submit"]');// Wait for search results to appearawaitexpect(page.locator('.search-results')).toBeVisible({timeout: 20000});awaitexpect(page.locator('.search-results .result-item')).toHaveCount.greaterThan(0);});test('should display image results',async({ page })=>{awaitpage.goto('/');awaitpage.fill('[data-testid="chat-input"]','show me images of forests');awaitpage.click('[data-testid="chat-submit"]');awaitexpect(page.locator('[data-testid="search-images"]')).toBeVisible({timeout: 20000});});});3. Calendar/Notepad Features (Not Tested)// tests/calendar.spec.ts - NEW FILE NEEDEDimport{test,expect}from'@playwright/test';test.describe('Calendar and Notes',()=>{test.beforeEach(async({ page })=>{awaitpage.goto('/');awaitpage.click('[data-testid="calendar-toggle"]');awaitexpect(page.locator('[data-testid="calendar-notepad"]')).toBeVisible();});test('should add a note to current date',async({ page })=>{constnoteContent='Test note content';awaitpage.fill('textarea[placeholder*="Add note"]',noteContent);awaitpage.keyboard.press('Meta+Enter');awaitexpect(page.locator(`text=${noteContent}`)).toBeVisible();});test('should tag location in note',async({ page })=>{awaitpage.fill('textarea[placeholder*="Add note"]','Location test');awaitpage.click('button:has-text("MapPin")');awaitexpect(page.locator('textarea')).toContainText('#location');});test('should navigate between dates',async({ page })=>{constnextWeekButton=page.locator('button:has(svg.lucide-chevron-right)');awaitnextWeekButton.click();// Verify date changed (check for different date numbers)awaitpage.waitForTimeout(500);});});4. Authentication & Authorization (Not Tested)// tests/auth.spec.ts - NEW FILE NEEDEDimport{test,expect}from'@playwright/test';test.describe('Authentication',()=>{test('should redirect unauthenticated users',async({ page })=>{// Clear cookies/storageawaitpage.context().clearCookies();awaitpage.goto('/');// Verify redirect or login promptawaitexpect(page).toHaveURL(/login|auth/);});test('should access protected API routes when authenticated',async({ request })=>{constresponse=awaitrequest.get('/api/chats/all');expect(response.status()).not.toBe(401);});});5. Error Handling & Edge Cases (Not Tested)// tests/error-handling.spec.ts - NEW FILE NEEDEDimport{test,expect}from'@playwright/test';test.describe('Error Handling',()=>{test('should handle network errors gracefully',async({ page, context })=>{awaitpage.goto('/');// Simulate offlineawaitcontext.setOffline(true);awaitpage.fill('[data-testid="chat-input"]','Test message');awaitpage.click('[data-testid="chat-submit"]');// Verify error message or retry behaviorawaitexpect(page.locator('text=/error|failed|retry/i')).toBeVisible({timeout: 10000});});test('should handle file upload size limit',async({ page })=>{awaitpage.goto('/');// Try to upload file > 10MB (would need actual large file)constfileInput=page.locator('input[type="file"]');// This would trigger the alert in chat-panel.tsx});test('should prevent empty message submission',async({ page })=>{awaitpage.goto('/');constsubmitButton=page.locator('[data-testid="chat-submit"]');awaitexpect(submitButton).toBeDisabled();awaitpage.fill('[data-testid="chat-input"]',' ');// Only spacesawaitexpect(submitButton).toBeDisabled();});});6. Map Drawing & GeoJSON Features (Not Tested)// tests/map-advanced.spec.ts - NEW FILE NEEDEDimport{test,expect}from'@playwright/test';test.describe('Advanced Map Features',()=>{test('should save drawn features',async({ page })=>{awaitpage.goto('/');awaitpage.click('[data-testid="map-toggle"]');awaitpage.click('[data-testid="map-mode-draw"]');// Wait for draw controlsawaitexpect(page.locator('.mapboxgl-ctrl-draw-btn')).toBeVisible();// Simulate drawing (complex - may need custom helpers)// Verify features are saved to context});test('should load GeoJSON layers',async({ page })=>{awaitpage.goto('/');// Send a message that triggers GeoJSON displayawaitpage.fill('[data-testid="chat-input"]','Show me earthquake data');awaitpage.click('[data-testid="chat-submit"]');// Verify GeoJSON layer is rendered on mapawaitpage.waitForTimeout(3000);constcanvas=page.locator('.mapboxgl-canvas');awaitexpect(canvas).toBeVisible();});});7. Settings & Profile Management (Incomplete)// tests/settings.spec.ts - NEW FILE NEEDEDimport{test,expect}from'@playwright/test';test.describe('Settings',()=>{test('should navigate through all settings sections',async({ page })=>{awaitpage.goto('/');awaitpage.click('[data-testid="profile-toggle"]');// Test each sectionawaitpage.click('[data-testid="profile-settings"]');awaitexpect(page.locator('text=/model|settings/i')).toBeVisible();awaitpage.click('[data-testid="profile-toggle"]');awaitpage.click('[data-testid="profile-appearance"]');awaitexpect(page.locator('text=/appearance|theme/i')).toBeVisible();awaitpage.click('[data-testid="profile-toggle"]');awaitpage.click('[data-testid="profile-security"]');awaitexpect(page.locator('text=/security|password/i')).toBeVisible();});});8. Accessibility Tests (Missing)// tests/accessibility.spec.ts - NEW FILE NEEDEDimport{test,expect}from'@playwright/test';importAxeBuilderfrom'@axe-core/playwright';test.describe('Accessibility',()=>{test('should have no accessibility violations on home page',async({ page })=>{awaitpage.goto('/');constaccessibilityScanResults=awaitnewAxeBuilder({ page }).analyze();expect(accessibilityScanResults.violations).toEqual([]);});test('should be keyboard navigable',async({ page })=>{awaitpage.goto('/');// Tab through interactive elementsawaitpage.keyboard.press('Tab');awaitexpect(page.locator('[data-testid="profile-toggle"]')).toBeFocused();awaitpage.keyboard.press('Tab');// Continue testing keyboard navigation});test('should have proper ARIA labels',async({ page })=>{awaitpage.goto('/');constsubmitButton=page.locator('[data-testid="chat-submit"]');awaitexpect(submitButton).toHaveAttribute('aria-label','Send message');});});Test Quality ImprovementsIssues with Current Tests:
// ❌ BAD: Arbitrary timeoutawaitpage.waitForTimeout(500);// ✅ GOOD: Wait for specific conditionawaitpage.waitForFunction(()=>(windowasany).map.getZoom()>initialZoom);
// ❌ BAD: Assumes global window.map existsconstgetZoom=()=>page.evaluate(()=>(windowasany).map.getZoom());// ✅ GOOD: Check if map exists first, or use data attributesawaitpage.waitForSelector('.mapboxgl-canvas');constzoomLevel=awaitpage.getAttribute('[data-map-zoom]','value');
// ❌ BAD: Creates file but doesn't clean uprequire('fs').writeFileSync(filePath,'This is a test file.');// ✅ GOOD: Use fixtures and cleanupimport{testasbase}from'@playwright/test';importfsfrom'fs/promises';consttest=base.extend({testFile: async({},use)=>{constfilePath=path.join(__dirname,'test-file.txt');awaitfs.writeFile(filePath,'This is a test file.');awaituse(filePath);awaitfs.unlink(filePath).catch(()=>{});// Cleanup}});Recommended Testing Strategy1. Add Test Fixtures// tests/fixtures/test-data.tsexportconsttestMessages=[{role: 'user',content: 'Hello'},{role: 'assistant',content: 'Hi there!'}];exportconsttestChat={id: 'test-chat-id',title: 'Test Chat',messages: testMessages};2. Mock API Responses// tests/chat.spec.tstest('should display mocked chat response',async({ page })=>{awaitpage.route('**/api/chat',route=>{route.fulfill({status: 200,body: JSON.stringify({message: 'Mocked response'})});});awaitpage.goto('/');awaitpage.fill('[data-testid="chat-input"]','Test');awaitpage.click('[data-testid="chat-submit"]');awaitexpect(page.locator('text=Mocked response')).toBeVisible();});3. Add Visual Regression Tests// playwright.config.ts - add to config
expect: {toHaveScreenshot: {maxDiffPixels: 100}}// tests/visual.spec.tstest('homepage matches screenshot',async({ page })=>{awaitpage.goto('/');awaitexpect(page).toHaveScreenshot('homepage.png');});4. Add Performance Tests// tests/performance.spec.tsimport{test,expect}from'@playwright/test';test('page should load within 3 seconds',async({ page })=>{conststartTime=Date.now();awaitpage.goto('/');awaitpage.waitForLoadState('networkidle');constloadTime=Date.now()-startTime;expect(loadTime).toBeLessThan(3000);});5. Integration Test Example// tests/integration/chat-to-map.spec.tstest('should update map when sending location query',async({ page })=>{awaitpage.goto('/');// Send message with locationawaitpage.fill('[data-testid="chat-input"]','Show me New York');awaitpage.click('[data-testid="chat-submit"]');// Wait for bot responseawaitpage.waitForSelector('div.items-start',{timeout: 15000});// Verify map updated (check for map markers or center change)awaitexpect(page.locator('.mapboxgl-marker')).toBeVisible();});Recommended Test OrganizationImmediate Action Items
Would you like me to generate any of these specific test files, or would you prefer I create an issue to track this comprehensive testing roadmap? |
There was a problem hiding this comment.
Several E2E tests are brittle or incorrect: the mobile UI test queries a non-existent .mobile-icons-bar selector, and the mobile profile button is inert while the test expects a dropdown to open. Theme assertions rely on exact class equality, which is fragile, and the map zoom test relies on a global window.map that may not exist. The chat attachment test writes a file into the repo without cleanup; use the test output directory instead.
Additional notes (1)
- Readability |
tests/sidebar.spec.ts:26-28
This final assertion is prone to race conditions when clearing history asynchronously and only checks visibility. Verifying that the list has zero items is more robust and self-documenting.
Summary of changes
- Added
data-testidattributes across multiple UI components (chat panel, header, history, map toggle, mobile icons bar, mode/profile toggles) to support E2E testing. - Introduced Playwright test infrastructure (
playwright.config.ts) with multi-browser/device projects and a dev web server. - Added comprehensive E2E tests for chat, header, map, mobile, and sidebar flows.
- Fixed an import path in
mapbox_mcp/hooks.tsfromQCX/lib/utilsto@/lib/utils. - Updated
.gitignoreto exclude Playwright artifacts and logs; added@playwright/testto devDependencies.
| <Plus className="h-[1.2rem] w-[1.2rem]" /> | ||
| </Button> | ||
| <Button variant="ghost" size="icon"> | ||
| <Button variant="ghost" size="icon" data-testid="mobile-profile-button"> |
There was a problem hiding this comment.
This mobile profile button does not open any menu; it’s just an inert icon button. Your mobile.spec.ts expects a profile menu to open and check [data-testid="profile-account"], which will fail. Reuse the existing ProfileToggle (as in the header) to provide the dropdown on mobile as well.
Suggestion
Replace the standalone profile button with the shared ProfileToggle component:
// at top of the fileimport{ProfileToggle}from'./profile-toggle'// in the JSX, replace the button with:<ProfileToggle/>Update the mobile test to click [data-testid="profile-toggle"] if needed. Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this change
| <Paperclip className="h-[1.2rem] w-[1.2rem] transition-all rotate-0 scale-100" /> | ||
| </Button> | ||
| <Button variant="ghost" size="icon"> | ||
| <Button variant="ghost" size="icon" data-testid="mobile-submit-button"> |
There was a problem hiding this comment.
This submit button on mobile has no onClick and thus does nothing; adding a data-testid to it may encourage tests to click it expecting to send messages. Either wire it to the actual submit logic or hide/disable it until implemented to avoid misleading tests and UX.
Suggestion
If the submit action is not implemented yet, disable the button to avoid confusion:
<Buttonvariant="ghost"size="icon"data-testid="mobile-submit-button"disabledaria-disabled="true"><ArrowRightclassName="h-[1.2rem] w-[1.2rem]"/></Button>Or, preferably, wire it to the message submit handler (e.g., by passing an onSubmit prop). Reply with "@CharlieHelps yes please" if you'd like me to add a commit with the disabling change
| const html = page.locator('html'); | ||
| await expect(html).toHaveAttribute('class', 'dark'); | ||
| await page.click('[data-testid="theme-toggle"]'); | ||
| await page.click('[data-testid="theme-light"]'); | ||
| await expect(html).not.toHaveAttribute('class', 'dark'); |
There was a problem hiding this comment.
Using exact toHaveAttribute('class', 'dark') is brittle, since the <html> element may have multiple classes (e.g., Tailwind injects others). This can make the test flaky or falsely failing. Prefer checking classList contains dark.
Suggestion
Update the assertions to check for the presence/absence of the dark class rather than exact equality:
awaitexpect(html).toHaveClass(/(^|\s)dark(\s|$)/);// ... laterawaitexpect(html).not.toHaveClass(/(^|\s)dark(\s|$)/);Alternatively:
awaitexpect(awaithtml.evaluate(el=>el.classList.contains('dark'))).toBe(true);awaitexpect(awaithtml.evaluate(el=>el.classList.contains('dark'))).toBe(false);Reply with "@CharlieHelps yes please" if you'd like me to add a commit updating the tests
| const filePath = path.join(__dirname, 'test-file.txt'); | ||
| // Create a dummy file for the test | ||
| require('fs').writeFileSync(filePath, 'This is a test file.'); | ||
There was a problem hiding this comment.
This test writes a file into the source tree via require('fs').writeFileSync(...) and never cleans it up. It mixes require with ES imports and leaves local artifacts behind, which can pollute the working tree and cause flaky runs. Use Playwright’s test output folder and async FS APIs.
Suggestion
Use the test output directory and async FS:
import{promisesasfs}from'fs';constfilePath=test.info().outputPath('test-file.txt');awaitfs.writeFile(filePath,'This is a test file.');This keeps artifacts out of the repo and ensures per-test isolation. Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this change
| const getZoom = () => page.evaluate(() => (window as any).map.getZoom()); | ||
There was a problem hiding this comment.
This test assumes a global window.map to read zoom, which is fragile and will throw if the app doesn’t expose the Mapbox instance globally. Tests should not depend on hidden globals; either instrument the app in test mode to expose the map or guard/skip when unavailable.
Suggestion
Guard against missing globals or expose the map in test mode. For a minimal test-side fix:
consthasMap=awaitpage.evaluate(()=>Boolean((windowasany).map));if(!hasMap)test.skip(true,'Map instance not available on window for E2E');constgetZoom=()=>page.evaluate(()=>(windowasany).map.getZoom());Alternatively, instrument the app (in dev/test only) to assign the map instance to window.map after initialization. Reply with "@CharlieHelps yes please" if you'd like me to add a commit to apply the test-side guard
| webServer: { | ||
| command: 'bun run dev', | ||
| url: 'http://localhost:3000', | ||
| reuseExistingServer: !process.env.CI, | ||
| timeout: 600000, // 10 minutes | ||
| }, |
There was a problem hiding this comment.
Running E2E against the dev server (bun run dev) in CI tends to be slower and less deterministic (HMR, additional logging, non-optimized builds). Using a production build for CI improves reliability and performance.
Suggestion
Switch to a production server in CI while keeping dev locally:
webServer: {command: process.env.CI ? 'bun run build && bun run start' : 'bun run dev',url: 'http://localhost:3000',reuseExistingServer: !process.env.CI,timeout: 600000,},Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this suggestion.
This commit adds a comprehensive and robust end-to-end test suite using Playwright to verify the functionality of all major UI components and user flows. It includes: - `data-testid` attributes added to all interactive UI elements for reliable testing. - New test files for the header, map, sidebar, and mobile-specific UI. - Expanded tests for the chat functionality, including file attachments and new chat creation. - Fixes for placeholder code and removal of unnecessary test files. - A fix for a build error caused by an incorrect import path in `mapbox_mcp/hooks.ts`. - Significant improvements to the quality and reliability of the tests based on detailed code review feedback. This includes: - Using more robust assertions. - Handling temporary files correctly. - Adding guards for fragile tests. - Improving the Playwright configuration for CI environments. - Fixing bugs in the UI that were discovered during testing.
fix: patch critical vulnerability in React Server Components
- Regenerate bun.lock to fix JSON parsing error at line 2683 - Remove 40+ duplicate package entries that were causing build failures - Add packageManager field to package.json to ensure Vercel uses Bun consistently - This fixes the build failure in PR #350 where Vercel was detecting pnpm but using Bun Fixes build error: 'Expected "," but found end of file at bun.lock:2683:2'
- Regenerate bun.lock to fix JSON parsing error at line 2683 - Remove 40+ duplicate package entries that were causing build failures - Add packageManager field to package.json to ensure Vercel uses Bun consistently - This fixes the build failure in PR #350 where Vercel was detecting pnpm but using Bun Fixes build error: 'Expected "," but found end of file at bun.lock:2683:2'
This empty commit forces Vercel to bypass build cache and reinstall dependencies fresh from the regenerated bun.lock file. The zod/v3 module error is caused by cached node_modules from the previous corrupted lockfile state.
Removing pnpm-lock.yaml because: 1. Project uses Bun exclusively (configured in package.json) 2. pnpm-lock.yaml confuses Vercel's package manager detection 3. This file change will force Vercel to bypass cached node_modules 4. Fixes persistent zod/v3 import error from cached dependencies
The @modelcontextprotocol/sdk package requires zod ^3.25 || ^4.0 to access the /v3 and /v4-mini export paths. The previous version (3.22.4) did not export these paths, causing build failures. This upgrade resolves the 'Module not found: Package path ./v3 is not exported from package zod' error.
- Add GitHub Actions workflow for automated Playwright test execution - Add test scripts to package.json (test:e2e, test:e2e:ui, test:e2e:headed, test:e2e:debug) - Add calendar functionality tests (10 test cases) - Add image attachment and response tests (11 test cases) - Add responsive design tests across multiple viewports (15+ test cases) Tests now run automatically on: - Every push to main and develop branches - Every pull request to main and develop branches Test reports and screenshots are uploaded as GitHub Actions artifacts.
Uh oh!
There was an error while loading. Please reload this page.
This commit documents that the branch has been synchronized with the latest changes from main branch. All recent updates have been merged including: - Gemini 3 Pro model support (PR #389) - Image attachment token fixes (PR #388) - Comprehensive E2E test suite (PR #350) - Playwright GitHub Actions CI/CD - All dependency updates and bug fixes The Supabase backend implementation and collaboration features from this PR have been preserved and are compatible with the latest main branch changes.
User description
This change introduces a comprehensive end-to-end test suite to the project. It adds
data-testidattributes to all interactive UI elements and implements a wide range of tests for the header, chat, map, sidebar, and mobile UI. This will help to prevent regressions and ensure the stability of the application. It also includes a fix for a build error that was discovered during the verification process.PR created automatically by Jules for task 12672328258465556184
PR Type
Tests, Enhancement, Bug fix
Description
Add comprehensive Playwright E2E test suite with tests for chat, header, map, sidebar, and mobile UI
Add
data-testidattributes to all interactive UI elements for reliable test targetingFix incorrect import path in
mapbox_mcp/hooks.tsfromQCX/lib/utilsto@/lib/utilsConfigure Playwright with multi-browser support (Chromium, Firefox, WebKit) and mobile device testing
Diagram Walkthrough
File Walkthrough
1 files
Fix incorrect import path in hooks6 files
Add Playwright configuration for E2E testingAdd comprehensive chat functionality testsAdd header and navigation testsAdd map functionality and controls testsAdd mobile UI interaction testsAdd sidebar and chat history tests9 files
Add data-testid attributes to chat elementsAdd data-testid to header buttonsAdd data-testid to history item linkAdd data-testid to history panel elementsAdd data-testid to map toggle menu itemsAdd data-testid to mobile icon buttonsAdd data-testid to theme toggle elementsAdd data-testid to profile menu itemsAdd data-testid to history clear dialog buttons1 files
Add Playwright test dependency