feat: Implement task history scanner and recovery tools by KJ7LNW · Pull Request #5546 · RooCodeInc/Roo-Code

KJ7LNW · 2025-07-10T02:32:44Z

Note to Reviewer

This is a PR series, so the line numbers shown by Github are exaggerated. The commit series clearly marks where each PR begins using lines that say NOTICE: PR ____ STARTS HERE

The commits tell a clean story, it will be easier to understand what is happening here by looking at each commit individually under "Commits" than by looking at all of the files that were changed.

Dependencies

This pull request implements a comprehensive task history management system with UI tools for scanning, diagnosing, and repairing history issues.

Actual diff without tests or langs:

]$gitdiff-w--statgithub/refactor-use-files-for-history..github/refactor-use-files-for-history-scanner--':(exclude)**/__tests__/**'':(exclude)**/i18n/**'// new task scanner and rebuild implementationsrc/core/task-persistence/taskScanner.ts|594++++++++++++++++++++webview-ui/src/components/settings/HistoryIndexTools.tsx|770++++++++++++++++++++++++++// new message handlers for that purpose src/core/webview/webviewMessageHandler.ts|129+++++// minor plumbingpackages/types/src/history.ts|94++++src/core/webview/ClineProvider.ts|2+-src/shared/ExtensionMessage.ts|4+-src/shared/WebviewMessage.ts|9+-webview-ui/src/components/settings/SettingsView.tsx|6+8fileschanged,1605insertions(+),3deletions(-)

Context

This PR addresses issue #5545 by providing tools to recover tasks that were lost due to global state race conditions. While issue #5544 prevents future task loss, this PR focuses on recovering tasks that were already lost before that fix.

Implementation

Added a new task history scanner that can identify valid, orphaned, and missing tasks
Implemented a task reconstruction system that can recover lost tasks from UI message history
Created a mutex pattern to ensure history operations are mutually exclusive
Added bidirectional sync capabilities between file-based history and legacy globalState
Developed a comprehensive UI for managing and repairing task history

History Index Tools

Missing/Orphaned Task Preview

Orphaned Task Recovery

UI Index Scan Logs

UI Index Rebuild Logs

After Recovery

How to Test

Open the Settings panel and navigate to the History section
Use the "Scan Task History" button to identify any inconsistencies
Review the scan results to see valid, orphaned, and missing tasks
Use the repair options to reconstruct and reindex lost tasks

Fixes#5545

ellipsis-dev · 2025-07-10T02:34:03Z

⚠️ This PR is too big for Ellipsis, but support for larger PRs is coming soon. If you want us to prioritize this feature, let us know at help@ellipsis.dev

Generated with ❤️ by ellipsis.dev

ellipsis-dev · 2025-07-10T03:39:42Z

⚠️ This PR is too big for Ellipsis, but support for larger PRs is coming soon. If you want us to prioritize this feature, let us know at help@ellipsis.dev

Generated with ❤️ by ellipsis.dev

daniel-lxs · 2025-07-10T22:47:50Z

Hey @KJ7LNW,Thanks for the work on this! Do you think we should consider simplifying the UI and options a bit? Maybe a single button to restore tasks under the "About Roo Code" section or somewhere similar that just restores tasks for the current workspace?

I’m a bit concerned that the current options might be too complex for users who are just trying to recover lost tasks quickly. Let me know what you think.

KJ7LNW · 2025-07-11T17:38:31Z

I’m a bit concerned that the current options might be too complex for users who are just trying to recover lost tasks quickly. Let me know what you think.

I had the same thought about complexity for that interface, it can certainly be simpler. These options were mostly used for my testing, and end users probably just need a big "fix it" button.

What if I leave the check boxes to what they should default as, and just move them into the collapsed advanced section?

KJ7LNW · 2025-07-11T18:17:02Z

@daniel-lxs I am moving this into draft, but the dependent pull requests can still be reviewed and merged before this one is ready.

…ions - Added HistorySearchOptions interface to packages/types - Updated WebviewMessage to use historySearchOptions field instead of individual fields - Added historyItems message type to ExtensionMessage - Implemented getHistoryItems handler in webviewMessageHandler - Refactored getHistoryItemsForSearch to accept HistorySearchOptions parameter - Completely replaced client-side filtering with server-side filtering - Removed dependency on Fzf for client-side search - Added loading state to history components with loading spinner - Updated HistoryPreview to use limit parameter and respect loading state - Updated tests to account for the new loading state - Set explicit limits for history items in ChatView (10) and HistoryPreview (3) This refactoring improves performance by moving filtering to the server side, enhances type safety with the dedicated HistorySearchOptions type, reduces duplication in the interface definitions, and improves the user experience with loading indicators. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org> refactor: improve history search sorting and filtering - Created dedicated HistorySortOption type in shared types package - Modified API to take year and month as direct parameters - Added helper functions to reduce code duplication: - _getTasksByWorkspace to extract tasks from month data - _fastSortFilterTasks for efficient pre-filtering and sorting - Ensured consistent sorting across all functions - Optimized filtering to happen before file reads - Added support for custom sort order in getAvailableHistoryMonths Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

- Move fuzzy search from frontend to backend using fzf library - Create dedicated taskHistorySearch module with configurable parameters - Add match position tracking for proper highlighting in UI - Implement debounced search in frontend to prevent flickering Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org> fix: maintain sort order during search When a search string is present, the sort order specified by the user wasn't being respected. This change ensures that: - Non-relevance sorts (newest, oldest, etc.) maintain their order when searching - The 'mostRelevant' sort option continues to use fuzzy search order Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Implemented automatic refresh of the task history list when tasks are deleted: - Added taskDeletedConfirmation message type to WebviewMessage and ExtensionMessage - Modified webviewMessageHandler to send confirmation after task deletion - Updated useTaskSearch hook to listen for deletion confirmation and refresh the list - Implemented non-flickering refresh that maintains current search parameters Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

- Created SpinnerOverlay component to darken the view during deletion - Added state to track deletion in progress in HistoryView - Updated DeleteTaskDialog and BatchDeleteTaskDialog to trigger the overlay - Added event listener to hide the overlay when deletion completes Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Add request ID tracking to useTaskSearch hook to ensure each component only processes responses to its own search requests. This prevents the issue where multiple components using the hook would all receive updates when a search response comes back, regardless of which component initiated the search. - Add global serial counter to generate unique request IDs - Add component-isolated ref to track current request ID - Modify message handler to only process matching responses - Pass request ID back in webviewMessageHandler response Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Removed taskHistory field and all its references from the codebase as part of migrating to file-based storage. - Removed taskHistory from GlobalSettings schema - Removed import of historyItemSchema - Removed taskHistory from ExtensionState interface - Cleared PASS_THROUGH_STATE_KEYS array in ContextProxy - Updated ClineProvider to use file-based API instead of global state - Updated UI components to work without taskHistory prop Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Removed redundant useTaskSearch call from ChatView since HistoryPreview already makes its own call to fetch the tasks it needs to display. This eliminates an unnecessary API call on application startup and simplifies the component by removing conditional rendering based on task count. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Remove unnecessary loading and returning of entire task history array. The return value was never used by any caller, so we can make this an O(1) operation instead of O(n) by simply saving the single item. This change significantly improves performance when updating task history, which happens frequently during task execution. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Allows you to filter tasks not just by all and current, but also by any historic workspace directory that exists in existing HistoryItem metadata - Added persistent workspace index with metadata (path, name, missing status, timestamp) - Created a rich workspace selector UI with filtering and grouping capabilities - Added visual indicators for missing workspaces (strikethrough) - Improved loading states and feedback during workspace changes and searches - Added special workspace paths handling ("all", "current", "unknown") - Standardized empty/undefined workspace paths to "unknown" for legacy items that do not have workspace stored in their metadata - Optimized batch processing for better performance This enhancement provides users with a more intuitive and powerful way to navigate their task history across multiple workspaces. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Added a limit filter dropdown to the history view that allows users to control how many results are displayed. The filter: - Defaults to 50 items - Offers options for 50, 100, 200, 500, 1000 items or all results - Shows loading spinner when changing limits - Integrates with existing workspace and sort filters - Maintains consistent search options across operations Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

This change modifies the copy button in task history to retrieve the task content from the backend storage using getHistoryItem before copying it to the clipboard. This ensures the most up-to-date content is copied. Fixes: #3648 Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Implement a structured upgrade system that manages the task history migration process: - Create a dedicated upgrade UI that blocks normal app usage until migration completes - Separate migration check from migration execution for better control flow - Add progress logging during migration to provide user feedback - Remove automatic migration during extension activation - Add new message types for upgrade status and completion This change improves the user experience during task history migration by providing visual feedback and ensuring the app is in a consistent state before allowing normal usage. The upgrade system is designed to be extensible for future structural upgrades beyond task history migration. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

- Add tests for cross-workspace functionality - Verify items can be found in all workspaces where they existed - Ensure workspace property reflects the latest workspace - Add tests for helper functions and edge cases Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

- Removed pass-through state tests from ContextProxy that no longer apply - Updated ClineProvider tests to use file-based history instead of global state - Modified ChatTextArea tests to use useTaskSearch hook instead of taskHistory prop - Completely rewrote useTaskSearch tests to use message-based architecture - Updated other tests to remove taskHistory references from mock states Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org> test: Fix ClineProvider test by mocking extension context and taskHistory This commit fixes the failing test 'correctly identifies subtask scenario for issue #4602' by: 1. Adding necessary Vitest imports 2. Mocking getExtensionContext to return a mock context with globalStorageUri 3. Mocking taskHistory module to prevent file system operations during tests Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Added missing translations across all 17 supported locales: - upgrade.* keys in common.json for task history index upgrade UI - workspace.unknown, workspace.available, workspace.recent, workspace.filterPlaceholder in history.json - limit.* keys and noItemsFound in history.json for pagination controls All translations now complete with proper localization for: - Catalan, German, Spanish, French, Hindi, Indonesian, Italian - Japanese, Korean, Dutch, Polish, Portuguese (Brazil) - Russian, Turkish, Vietnamese, Chinese (Simplified/Traditional) Fixes missing UI text that would appear in English fallback Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

#5546

Implement a comprehensive task history management system with UI tools for scanning, diagnosing, and repairing history issues: - Add new HistoryScanResults and HistoryRebuildOptions interfaces - Refactor reindexHistoryItems into modular components: - scanTaskHistory: Identifies valid, orphaned, and missing tasks - rebuildIndexes: Rebuilds indexes with configurable options - reconstructTask: Recovers orphaned tasks from UI messages - Create new HistoryIndexTools UI component with: - Task history scanning capabilities - Configurable rebuild options (merge/replace modes) - Task preview and inspection tools - Real-time operation logging - Improve error handling and provide detailed logging - Update message handlers and type definitions - Add comprehensive translations for the new UI This change helps users diagnose and fix task history inconsistencies between global state and filesystem, preventing "lost" tasks and improving history reliability. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org> fix: prevent ReDoS vulnerability in log message regex Replace non-greedy wildcard pattern (.*?) with a more specific character class [^\]]* to avoid potential catastrophic backtracking on malicious input. This addresses a security vulnerability where the regex could run slow on strings starting with '[]' and containing many repetitions of '\t'. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org> fix: replace regex with string indexes in logMessage Replace regex pattern with string index operations in logMessage function to avoid potential ReDoS (Regular Expression Denial of Service) vulnerabilities. The change uses indexOf and substring instead of regex matching, which is more efficient and safer for processing log messages with tags. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org> refactor: implement mutual exclusion for history operations Created a mutex pattern to ensure history operations are mutually exclusive. This prevents concurrent execution of reindexHistoryItems and search operations, maintaining data consistency during indexing. Extracted common mutex logic into a reusable _withMutex helper function. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org> feat: add bidirectional sync with legacy globalState Adds capability to synchronize tasks between the new file-based history system and legacy VSCode globalState storage: - Renamed mergeGlobal to mergeFromGlobal for clarity - Added new mergeToGlobal option to update globalState with file index data - Added tracking of tasks that exist only in file indexes via tasksOnlyInTaskHistoryIndexes - Added developer UI controls in advanced section for managing globalState sync - Updated tests and documentation to reflect new bidirectional capabilities This feature enables backward compatibility testing with older versions of Roo Code by ensuring tasks created in the new storage format are also available in the legacy format. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org> refactor: make rebuildIndexes private by renaming to _rebuildIndexes This change makes the rebuildIndexes function private by adding an underscore prefix to its name, following the project's convention for private functions. All references to this function have been updated accordingly in both the implementation and test files. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Add translations for task history reindexing features across all supported languages: - Add backend translations for history reindex success/failure messages - Add frontend translations for the history index tools UI - Update settings and common translation files for all locales Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

Simplifies the task history recovery interface by moving all recovery and rebuild options into a single collapsed 'Advanced' section. This addresses feedback that the previous UI was too complex for users who just need to quickly restore tasks. The main 'Rebuild' and 'Rescan' action buttons remain directly visible, while all configuration checkboxes and mode selections are now neatly tucked away, available on-demand, and default to sensible values for a simple 'fix it' experience. Signed-off-by: Eric Wheeler <roo-code@z.ewheeler.org>

KJ7LNW · 2025-07-11T23:51:13Z

All configuration checkboxes and mode selections are now neatly tucked away, and default to sensible values for a simple 'fix it' experience:

ellipsis-dev · 2025-07-11T23:52:42Z

⚠️ This PR is too big for Ellipsis, but support for larger PRs is coming soon. If you want us to prioritize this feature, let us know at help@ellipsis.dev

Generated with ❤️ by ellipsis.dev

KJ7LNW · 2025-07-12T20:51:14Z

Closing this for now because it is not that important, but the following are critical, so please review them first:

1 of 3: memfix: implement and use safeReadJson to reduce memory usage #5332
2 of 3: corruption fix: Implement atomic read-modify-write for JSON files #5544
3 of 3: scalability fix: refactor task history persistence to use file-based storage #3785

KJ7LNW · 2025-07-10T02:32:44Z

Note to Reviewer

This is a PR series, so the line numbers shown by Github are exaggerated. The commit series clearly marks where each PR begins using lines that say NOTICE: PR ____ STARTS HERE

The commits tell a clean story, it will be easier to understand what is happening here by looking at each commit individually under "Commits" than by looking at all of the files that were changed.

Dependencies

This pull request implements a comprehensive task history management system with UI tools for scanning, diagnosing, and repairing history issues.

Actual diff without tests or langs:

]$gitdiff-w--statgithub/refactor-use-files-for-history..github/refactor-use-files-for-history-scanner--':(exclude)**/__tests__/**'':(exclude)**/i18n/**'// new task scanner and rebuild implementationsrc/core/task-persistence/taskScanner.ts|594++++++++++++++++++++webview-ui/src/components/settings/HistoryIndexTools.tsx|770++++++++++++++++++++++++++// new message handlers for that purpose src/core/webview/webviewMessageHandler.ts|129+++++// minor plumbingpackages/types/src/history.ts|94++++src/core/webview/ClineProvider.ts|2+-src/shared/ExtensionMessage.ts|4+-src/shared/WebviewMessage.ts|9+-webview-ui/src/components/settings/SettingsView.tsx|6+8fileschanged,1605insertions(+),3deletions(-)

Context

This PR addresses issue #5545 by providing tools to recover tasks that were lost due to global state race conditions. While issue #5544 prevents future task loss, this PR focuses on recovering tasks that were already lost before that fix.

Implementation

Added a new task history scanner that can identify valid, orphaned, and missing tasks
Implemented a task reconstruction system that can recover lost tasks from UI message history
Created a mutex pattern to ensure history operations are mutually exclusive
Added bidirectional sync capabilities between file-based history and legacy globalState
Developed a comprehensive UI for managing and repairing task history

History Index Tools

Missing/Orphaned Task Preview

Orphaned Task Recovery

UI Index Scan Logs

UI Index Rebuild Logs

After Recovery

How to Test

Open the Settings panel and navigate to the History section
Use the "Scan Task History" button to identify any inconsistencies
Review the scan results to see valid, orphaned, and missing tasks
Use the repair options to reconstruct and reindex lost tasks

Fixes#5545

ellipsis-dev · 2025-07-10T02:34:03Z

⚠️ This PR is too big for Ellipsis, but support for larger PRs is coming soon. If you want us to prioritize this feature, let us know at help@ellipsis.dev

Generated with ❤️ by ellipsis.dev

ellipsis-dev · 2025-07-10T03:39:42Z

⚠️ This PR is too big for Ellipsis, but support for larger PRs is coming soon. If you want us to prioritize this feature, let us know at help@ellipsis.dev

Generated with ❤️ by ellipsis.dev

daniel-lxs · 2025-07-10T22:47:50Z

Hey @KJ7LNW,Thanks for the work on this! Do you think we should consider simplifying the UI and options a bit? Maybe a single button to restore tasks under the "About Roo Code" section or somewhere similar that just restores tasks for the current workspace?

I’m a bit concerned that the current options might be too complex for users who are just trying to recover lost tasks quickly. Let me know what you think.

KJ7LNW · 2025-07-11T17:38:31Z

I’m a bit concerned that the current options might be too complex for users who are just trying to recover lost tasks quickly. Let me know what you think.

I had the same thought about complexity for that interface, it can certainly be simpler. These options were mostly used for my testing, and end users probably just need a big "fix it" button.

What if I leave the check boxes to what they should default as, and just move them into the collapsed advanced section?

KJ7LNW · 2025-07-11T18:17:02Z