fix: implement proper unicode-aware word wrapping for text overflow

[echobt]

Summary

Fixes text overflow issue where streaming text or conversation text sometimes extends beyond terminal borders.

Root Causes Fixed

1. Assistant messages (markdown-rendered)

The MarkdownRenderer in cortex-core/src/markdown/renderer did not implement word wrapping for paragraph text. The width parameter was only used for code blocks and tables, not for regular text paragraphs.

Fix: Added proper word wrapping in handlers.rs:

• New wrap_paragraph_text() method that wraps text at word boundaries
• New wrap_long_word() method for breaking oversized words
• Tracks current_line_width in state for accurate wrapping decisions
• Properly accounts for blockquote prefix width

2. User/System messages

The wrap_text function in cortex-tui/src/views/minimal_session/text_utils.rs used chars().count() for width calculation instead of Unicode-aware width (UnicodeWidthStr), causing incorrect wrapping for wide characters (CJK, emoji).

Fix: Replaced all chars().count() with UnicodeWidthStr::width():

• Added split_long_word_into_lines() and split_long_word_returning_last() helpers
• Proper visual width calculation for all text

3. cortex-core wrapping utilities

Same issue in cortex-core/src/widgets/chat/wrapping.rs.

Fix:

• Added split_at_visual_width() function for visual-width-aware splitting
• Updated all width calculations to use UnicodeWidthStr::width()
• Added tests for CJK character handling

Changes

• src/cortex-core/src/markdown/renderer/handlers.rs - Added paragraph word wrapping
• src/cortex-core/src/markdown/renderer/state.rs - Added current_line_width field
• src/cortex-core/src/widgets/chat/wrapping.rs - Unicode-aware width calculations
• src/cortex-tui/src/views/minimal_session/text_utils.rs - Unicode-aware width calculations

Testing

• All existing tests pass
• Added new tests for CJK character and visual width handling
• cargo check -p cortex-core -p cortex-tui passes
• cargo test -p cortex-core --lib wrapping - 8 tests pass

Acceptance Criteria Met

• ✅ Assistant messages (markdown-rendered) properly wrap to terminal width
• ✅ User and System messages properly wrap with Unicode-aware width calculation
• ✅ Wide characters (CJK, emoji) are measured correctly and don't cause overflow
• ✅ Streaming text respects terminal boundaries throughout the streaming process
• ✅ Code blocks and tables continue to work correctly (already have width handling)

[greptile-apps]

Greptile Overview

Greptile Summary

Fixed text overflow issues where streaming and conversation text extended beyond terminal borders by implementing Unicode-aware word wrapping across three subsystems.

Key Changes:

• Markdown rendering (handlers.rs): Added wrap_paragraph_text() and wrap_long_word() methods that wrap paragraph text at word boundaries, properly account for blockquote prefix width, and track visual column position via current_line_width
• State tracking (state.rs): Added current_line_width field to maintain accurate visual width tracking during paragraph rendering
• Width calculations (2 files): Replaced all chars().count() calls with UnicodeWidthStr::width() to correctly measure CJK characters (2 columns) and emoji

Testing:

• Added comprehensive tests for CJK character handling and visual width splitting
• All existing tests pass

Minor Issue:

• handle_soft_break() in paragraphs adds a space span without updating current_line_width, which could cause width tracking drift (see inline comment)

Confidence Score: 4/5

• Safe to merge with one minor width tracking issue in soft break handling
• The implementation correctly addresses the Unicode width calculation issues and adds proper word wrapping. However, there's a logic issue where soft breaks in paragraphs don't update current_line_width, which could cause incorrect wrapping decisions in edge cases with multiple soft breaks.
• Pay attention to handlers.rs - the soft break handler needs to update line width tracking for paragraphs

Important Files Changed

Filename	Overview
src/cortex-core/src/markdown/renderer/handlers.rs	Added Unicode-aware paragraph text wrapping with proper handling of blockquotes and word boundaries
src/cortex-core/src/markdown/renderer/state.rs	Added current_line_width field to track visual column width during paragraph wrapping
src/cortex-core/src/widgets/chat/wrapping.rs	Replaced character-based width calculations with Unicode-aware visual width using UnicodeWidthStr, added CJK tests
src/cortex-tui/src/views/minimal_session/text_utils.rs	Replaced character-based width calculations with Unicode-aware visual width using UnicodeWidthStr

Sequence Diagram

sequenceDiagram
participant User as User/System Message
participant TUI as cortex-tui text_utils
participant Core as cortex-core wrapping
participant MD as MarkdownRenderer
participant State as RenderState
participant UW as unicode-width
Note over User,UW: User/System Message Flow
User->>TUI: Display message text
TUI->>UW: UnicodeWidthStr::width(line)
UW-->>TUI: Visual width
TUI->>TUI: wrap_text() - split by words
TUI->>TUI: split_long_word_into_lines()
TUI->>UW: UnicodeWidthChar::width(ch)
UW-->>TUI: Character width
TUI-->>User: Wrapped lines
Note over User,UW: Assistant Message (Markdown) Flow
User->>MD: Display assistant message
MD->>State: Create RenderState
MD->>State: handle_event(Start(Paragraph))
State->>State: start_paragraph() - set current_line_width=0
loop For each text chunk
MD->>State: handle_event(Text)
State->>State: wrap_paragraph_text()
State->>UW: UnicodeWidthStr::width(word)
UW-->>State: Word width
alt Word fits on current line
State->>State: push_span() + update current_line_width
else Word too long for line
State->>State: flush_line() - reset current_line_width
State->>State: wrap_long_word()
State->>UW: UnicodeWidthChar::width(ch)
UW-->>State: Character width
end
end
MD->>State: handle_event(End(Paragraph))
State->>State: end_paragraph() - flush_line()
State-->>MD: Wrapped lines
MD-->>User: Rendered markdown
Note over Core: Common wrapping utilities
Core->>UW: UnicodeWidthStr::width()
Core->>Core: split_at_visual_width()
Core->>UW: UnicodeWidthChar::width(ch)

Loading

[greptile-apps]

_{4 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

src/cortex-core/src/markdown/renderer/handlers.rs
soft_break in paragraphs adds a space span but doesn't update current_line_width, potentially causing width tracking to drift out of sync

 pub(super) fn handle_soft_break(&mut self) {
if self.in_code_block {
self.code_buffer.push('\n');
return;
}
if !self.list_stack.is_empty() {
self.current_list_item.push(Span::raw(" "));
return;
}
// Treat soft break as space
self.push_span(Span::raw(" "));
if self.in_paragraph {
self.current_line_width += 1;
}
}

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/cortex-core/src/markdown/renderer/handlers.rs
Line: 458:471
Comment:
`soft_break` in paragraphs adds a space span but doesn't update `current_line_width`, potentially causing width tracking to drift out of sync
```suggestion pub(super) fn handle_soft_break(&mut self) { if self.in_code_block { self.code_buffer.push('\n'); return; } if !self.list_stack.is_empty() { self.current_list_item.push(Span::raw(" ")); return; } // Treat soft break as space self.push_span(Span::raw(" ")); if self.in_paragraph { self.current_line_width += 1; } }```
How can I resolve this? If you propose a fix, please make it concise.