Skip to content

fix(opencode): make text/reasoning delta accumulation O(N) instead of O(N²) - #42150

Open
hardes11 wants to merge 2 commits into
anomalyco:devfrom
hardes11:fix/session-processor-chunked-text-accumulation
Open

fix(opencode): make text/reasoning delta accumulation O(N) instead of O(N²)#42150
hardes11 wants to merge 2 commits into
anomalyco:devfrom
hardes11:fix/session-processor-chunked-text-accumulation

Conversation

@hardes11

@hardes11hardes11 commented Aug 12, 2026

Copy link
Copy Markdown

Issue for this PR

Fixes#30067

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Resubmission of #30058 by meitalbensinai (auto-closed for missing PR template; commit cherry-picked unchanged, authorship preserved).

Replaces text += delta accumulation in the reasoning-delta and text-delta handlers of session/processor.ts with a lazy chunked-text getter. Once anything reads .text between writes (UI render loop, NDJSON serializer, bus event broadcaster), the runtime flattens the rope string and the next += re-copies the whole buffer — O(N²) in cumulative part length. On thinking-mode models emitting 1500+ small reasoning tokens per turn, this grows per-step latency from ~6s to 100s+ by turn 50-80 (perf evidence in #30067: memmove at ~30% of CPU, GC pinned). Same shape as the fix that landed in vercel/ai for processUIMessageStream.

The getter is non-enumerable, so it does not leak through JSON.stringify/structuredClone; .text reassignment still works via the setter.

How did you verify your code works?

  • Cherry-picked the original commit onto current dev and resolved the conflict in the import region (dev had since dropped an unused log variable; the incoming function is the only addition).
  • Diff is one file, +48/-2; confirmed no += accumulation remains in either handler and no orphaned references were introduced.
  • Ran a local harness (Bun 1.3.14) executing the exact installChunkedText implementation from this diff:
    • Functional: after 80k appended deltas, .text is byte-identical to naive += accumulation; the getter caches the joined string after first read; the setter resets the buffer and subsequent pushes accumulate correctly; _chunks is absent from JSON.stringify and object spread (non-enumerable, no leakage).
    • Performance: with flatten-forcing reads (JSON.stringify every 100 deltas), the chunked path is ~2.9x faster at 80k deltas / 1.6MB accumulated text, and the naive path re-copies the whole buffer on every append once a read forces flatten, so the gap widens with length.
  • Did not build the app or run the full test suite; the harness exercises the changed mechanism in isolation, not the whole binary.

Screenshots / recordings

n/a

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actionsgithub-actionsBot added needs:compliance This means the issue will auto-close after 2 hours. and removed needs:compliance This means the issue will auto-close after 2 hours. labels Aug 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@hardes11
hardes11force-pushed the fix/session-processor-chunked-text-accumulation branch from bd8da24 to 37c4c0fCompareAugust 13, 2026 14:02
renekris added a commit to renekris/opencode-lowmem that referenced this pull request Aug 22, 2026
…co#42150anomalyco#42176anomalyco#43881anomalyco#43607)
- O(N) text/reasoning delta accumulation instead of O(N^2) string
concat (anomalyco#42150) — the lazy chunk buffer joins on read
- finish reason 'error' is set when a stream fails mid-flight (anomalyco#42176)
- clean-EOF empty provider streams retry like transient errors (anomalyco#43881),
narrowed from the upstream patch: only an attempt that produced no
text/reasoning delta and no tool call qualifies, so providers that
stream content but omit usage/finish are not retried into duplicate
output
- SSE comment heartbeats no longer reset the chunk timeout (anomalyco#43607);
the streaming TextDecoder handles multi-byte characters split across
reads (regression covered)
renekris added a commit to renekris/opencode-lowmem that referenced this pull request Aug 23, 2026
Three fixes: coalesce message.part.delta writes through a bounded
buffer (TUI twin of upstream anomalyco#42150), publish PartUpdated with a
shallow copy instead of structuredClone (upstream anomalyco#35107), and drop
message/part/diff/status buckets on session.deleted (upstream anomalyco#12351).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

session/processor: text/reasoning delta accumulation is O(N²); long agent loops hang at 50-80 turns

2 participants

@hardes11@meitalbensinai