Uh oh!
There was an error while loading. Please reload this page.
fix(session): remove compaction buffer cap on reserved tokens - #32660
fix(session): remove compaction buffer cap on reserved tokens#32660hasitpbhatt wants to merge 1 commit into
Conversation
Fixes#32656 The usable() function was capping the output reservation at 20K (COMPACTION_BUFFER) for models with limit.input, while models without limit.input correctly reserved the full maxOutputTokens (typically 32K). This 12K gap meant compaction triggered later for input-limited models, creating an overflow risk on the next API call. Fixed by removing the Math.min(COMPACTION_BUFFER, ...) wrapper so both paths reserve the same headroom. The COMPACTION_BUFFER constant is now unused and deleted. Verified by adding a test that specifically exercises the 12K window (tokens between limit.input - 32K and limit.input - 20K) where the old code would incorrectly skip compaction but the new code correctly fires it.
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
The following comment was made by an LLM, it may be inaccurate: Based on the search results, I found one potentially related PR: #14393 - "fix: preserve thinking block signatures and fix compaction headroom asymmetry" This PR appears related as it also addresses compaction headroom asymmetry, which is the core issue in PR #32660. The asymmetry between models with and without The other results (PR #29860) are about bounding compaction request payloads, which is a different concern, and PR #32660 is the current PR itself. |
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
Fixes#32656
The \usable()\ function was capping the output reservation at 20K (COMPACTION_BUFFER) for models with \limit.input, while models without \limit.input\ correctly reserved the full \maxOutputTokens\ (typically 32K). This 12K gap meant compaction triggered later for input-limited models, creating an overflow risk on the next API call.
Fixed by removing the \Math.min(COMPACTION_BUFFER, ...)\ wrapper so both paths reserve the same headroom. The \COMPACTION_BUFFER\ constant is now unused and deleted.
Verified by adding a test that specifically exercises the 12K window (tokens between \limit.input - 32K\ and \limit.input - 20K) where the old code would incorrectly skip compaction but the new code correctly fires it.