Uh oh!
There was an error while loading. Please reload this page.
feat(runtime): report a provider dropping context across the send boundary - #4669
Conversation
Astro-Han
left a comment
There was a problem hiding this comment.
The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.
One change I would make before merging, and it is one line.
Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.
The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.
The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.
The third is the one in your self-review, editing or branching history between turns.
The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.
The three new tests pass on the old code
All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.
Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.
Two smaller ones
The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.
contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.
Shape
This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.
Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.
Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.
AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.
简体中文
追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。
合并前我想改一处,一行的事。
跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。
我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumber 和 pruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226 的 <= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。
第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。
第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。
#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。
三个新测试在旧代码上都会通过
三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 4000 和 3200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。
改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。
两条小的
这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokens 和 priorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。
contextProviderDroppingReported 是 AiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。
形状
这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。
把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。
Joob1n
commented
Sep 3, 2026
Ready for review. CI is green. @me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too. @likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself. |
Astro-Han
commented
Sep 3, 2026
Severities for the review above, which I should have included with it. P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. P3 — once per backend activation, not once per session. No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small. 简体中文上面那份 review 的分级,应该跟着一起给的。 P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 P3 —— 实际是每次 backend 激活一次,不是每会话一次。 无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。 |
Astro-Han
left a comment
There was a problem hiding this comment.
Putting the local ones inline with their grades. Reasoning is in my review above.
| Number.isFinite(stepUsage.inputTokens) && | ||
| stepUsage.inputTokens > 0 && | ||
| stepUsage.inputTokens <= lastStepInputTokens | ||
| stepUsage.inputTokens <= priorInput |
There was a problem hiding this comment.
P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.
The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.
The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.
The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.
| const toolSchemaShrank = | ||
| lastStepActiveToolCount !== undefined && | ||
| activeToolsForRequest.length < lastStepActiveToolCount; | ||
| // Across the send boundary the comparison is the same one, |
There was a problem hiding this comment.
P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.
The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.
| /** | ||
| * The provider has been reported dropping context in this session. | ||
| * | ||
| * Session-scoped, not per send: the condition persists once a provider |
There was a problem hiding this comment.
P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.
Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.
| await runFixtureTurn(fixture, consumer); | ||
| const note = fixture.messages.find( | ||
| (message): message is { type: 'system_note'; kind: string } => |
There was a problem hiding this comment.
P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.
This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.
Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.
| if (persisted) { | ||
| state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0); | ||
| state.lastAcceptedTotalTokens = state.baselineTokens; | ||
| state.priorAcceptedInputTokens = persisted.inputTokens; |
There was a problem hiding this comment.
P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.
Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.
94b4032 to
82e454eCompare…ndary The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there: a provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns, and a send of one or two steps has no earlier step to compare with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight turns with nothing reported, which is the case the note was written for. The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor, which is route-validated where it is read. A fold before that request would explain a smaller input by itself, so it disables the comparison, as prunes, image omissions and a shrinking tool set already do. The note is now reported once per session rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told. Refs apache#4559, apache#4623 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
82e454e to
cb7a4ddCompareJoob1n
commented
Sep 3, 2026
All four are addressed on The false positive. Your one-line fix is the one I took: equality across the boundary, The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to The numbers. The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question. |
Astro-Han
left a comment
There was a problem hiding this comment.
Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.
The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.
The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.
Approving. Merging when the checks land.
Joob1n
commented
Sep 3, 2026
@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what The two P2s in particular:
P3s: the note carries @likun666661 — your approval predates all of this, so it should not carry: the comparison changed from
|
Uh oh!
There was an error while loading. Please reload this page.
Summary
The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.
A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.
The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.
Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.
The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.
The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.
Refs #4559, #4623
Not in this PR
"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.
Verification
mid-turn-capacity-backend76/76,overflow-reactive-recovery50/50,context-budget7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to
<=fails the shrink case.Self-review
input + outputis not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.<=across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.AI use
Select exactly one:
Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.
Checklist
Does this PR entail a change in behavior?
https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J