feat(runtime): report a provider dropping context across the send boundary - #4669

Merged
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping
Sep 3, 2026
Merged

feat(runtime): report a provider dropping context across the send boundary#4669
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping

Conversation

@Joob1n

@Joob1nJoob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.

A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.

The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.

Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.

The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.

The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.

Refs #4559, #4623

Not in this PR

"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.

Verification

mid-turn-capacity-backend 76/76, overflow-reactive-recovery 50/50, context-budget 7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.

Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to <= fails the shrink case.

  • "reports provider context dropping across the send boundary" — a persisted anchor at 3,716 input tokens and a request the provider also counts at 3,716; asserts the note's data, not just its kind.
  • "does not report dropping across the boundary when the input grew" — the ordinary case.
  • "does not report dropping across the boundary when the input merely shrank" — the manual-compaction, smaller-tool-set and edited-history cases in one shape.
  • "does not report dropping across the boundary when this send folded first" — the fold explains the smaller input, so the comparison is not made.

Self-review

  • The cross-turn comparison is input against input, never against the baseline: input + output is not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.
  • The first version of this PR compared with <= across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.
  • Its tests also passed on the pre-change code, because a two-step send let the in-send comparison write the same note. Single-request sends and two mutation checks fix that; it is the reason the second round exists.
  • Backend-scoped rather than per-send is a behaviour change to the in-send note as well. That is deliberate: the two now share one latch because they report one condition.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.

One change I would make before merging, and it is one line.

Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.

The third is the one in your self-review, editing or branching history between turns.

The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

The three new tests pass on the old code

All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.

Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

Two smaller ones

The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.

contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

Shape

This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.

Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.

Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.

AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.

简体中文

追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。

合并前我想改一处,一行的事。

跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。

我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumberpruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226<= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。

第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。

第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。

#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。

三个新测试在旧代码上都会通过

三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 40003200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。

改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。

两条小的

这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokenspriorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。

contextProviderDroppingReportedAiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。

形状

这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。

把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

Ready for review. CI is green.

@me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too.

@likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself.

@Astro-Han

Copy link
Copy Markdown
Contributor

Severities for the review above, which I should have included with it.

P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual /compact then the next message (the reverse anchor scan skips the synthetic no-anchor record and returns the pre-compaction anchor); a smaller tool set in the next turn (toolSchemaShrank reads lastStepActiveToolCount, send-local and always undefined at step 0, so that exclusion is dead across the boundary); and the edit-or-branch case in your self-review. Consequence is bounded but asymmetric: a persisted note the user cannot dismiss, asserting something untrue, and the session latch then silences the real one. ai-sdk-backend.ts:2226. One line: compare for equality across the boundary, keep <= inside a send.

P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step takes the fixture default usage(120, 10) and the old in-send comparison writes the same note from 120 <= 3716; the assertion only checks kind. The two negative cases hold on the old code as well. Single-request sends leave only the step-0 comparison able to write it.

P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. inputTokens and priorInput in data.

P3 — once per backend activation, not once per session.contextProviderDroppingReported is an instance field and the backend is disposed in a dozen places in session-manager.ts. Behavior is fine; the CHANGELOG and the comment say session. The Session-scoped running total comment also now sits between cumulativeUsageCheckpoint and its target.

No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets compactionAppliedThisSend, and no textual overlap), and on scope or ownership.

No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small.

简体中文

上面那份 review 的分级,应该跟着一起给的。

P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 /compact 之后发下一条(倒序扫描跳过不带 anchor 的合成记录,拿回压缩前的 anchor);下一轮工具集变小(toolSchemaShrank 读的 lastStepActiveToolCount 是 send 内局部变量,step 0 恒 undefined,这条排除项跨边界是死的);以及你 self-review 里写的编辑或分支那种。后果有界但不对称:一条持久化、用户无法撤销、且事实错误的提示,随后 latch 让真正的那一次永远沉默。ai-sdk-backend.ts:2226。一行:跨边界判等,send 内维持 <=

P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 firstStepUsage: { input: 3_716 } 却没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧的 send 内比较用 120 <= 3716 就写出了同一条 note,而断言只查 kind。两条否定断言在旧代码上同样成立。改成单请求 send,就只剩 step 0 的比较能写出它。

P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 inputTokenspriorInput 放进 data

P3 —— 实际是每次 backend 激活一次,不是每会话一次。contextProviderDroppingReported 是实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉。行为没问题,但 CHANGELOG 和注释写的是 session。那句 Session-scoped running total 现在也夹在 cumulativeUsageCheckpoint 和它要描述的目标之间。

无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 compactionAppliedThisSend,也没有文本重叠)、以及范围与归属。

没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the local ones inline with their grades. Reasoning is in my review above.

Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens <= lastStepInputTokens
stepUsage.inputTokens <= priorInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.

The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

const toolSchemaShrank =
lastStepActiveToolCount !== undefined &&
activeToolsForRequest.length < lastStepActiveToolCount;
// Across the send boundary the comparison is the same one,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.

The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.

/**
* The provider has been reported dropping context in this session.
*
* Session-scoped, not per send: the condition persists once a provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.

Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

await runFixtureTurn(fixture, consumer);

const note = fixture.messages.find(
(message): message is { type: 'system_note'; kind: string } =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.

This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.

Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

if (persisted) {
state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0);
state.lastAcceptedTotalTokens = state.baselineTokens;
state.priorAcceptedInputTokens = persisted.inputTokens;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.

Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.

@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 94b4032 to 82e454eCompareSeptember 3, 2026 16:35
…ndary
The provider-dropping note compared each step's input against the previous
step's, so it only saw a provider evicting context from inside one send. The
shape it exists for is not visible there: a provider that truncates to a fixed
window reports the same input on every later request while the user keeps
adding turns, and a send of one or two steps has no earlier step to compare
with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight
turns with nothing reported, which is the case the note was written for.
The first request of a send now compares against the last request a provider
accepted before it, read from the persisted anchor, which is route-validated
where it is read. A fold before that request would explain a smaller input by
itself, so it disables the comparison, as prunes, image omissions and a
shrinking tool set already do.
The note is now reported once per session rather than once per send. The
condition persists once a provider starts truncating, so a note on every later
turn would repeat one fact the user has already been told.
Refs apache#4559, apache#4623
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 82e454e to cb7a4ddCompareSeptember 3, 2026 16:36
@Joob1n

Copy link
Copy Markdown
ContributorAuthor

All four are addressed on cb7a4dd0d, and both P2s were right.

The false positive. Your one-line fix is the one I took: equality across the boundary, <= inside a send. The reasoning that convinced me is that inside a send Maka knows it only appended, and across the boundary it does not — and none of the three legitimate shrinks lands on exactly the same count, while truncation to a fixed window lands on it every time. You were also right that toolSchemaShrank is dead across the boundary: lastStepActiveToolCount is send-local and undefined at step 0. Equality covers that case too, so I left the exclusion where it is rather than making it reach across sends for no additional cover.

The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to <= fails the shrink case. The positive one asserts the note's data rather than its kind.

The numbers.inputTokens and priorInputTokens are in data now, rendered in both locales and in the CLI transcript.

The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and session-manager.ts disposes the backend in a dozen places. I had also displaced the Session-scoped running total comment from cumulativeUsageCheckpoint; it is back on its field.

Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.

The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.

The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.

Approving. Merging when the checks land.

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what cb7a4dd0d changed and you are the one who can tell whether the fix matches what you meant.

The two P2s in particular:

  • Equality across the boundary is your one-line fix, taken as given. It also subsumes the dead toolSchemaShrank exclusion you found, so I left that exclusion send-local rather than making it reach across for no additional cover. Worth a look at whether you agree with leaving it.
  • The regression protection is rebuilt on single-request sends, so only the step-0 comparison can write the note, plus two mutation checks: removing the cross-boundary comparison fails the positive case, relaxing equality to <= fails the shrink case. This was a process failure on my side — I ran the gates and skipped the mutation check I normally run — so it is the part I would most like a second pair of eyes on.

P3s: the note carries inputTokens and priorInputTokens now, rendered in both locales and the CLI; the scope wording is "once per backend" everywhere; and the Session-scoped running total comment is back on cumulativeUsageCheckpoint.

@likun666661 — your approval predates all of this, so it should not carry: the comparison changed from <= to equality across the boundary, and the note now carries data. The send-module invariants you set on #4653 are unchanged.

test is still running on this head; I will confirm when it lands.

@Astro-Han
Astro-Han merged commit 3cb79b2 into apache:mainSep 3, 2026
1 check passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Joob1n@Astro-Han@likun666661
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat(runtime): report a provider dropping context across the send boundary - #4669

Merged
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping
Sep 3, 2026
Merged

feat(runtime): report a provider dropping context across the send boundary#4669
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping

Conversation

@Joob1n

@Joob1nJoob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.

A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.

The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.

Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.

The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.

The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.

Refs #4559, #4623

Not in this PR

"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.

Verification

mid-turn-capacity-backend 76/76, overflow-reactive-recovery 50/50, context-budget 7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.

Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to <= fails the shrink case.

  • "reports provider context dropping across the send boundary" — a persisted anchor at 3,716 input tokens and a request the provider also counts at 3,716; asserts the note's data, not just its kind.
  • "does not report dropping across the boundary when the input grew" — the ordinary case.
  • "does not report dropping across the boundary when the input merely shrank" — the manual-compaction, smaller-tool-set and edited-history cases in one shape.
  • "does not report dropping across the boundary when this send folded first" — the fold explains the smaller input, so the comparison is not made.

Self-review

  • The cross-turn comparison is input against input, never against the baseline: input + output is not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.
  • The first version of this PR compared with <= across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.
  • Its tests also passed on the pre-change code, because a two-step send let the in-send comparison write the same note. Single-request sends and two mutation checks fix that; it is the reason the second round exists.
  • Backend-scoped rather than per-send is a behaviour change to the in-send note as well. That is deliberate: the two now share one latch because they report one condition.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.

One change I would make before merging, and it is one line.

Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.

The third is the one in your self-review, editing or branching history between turns.

The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

The three new tests pass on the old code

All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.

Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

Two smaller ones

The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.

contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

Shape

This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.

Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.

Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.

AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.

简体中文

追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。

合并前我想改一处,一行的事。

跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。

我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumberpruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226<= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。

第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。

第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。

#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。

三个新测试在旧代码上都会通过

三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 40003200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。

改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。

两条小的

这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokenspriorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。

contextProviderDroppingReportedAiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。

形状

这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。

把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

Ready for review. CI is green.

@me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too.

@likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself.

@Astro-Han

Copy link
Copy Markdown
Contributor

Severities for the review above, which I should have included with it.

P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual /compact then the next message (the reverse anchor scan skips the synthetic no-anchor record and returns the pre-compaction anchor); a smaller tool set in the next turn (toolSchemaShrank reads lastStepActiveToolCount, send-local and always undefined at step 0, so that exclusion is dead across the boundary); and the edit-or-branch case in your self-review. Consequence is bounded but asymmetric: a persisted note the user cannot dismiss, asserting something untrue, and the session latch then silences the real one. ai-sdk-backend.ts:2226. One line: compare for equality across the boundary, keep <= inside a send.

P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step takes the fixture default usage(120, 10) and the old in-send comparison writes the same note from 120 <= 3716; the assertion only checks kind. The two negative cases hold on the old code as well. Single-request sends leave only the step-0 comparison able to write it.

P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. inputTokens and priorInput in data.

P3 — once per backend activation, not once per session.contextProviderDroppingReported is an instance field and the backend is disposed in a dozen places in session-manager.ts. Behavior is fine; the CHANGELOG and the comment say session. The Session-scoped running total comment also now sits between cumulativeUsageCheckpoint and its target.

No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets compactionAppliedThisSend, and no textual overlap), and on scope or ownership.

No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small.

简体中文

上面那份 review 的分级,应该跟着一起给的。

P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 /compact 之后发下一条(倒序扫描跳过不带 anchor 的合成记录,拿回压缩前的 anchor);下一轮工具集变小(toolSchemaShrank 读的 lastStepActiveToolCount 是 send 内局部变量,step 0 恒 undefined,这条排除项跨边界是死的);以及你 self-review 里写的编辑或分支那种。后果有界但不对称:一条持久化、用户无法撤销、且事实错误的提示,随后 latch 让真正的那一次永远沉默。ai-sdk-backend.ts:2226。一行:跨边界判等,send 内维持 <=

P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 firstStepUsage: { input: 3_716 } 却没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧的 send 内比较用 120 <= 3716 就写出了同一条 note,而断言只查 kind。两条否定断言在旧代码上同样成立。改成单请求 send,就只剩 step 0 的比较能写出它。

P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 inputTokenspriorInput 放进 data

P3 —— 实际是每次 backend 激活一次,不是每会话一次。contextProviderDroppingReported 是实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉。行为没问题,但 CHANGELOG 和注释写的是 session。那句 Session-scoped running total 现在也夹在 cumulativeUsageCheckpoint 和它要描述的目标之间。

无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 compactionAppliedThisSend,也没有文本重叠)、以及范围与归属。

没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the local ones inline with their grades. Reasoning is in my review above.

Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens <= lastStepInputTokens
stepUsage.inputTokens <= priorInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.

The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

const toolSchemaShrank =
lastStepActiveToolCount !== undefined &&
activeToolsForRequest.length < lastStepActiveToolCount;
// Across the send boundary the comparison is the same one,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.

The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.

/**
* The provider has been reported dropping context in this session.
*
* Session-scoped, not per send: the condition persists once a provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.

Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

await runFixtureTurn(fixture, consumer);

const note = fixture.messages.find(
(message): message is { type: 'system_note'; kind: string } =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.

This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.

Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

if (persisted) {
state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0);
state.lastAcceptedTotalTokens = state.baselineTokens;
state.priorAcceptedInputTokens = persisted.inputTokens;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.

Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.

@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 94b4032 to 82e454eCompareSeptember 3, 2026 16:35
…ndary
The provider-dropping note compared each step's input against the previous
step's, so it only saw a provider evicting context from inside one send. The
shape it exists for is not visible there: a provider that truncates to a fixed
window reports the same input on every later request while the user keeps
adding turns, and a send of one or two steps has no earlier step to compare
with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight
turns with nothing reported, which is the case the note was written for.
The first request of a send now compares against the last request a provider
accepted before it, read from the persisted anchor, which is route-validated
where it is read. A fold before that request would explain a smaller input by
itself, so it disables the comparison, as prunes, image omissions and a
shrinking tool set already do.
The note is now reported once per session rather than once per send. The
condition persists once a provider starts truncating, so a note on every later
turn would repeat one fact the user has already been told.
Refs apache#4559, apache#4623
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 82e454e to cb7a4ddCompareSeptember 3, 2026 16:36
@Joob1n

Copy link
Copy Markdown
ContributorAuthor

All four are addressed on cb7a4dd0d, and both P2s were right.

The false positive. Your one-line fix is the one I took: equality across the boundary, <= inside a send. The reasoning that convinced me is that inside a send Maka knows it only appended, and across the boundary it does not — and none of the three legitimate shrinks lands on exactly the same count, while truncation to a fixed window lands on it every time. You were also right that toolSchemaShrank is dead across the boundary: lastStepActiveToolCount is send-local and undefined at step 0. Equality covers that case too, so I left the exclusion where it is rather than making it reach across sends for no additional cover.

The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to <= fails the shrink case. The positive one asserts the note's data rather than its kind.

The numbers.inputTokens and priorInputTokens are in data now, rendered in both locales and in the CLI transcript.

The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and session-manager.ts disposes the backend in a dozen places. I had also displaced the Session-scoped running total comment from cumulativeUsageCheckpoint; it is back on its field.

Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.

The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.

The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.

Approving. Merging when the checks land.

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what cb7a4dd0d changed and you are the one who can tell whether the fix matches what you meant.

The two P2s in particular:

  • Equality across the boundary is your one-line fix, taken as given. It also subsumes the dead toolSchemaShrank exclusion you found, so I left that exclusion send-local rather than making it reach across for no additional cover. Worth a look at whether you agree with leaving it.
  • The regression protection is rebuilt on single-request sends, so only the step-0 comparison can write the note, plus two mutation checks: removing the cross-boundary comparison fails the positive case, relaxing equality to <= fails the shrink case. This was a process failure on my side — I ran the gates and skipped the mutation check I normally run — so it is the part I would most like a second pair of eyes on.

P3s: the note carries inputTokens and priorInputTokens now, rendered in both locales and the CLI; the scope wording is "once per backend" everywhere; and the Session-scoped running total comment is back on cumulativeUsageCheckpoint.

@likun666661 — your approval predates all of this, so it should not carry: the comparison changed from <= to equality across the boundary, and the note now carries data. The send-module invariants you set on #4653 are unchanged.

test is still running on this head; I will confirm when it lands.

@Astro-Han
Astro-Han merged commit 3cb79b2 into apache:mainSep 3, 2026
1 check passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Joob1n@Astro-Han@likun666661
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): report a provider dropping context across the send boundary - #4669

Merged
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping
Sep 3, 2026
Merged

feat(runtime): report a provider dropping context across the send boundary#4669
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping

Conversation

@Joob1n

@Joob1nJoob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.

A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.

The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.

Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.

The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.

The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.

Refs #4559, #4623

Not in this PR

"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.

Verification

mid-turn-capacity-backend 76/76, overflow-reactive-recovery 50/50, context-budget 7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.

Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to <= fails the shrink case.

  • "reports provider context dropping across the send boundary" — a persisted anchor at 3,716 input tokens and a request the provider also counts at 3,716; asserts the note's data, not just its kind.
  • "does not report dropping across the boundary when the input grew" — the ordinary case.
  • "does not report dropping across the boundary when the input merely shrank" — the manual-compaction, smaller-tool-set and edited-history cases in one shape.
  • "does not report dropping across the boundary when this send folded first" — the fold explains the smaller input, so the comparison is not made.

Self-review

  • The cross-turn comparison is input against input, never against the baseline: input + output is not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.
  • The first version of this PR compared with <= across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.
  • Its tests also passed on the pre-change code, because a two-step send let the in-send comparison write the same note. Single-request sends and two mutation checks fix that; it is the reason the second round exists.
  • Backend-scoped rather than per-send is a behaviour change to the in-send note as well. That is deliberate: the two now share one latch because they report one condition.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.

One change I would make before merging, and it is one line.

Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.

The third is the one in your self-review, editing or branching history between turns.

The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

The three new tests pass on the old code

All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.

Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

Two smaller ones

The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.

contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

Shape

This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.

Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.

Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.

AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.

简体中文

追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。

合并前我想改一处,一行的事。

跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。

我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumberpruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226<= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。

第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。

第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。

#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。

三个新测试在旧代码上都会通过

三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 40003200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。

改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。

两条小的

这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokenspriorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。

contextProviderDroppingReportedAiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。

形状

这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。

把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

Ready for review. CI is green.

@me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too.

@likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself.

@Astro-Han

Copy link
Copy Markdown
Contributor

Severities for the review above, which I should have included with it.

P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual /compact then the next message (the reverse anchor scan skips the synthetic no-anchor record and returns the pre-compaction anchor); a smaller tool set in the next turn (toolSchemaShrank reads lastStepActiveToolCount, send-local and always undefined at step 0, so that exclusion is dead across the boundary); and the edit-or-branch case in your self-review. Consequence is bounded but asymmetric: a persisted note the user cannot dismiss, asserting something untrue, and the session latch then silences the real one. ai-sdk-backend.ts:2226. One line: compare for equality across the boundary, keep <= inside a send.

P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step takes the fixture default usage(120, 10) and the old in-send comparison writes the same note from 120 <= 3716; the assertion only checks kind. The two negative cases hold on the old code as well. Single-request sends leave only the step-0 comparison able to write it.

P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. inputTokens and priorInput in data.

P3 — once per backend activation, not once per session.contextProviderDroppingReported is an instance field and the backend is disposed in a dozen places in session-manager.ts. Behavior is fine; the CHANGELOG and the comment say session. The Session-scoped running total comment also now sits between cumulativeUsageCheckpoint and its target.

No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets compactionAppliedThisSend, and no textual overlap), and on scope or ownership.

No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small.

简体中文

上面那份 review 的分级,应该跟着一起给的。

P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 /compact 之后发下一条(倒序扫描跳过不带 anchor 的合成记录,拿回压缩前的 anchor);下一轮工具集变小(toolSchemaShrank 读的 lastStepActiveToolCount 是 send 内局部变量,step 0 恒 undefined,这条排除项跨边界是死的);以及你 self-review 里写的编辑或分支那种。后果有界但不对称:一条持久化、用户无法撤销、且事实错误的提示,随后 latch 让真正的那一次永远沉默。ai-sdk-backend.ts:2226。一行:跨边界判等,send 内维持 <=

P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 firstStepUsage: { input: 3_716 } 却没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧的 send 内比较用 120 <= 3716 就写出了同一条 note,而断言只查 kind。两条否定断言在旧代码上同样成立。改成单请求 send,就只剩 step 0 的比较能写出它。

P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 inputTokenspriorInput 放进 data

P3 —— 实际是每次 backend 激活一次,不是每会话一次。contextProviderDroppingReported 是实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉。行为没问题,但 CHANGELOG 和注释写的是 session。那句 Session-scoped running total 现在也夹在 cumulativeUsageCheckpoint 和它要描述的目标之间。

无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 compactionAppliedThisSend,也没有文本重叠)、以及范围与归属。

没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the local ones inline with their grades. Reasoning is in my review above.

Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens <= lastStepInputTokens
stepUsage.inputTokens <= priorInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.

The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

const toolSchemaShrank =
lastStepActiveToolCount !== undefined &&
activeToolsForRequest.length < lastStepActiveToolCount;
// Across the send boundary the comparison is the same one,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.

The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.

/**
* The provider has been reported dropping context in this session.
*
* Session-scoped, not per send: the condition persists once a provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.

Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

await runFixtureTurn(fixture, consumer);

const note = fixture.messages.find(
(message): message is { type: 'system_note'; kind: string } =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.

This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.

Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

if (persisted) {
state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0);
state.lastAcceptedTotalTokens = state.baselineTokens;
state.priorAcceptedInputTokens = persisted.inputTokens;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.

Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.

@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 94b4032 to 82e454eCompareSeptember 3, 2026 16:35
…ndary
The provider-dropping note compared each step's input against the previous
step's, so it only saw a provider evicting context from inside one send. The
shape it exists for is not visible there: a provider that truncates to a fixed
window reports the same input on every later request while the user keeps
adding turns, and a send of one or two steps has no earlier step to compare
with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight
turns with nothing reported, which is the case the note was written for.
The first request of a send now compares against the last request a provider
accepted before it, read from the persisted anchor, which is route-validated
where it is read. A fold before that request would explain a smaller input by
itself, so it disables the comparison, as prunes, image omissions and a
shrinking tool set already do.
The note is now reported once per session rather than once per send. The
condition persists once a provider starts truncating, so a note on every later
turn would repeat one fact the user has already been told.
Refs apache#4559, apache#4623
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 82e454e to cb7a4ddCompareSeptember 3, 2026 16:36
@Joob1n

Copy link
Copy Markdown
ContributorAuthor

All four are addressed on cb7a4dd0d, and both P2s were right.

The false positive. Your one-line fix is the one I took: equality across the boundary, <= inside a send. The reasoning that convinced me is that inside a send Maka knows it only appended, and across the boundary it does not — and none of the three legitimate shrinks lands on exactly the same count, while truncation to a fixed window lands on it every time. You were also right that toolSchemaShrank is dead across the boundary: lastStepActiveToolCount is send-local and undefined at step 0. Equality covers that case too, so I left the exclusion where it is rather than making it reach across sends for no additional cover.

The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to <= fails the shrink case. The positive one asserts the note's data rather than its kind.

The numbers.inputTokens and priorInputTokens are in data now, rendered in both locales and in the CLI transcript.

The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and session-manager.ts disposes the backend in a dozen places. I had also displaced the Session-scoped running total comment from cumulativeUsageCheckpoint; it is back on its field.

Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.

The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.

The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.

Approving. Merging when the checks land.

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what cb7a4dd0d changed and you are the one who can tell whether the fix matches what you meant.

The two P2s in particular:

  • Equality across the boundary is your one-line fix, taken as given. It also subsumes the dead toolSchemaShrank exclusion you found, so I left that exclusion send-local rather than making it reach across for no additional cover. Worth a look at whether you agree with leaving it.
  • The regression protection is rebuilt on single-request sends, so only the step-0 comparison can write the note, plus two mutation checks: removing the cross-boundary comparison fails the positive case, relaxing equality to <= fails the shrink case. This was a process failure on my side — I ran the gates and skipped the mutation check I normally run — so it is the part I would most like a second pair of eyes on.

P3s: the note carries inputTokens and priorInputTokens now, rendered in both locales and the CLI; the scope wording is "once per backend" everywhere; and the Session-scoped running total comment is back on cumulativeUsageCheckpoint.

@likun666661 — your approval predates all of this, so it should not carry: the comparison changed from <= to equality across the boundary, and the note now carries data. The send-module invariants you set on #4653 are unchanged.

test is still running on this head; I will confirm when it lands.

@Astro-Han
Astro-Han merged commit 3cb79b2 into apache:mainSep 3, 2026
1 check passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Joob1n@Astro-Han@likun666661
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): report a provider dropping context across the send boundary - #4669

Merged
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping
Sep 3, 2026
Merged

feat(runtime): report a provider dropping context across the send boundary#4669
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping

Conversation

@Joob1n

@Joob1nJoob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.

A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.

The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.

Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.

The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.

The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.

Refs #4559, #4623

Not in this PR

"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.

Verification

mid-turn-capacity-backend 76/76, overflow-reactive-recovery 50/50, context-budget 7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.

Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to <= fails the shrink case.

  • "reports provider context dropping across the send boundary" — a persisted anchor at 3,716 input tokens and a request the provider also counts at 3,716; asserts the note's data, not just its kind.
  • "does not report dropping across the boundary when the input grew" — the ordinary case.
  • "does not report dropping across the boundary when the input merely shrank" — the manual-compaction, smaller-tool-set and edited-history cases in one shape.
  • "does not report dropping across the boundary when this send folded first" — the fold explains the smaller input, so the comparison is not made.

Self-review

  • The cross-turn comparison is input against input, never against the baseline: input + output is not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.
  • The first version of this PR compared with <= across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.
  • Its tests also passed on the pre-change code, because a two-step send let the in-send comparison write the same note. Single-request sends and two mutation checks fix that; it is the reason the second round exists.
  • Backend-scoped rather than per-send is a behaviour change to the in-send note as well. That is deliberate: the two now share one latch because they report one condition.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.

One change I would make before merging, and it is one line.

Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.

The third is the one in your self-review, editing or branching history between turns.

The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

The three new tests pass on the old code

All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.

Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

Two smaller ones

The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.

contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

Shape

This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.

Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.

Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.

AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.

简体中文

追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。

合并前我想改一处,一行的事。

跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。

我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumberpruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226<= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。

第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。

第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。

#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。

三个新测试在旧代码上都会通过

三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 40003200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。

改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。

两条小的

这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokenspriorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。

contextProviderDroppingReportedAiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。

形状

这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。

把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

Ready for review. CI is green.

@me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too.

@likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself.

@Astro-Han

Copy link
Copy Markdown
Contributor

Severities for the review above, which I should have included with it.

P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual /compact then the next message (the reverse anchor scan skips the synthetic no-anchor record and returns the pre-compaction anchor); a smaller tool set in the next turn (toolSchemaShrank reads lastStepActiveToolCount, send-local and always undefined at step 0, so that exclusion is dead across the boundary); and the edit-or-branch case in your self-review. Consequence is bounded but asymmetric: a persisted note the user cannot dismiss, asserting something untrue, and the session latch then silences the real one. ai-sdk-backend.ts:2226. One line: compare for equality across the boundary, keep <= inside a send.

P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step takes the fixture default usage(120, 10) and the old in-send comparison writes the same note from 120 <= 3716; the assertion only checks kind. The two negative cases hold on the old code as well. Single-request sends leave only the step-0 comparison able to write it.

P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. inputTokens and priorInput in data.

P3 — once per backend activation, not once per session.contextProviderDroppingReported is an instance field and the backend is disposed in a dozen places in session-manager.ts. Behavior is fine; the CHANGELOG and the comment say session. The Session-scoped running total comment also now sits between cumulativeUsageCheckpoint and its target.

No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets compactionAppliedThisSend, and no textual overlap), and on scope or ownership.

No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small.

简体中文

上面那份 review 的分级,应该跟着一起给的。

P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 /compact 之后发下一条(倒序扫描跳过不带 anchor 的合成记录,拿回压缩前的 anchor);下一轮工具集变小(toolSchemaShrank 读的 lastStepActiveToolCount 是 send 内局部变量,step 0 恒 undefined,这条排除项跨边界是死的);以及你 self-review 里写的编辑或分支那种。后果有界但不对称:一条持久化、用户无法撤销、且事实错误的提示,随后 latch 让真正的那一次永远沉默。ai-sdk-backend.ts:2226。一行:跨边界判等,send 内维持 <=

P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 firstStepUsage: { input: 3_716 } 却没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧的 send 内比较用 120 <= 3716 就写出了同一条 note,而断言只查 kind。两条否定断言在旧代码上同样成立。改成单请求 send,就只剩 step 0 的比较能写出它。

P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 inputTokenspriorInput 放进 data

P3 —— 实际是每次 backend 激活一次,不是每会话一次。contextProviderDroppingReported 是实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉。行为没问题,但 CHANGELOG 和注释写的是 session。那句 Session-scoped running total 现在也夹在 cumulativeUsageCheckpoint 和它要描述的目标之间。

无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 compactionAppliedThisSend,也没有文本重叠)、以及范围与归属。

没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the local ones inline with their grades. Reasoning is in my review above.

Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens <= lastStepInputTokens
stepUsage.inputTokens <= priorInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.

The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

const toolSchemaShrank =
lastStepActiveToolCount !== undefined &&
activeToolsForRequest.length < lastStepActiveToolCount;
// Across the send boundary the comparison is the same one,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.

The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.

/**
* The provider has been reported dropping context in this session.
*
* Session-scoped, not per send: the condition persists once a provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.

Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

await runFixtureTurn(fixture, consumer);

const note = fixture.messages.find(
(message): message is { type: 'system_note'; kind: string } =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.

This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.

Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

if (persisted) {
state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0);
state.lastAcceptedTotalTokens = state.baselineTokens;
state.priorAcceptedInputTokens = persisted.inputTokens;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.

Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.

@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 94b4032 to 82e454eCompareSeptember 3, 2026 16:35
…ndary
The provider-dropping note compared each step's input against the previous
step's, so it only saw a provider evicting context from inside one send. The
shape it exists for is not visible there: a provider that truncates to a fixed
window reports the same input on every later request while the user keeps
adding turns, and a send of one or two steps has no earlier step to compare
with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight
turns with nothing reported, which is the case the note was written for.
The first request of a send now compares against the last request a provider
accepted before it, read from the persisted anchor, which is route-validated
where it is read. A fold before that request would explain a smaller input by
itself, so it disables the comparison, as prunes, image omissions and a
shrinking tool set already do.
The note is now reported once per session rather than once per send. The
condition persists once a provider starts truncating, so a note on every later
turn would repeat one fact the user has already been told.
Refs apache#4559, apache#4623
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 82e454e to cb7a4ddCompareSeptember 3, 2026 16:36
@Joob1n

Copy link
Copy Markdown
ContributorAuthor

All four are addressed on cb7a4dd0d, and both P2s were right.

The false positive. Your one-line fix is the one I took: equality across the boundary, <= inside a send. The reasoning that convinced me is that inside a send Maka knows it only appended, and across the boundary it does not — and none of the three legitimate shrinks lands on exactly the same count, while truncation to a fixed window lands on it every time. You were also right that toolSchemaShrank is dead across the boundary: lastStepActiveToolCount is send-local and undefined at step 0. Equality covers that case too, so I left the exclusion where it is rather than making it reach across sends for no additional cover.

The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to <= fails the shrink case. The positive one asserts the note's data rather than its kind.

The numbers.inputTokens and priorInputTokens are in data now, rendered in both locales and in the CLI transcript.

The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and session-manager.ts disposes the backend in a dozen places. I had also displaced the Session-scoped running total comment from cumulativeUsageCheckpoint; it is back on its field.

Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.

The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.

The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.

Approving. Merging when the checks land.

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what cb7a4dd0d changed and you are the one who can tell whether the fix matches what you meant.

The two P2s in particular:

  • Equality across the boundary is your one-line fix, taken as given. It also subsumes the dead toolSchemaShrank exclusion you found, so I left that exclusion send-local rather than making it reach across for no additional cover. Worth a look at whether you agree with leaving it.
  • The regression protection is rebuilt on single-request sends, so only the step-0 comparison can write the note, plus two mutation checks: removing the cross-boundary comparison fails the positive case, relaxing equality to <= fails the shrink case. This was a process failure on my side — I ran the gates and skipped the mutation check I normally run — so it is the part I would most like a second pair of eyes on.

P3s: the note carries inputTokens and priorInputTokens now, rendered in both locales and the CLI; the scope wording is "once per backend" everywhere; and the Session-scoped running total comment is back on cumulativeUsageCheckpoint.

@likun666661 — your approval predates all of this, so it should not carry: the comparison changed from <= to equality across the boundary, and the note now carries data. The send-module invariants you set on #4653 are unchanged.

test is still running on this head; I will confirm when it lands.

@Astro-Han
Astro-Han merged commit 3cb79b2 into apache:mainSep 3, 2026
1 check passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Joob1n@Astro-Han@likun666661
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat(runtime): report a provider dropping context across the send boundary - #4669

Merged
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping
Sep 3, 2026
Merged

feat(runtime): report a provider dropping context across the send boundary#4669
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping

Conversation

@Joob1n

@Joob1nJoob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.

A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.

The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.

Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.

The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.

The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.

Refs #4559, #4623

Not in this PR

"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.

Verification

mid-turn-capacity-backend 76/76, overflow-reactive-recovery 50/50, context-budget 7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.

Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to <= fails the shrink case.

  • "reports provider context dropping across the send boundary" — a persisted anchor at 3,716 input tokens and a request the provider also counts at 3,716; asserts the note's data, not just its kind.
  • "does not report dropping across the boundary when the input grew" — the ordinary case.
  • "does not report dropping across the boundary when the input merely shrank" — the manual-compaction, smaller-tool-set and edited-history cases in one shape.
  • "does not report dropping across the boundary when this send folded first" — the fold explains the smaller input, so the comparison is not made.

Self-review

  • The cross-turn comparison is input against input, never against the baseline: input + output is not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.
  • The first version of this PR compared with <= across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.
  • Its tests also passed on the pre-change code, because a two-step send let the in-send comparison write the same note. Single-request sends and two mutation checks fix that; it is the reason the second round exists.
  • Backend-scoped rather than per-send is a behaviour change to the in-send note as well. That is deliberate: the two now share one latch because they report one condition.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.

One change I would make before merging, and it is one line.

Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.

The third is the one in your self-review, editing or branching history between turns.

The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

The three new tests pass on the old code

All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.

Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

Two smaller ones

The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.

contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

Shape

This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.

Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.

Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.

AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.

简体中文

追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。

合并前我想改一处,一行的事。

跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。

我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumberpruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226<= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。

第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。

第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。

#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。

三个新测试在旧代码上都会通过

三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 40003200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。

改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。

两条小的

这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokenspriorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。

contextProviderDroppingReportedAiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。

形状

这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。

把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

Ready for review. CI is green.

@me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too.

@likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself.

@Astro-Han

Copy link
Copy Markdown
Contributor

Severities for the review above, which I should have included with it.

P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual /compact then the next message (the reverse anchor scan skips the synthetic no-anchor record and returns the pre-compaction anchor); a smaller tool set in the next turn (toolSchemaShrank reads lastStepActiveToolCount, send-local and always undefined at step 0, so that exclusion is dead across the boundary); and the edit-or-branch case in your self-review. Consequence is bounded but asymmetric: a persisted note the user cannot dismiss, asserting something untrue, and the session latch then silences the real one. ai-sdk-backend.ts:2226. One line: compare for equality across the boundary, keep <= inside a send.

P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step takes the fixture default usage(120, 10) and the old in-send comparison writes the same note from 120 <= 3716; the assertion only checks kind. The two negative cases hold on the old code as well. Single-request sends leave only the step-0 comparison able to write it.

P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. inputTokens and priorInput in data.

P3 — once per backend activation, not once per session.contextProviderDroppingReported is an instance field and the backend is disposed in a dozen places in session-manager.ts. Behavior is fine; the CHANGELOG and the comment say session. The Session-scoped running total comment also now sits between cumulativeUsageCheckpoint and its target.

No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets compactionAppliedThisSend, and no textual overlap), and on scope or ownership.

No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small.

简体中文

上面那份 review 的分级,应该跟着一起给的。

P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 /compact 之后发下一条(倒序扫描跳过不带 anchor 的合成记录,拿回压缩前的 anchor);下一轮工具集变小(toolSchemaShrank 读的 lastStepActiveToolCount 是 send 内局部变量,step 0 恒 undefined,这条排除项跨边界是死的);以及你 self-review 里写的编辑或分支那种。后果有界但不对称:一条持久化、用户无法撤销、且事实错误的提示,随后 latch 让真正的那一次永远沉默。ai-sdk-backend.ts:2226。一行:跨边界判等,send 内维持 <=

P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 firstStepUsage: { input: 3_716 } 却没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧的 send 内比较用 120 <= 3716 就写出了同一条 note,而断言只查 kind。两条否定断言在旧代码上同样成立。改成单请求 send,就只剩 step 0 的比较能写出它。

P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 inputTokenspriorInput 放进 data

P3 —— 实际是每次 backend 激活一次,不是每会话一次。contextProviderDroppingReported 是实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉。行为没问题,但 CHANGELOG 和注释写的是 session。那句 Session-scoped running total 现在也夹在 cumulativeUsageCheckpoint 和它要描述的目标之间。

无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 compactionAppliedThisSend,也没有文本重叠)、以及范围与归属。

没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the local ones inline with their grades. Reasoning is in my review above.

Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens <= lastStepInputTokens
stepUsage.inputTokens <= priorInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.

The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

const toolSchemaShrank =
lastStepActiveToolCount !== undefined &&
activeToolsForRequest.length < lastStepActiveToolCount;
// Across the send boundary the comparison is the same one,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.

The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.

/**
* The provider has been reported dropping context in this session.
*
* Session-scoped, not per send: the condition persists once a provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.

Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

await runFixtureTurn(fixture, consumer);

const note = fixture.messages.find(
(message): message is { type: 'system_note'; kind: string } =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.

This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.

Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

if (persisted) {
state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0);
state.lastAcceptedTotalTokens = state.baselineTokens;
state.priorAcceptedInputTokens = persisted.inputTokens;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.

Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.

@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 94b4032 to 82e454eCompareSeptember 3, 2026 16:35
…ndary
The provider-dropping note compared each step's input against the previous
step's, so it only saw a provider evicting context from inside one send. The
shape it exists for is not visible there: a provider that truncates to a fixed
window reports the same input on every later request while the user keeps
adding turns, and a send of one or two steps has no earlier step to compare
with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight
turns with nothing reported, which is the case the note was written for.
The first request of a send now compares against the last request a provider
accepted before it, read from the persisted anchor, which is route-validated
where it is read. A fold before that request would explain a smaller input by
itself, so it disables the comparison, as prunes, image omissions and a
shrinking tool set already do.
The note is now reported once per session rather than once per send. The
condition persists once a provider starts truncating, so a note on every later
turn would repeat one fact the user has already been told.
Refs apache#4559, apache#4623
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 82e454e to cb7a4ddCompareSeptember 3, 2026 16:36
@Joob1n

Copy link
Copy Markdown
ContributorAuthor

All four are addressed on cb7a4dd0d, and both P2s were right.

The false positive. Your one-line fix is the one I took: equality across the boundary, <= inside a send. The reasoning that convinced me is that inside a send Maka knows it only appended, and across the boundary it does not — and none of the three legitimate shrinks lands on exactly the same count, while truncation to a fixed window lands on it every time. You were also right that toolSchemaShrank is dead across the boundary: lastStepActiveToolCount is send-local and undefined at step 0. Equality covers that case too, so I left the exclusion where it is rather than making it reach across sends for no additional cover.

The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to <= fails the shrink case. The positive one asserts the note's data rather than its kind.

The numbers.inputTokens and priorInputTokens are in data now, rendered in both locales and in the CLI transcript.

The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and session-manager.ts disposes the backend in a dozen places. I had also displaced the Session-scoped running total comment from cumulativeUsageCheckpoint; it is back on its field.

Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.

The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.

The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.

Approving. Merging when the checks land.

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what cb7a4dd0d changed and you are the one who can tell whether the fix matches what you meant.

The two P2s in particular:

  • Equality across the boundary is your one-line fix, taken as given. It also subsumes the dead toolSchemaShrank exclusion you found, so I left that exclusion send-local rather than making it reach across for no additional cover. Worth a look at whether you agree with leaving it.
  • The regression protection is rebuilt on single-request sends, so only the step-0 comparison can write the note, plus two mutation checks: removing the cross-boundary comparison fails the positive case, relaxing equality to <= fails the shrink case. This was a process failure on my side — I ran the gates and skipped the mutation check I normally run — so it is the part I would most like a second pair of eyes on.

P3s: the note carries inputTokens and priorInputTokens now, rendered in both locales and the CLI; the scope wording is "once per backend" everywhere; and the Session-scoped running total comment is back on cumulativeUsageCheckpoint.

@likun666661 — your approval predates all of this, so it should not carry: the comparison changed from <= to equality across the boundary, and the note now carries data. The send-module invariants you set on #4653 are unchanged.

test is still running on this head; I will confirm when it lands.

@Astro-Han
Astro-Han merged commit 3cb79b2 into apache:mainSep 3, 2026
1 check passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Joob1n@Astro-Han@likun666661
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): report a provider dropping context across the send boundary - #4669

Merged
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping
Sep 3, 2026
Merged

feat(runtime): report a provider dropping context across the send boundary#4669
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping

Conversation

@Joob1n

@Joob1nJoob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.

A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.

The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.

Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.

The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.

The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.

Refs #4559, #4623

Not in this PR

"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.

Verification

mid-turn-capacity-backend 76/76, overflow-reactive-recovery 50/50, context-budget 7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.

Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to <= fails the shrink case.

  • "reports provider context dropping across the send boundary" — a persisted anchor at 3,716 input tokens and a request the provider also counts at 3,716; asserts the note's data, not just its kind.
  • "does not report dropping across the boundary when the input grew" — the ordinary case.
  • "does not report dropping across the boundary when the input merely shrank" — the manual-compaction, smaller-tool-set and edited-history cases in one shape.
  • "does not report dropping across the boundary when this send folded first" — the fold explains the smaller input, so the comparison is not made.

Self-review

  • The cross-turn comparison is input against input, never against the baseline: input + output is not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.
  • The first version of this PR compared with <= across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.
  • Its tests also passed on the pre-change code, because a two-step send let the in-send comparison write the same note. Single-request sends and two mutation checks fix that; it is the reason the second round exists.
  • Backend-scoped rather than per-send is a behaviour change to the in-send note as well. That is deliberate: the two now share one latch because they report one condition.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.

One change I would make before merging, and it is one line.

Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.

The third is the one in your self-review, editing or branching history between turns.

The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

The three new tests pass on the old code

All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.

Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

Two smaller ones

The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.

contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

Shape

This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.

Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.

Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.

AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.

简体中文

追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。

合并前我想改一处,一行的事。

跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。

我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumberpruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226<= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。

第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。

第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。

#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。

三个新测试在旧代码上都会通过

三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 40003200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。

改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。

两条小的

这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokenspriorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。

contextProviderDroppingReportedAiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。

形状

这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。

把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

Ready for review. CI is green.

@me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too.

@likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself.

@Astro-Han

Copy link
Copy Markdown
Contributor

Severities for the review above, which I should have included with it.

P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual /compact then the next message (the reverse anchor scan skips the synthetic no-anchor record and returns the pre-compaction anchor); a smaller tool set in the next turn (toolSchemaShrank reads lastStepActiveToolCount, send-local and always undefined at step 0, so that exclusion is dead across the boundary); and the edit-or-branch case in your self-review. Consequence is bounded but asymmetric: a persisted note the user cannot dismiss, asserting something untrue, and the session latch then silences the real one. ai-sdk-backend.ts:2226. One line: compare for equality across the boundary, keep <= inside a send.

P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step takes the fixture default usage(120, 10) and the old in-send comparison writes the same note from 120 <= 3716; the assertion only checks kind. The two negative cases hold on the old code as well. Single-request sends leave only the step-0 comparison able to write it.

P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. inputTokens and priorInput in data.

P3 — once per backend activation, not once per session.contextProviderDroppingReported is an instance field and the backend is disposed in a dozen places in session-manager.ts. Behavior is fine; the CHANGELOG and the comment say session. The Session-scoped running total comment also now sits between cumulativeUsageCheckpoint and its target.

No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets compactionAppliedThisSend, and no textual overlap), and on scope or ownership.

No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small.

简体中文

上面那份 review 的分级,应该跟着一起给的。

P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 /compact 之后发下一条(倒序扫描跳过不带 anchor 的合成记录,拿回压缩前的 anchor);下一轮工具集变小(toolSchemaShrank 读的 lastStepActiveToolCount 是 send 内局部变量,step 0 恒 undefined,这条排除项跨边界是死的);以及你 self-review 里写的编辑或分支那种。后果有界但不对称:一条持久化、用户无法撤销、且事实错误的提示,随后 latch 让真正的那一次永远沉默。ai-sdk-backend.ts:2226。一行:跨边界判等,send 内维持 <=

P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 firstStepUsage: { input: 3_716 } 却没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧的 send 内比较用 120 <= 3716 就写出了同一条 note,而断言只查 kind。两条否定断言在旧代码上同样成立。改成单请求 send,就只剩 step 0 的比较能写出它。

P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 inputTokenspriorInput 放进 data

P3 —— 实际是每次 backend 激活一次,不是每会话一次。contextProviderDroppingReported 是实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉。行为没问题,但 CHANGELOG 和注释写的是 session。那句 Session-scoped running total 现在也夹在 cumulativeUsageCheckpoint 和它要描述的目标之间。

无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 compactionAppliedThisSend,也没有文本重叠)、以及范围与归属。

没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the local ones inline with their grades. Reasoning is in my review above.

Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens <= lastStepInputTokens
stepUsage.inputTokens <= priorInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.

The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

const toolSchemaShrank =
lastStepActiveToolCount !== undefined &&
activeToolsForRequest.length < lastStepActiveToolCount;
// Across the send boundary the comparison is the same one,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.

The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.

/**
* The provider has been reported dropping context in this session.
*
* Session-scoped, not per send: the condition persists once a provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.

Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

await runFixtureTurn(fixture, consumer);

const note = fixture.messages.find(
(message): message is { type: 'system_note'; kind: string } =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.

This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.

Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

if (persisted) {
state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0);
state.lastAcceptedTotalTokens = state.baselineTokens;
state.priorAcceptedInputTokens = persisted.inputTokens;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.

Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.

@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 94b4032 to 82e454eCompareSeptember 3, 2026 16:35
…ndary
The provider-dropping note compared each step's input against the previous
step's, so it only saw a provider evicting context from inside one send. The
shape it exists for is not visible there: a provider that truncates to a fixed
window reports the same input on every later request while the user keeps
adding turns, and a send of one or two steps has no earlier step to compare
with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight
turns with nothing reported, which is the case the note was written for.
The first request of a send now compares against the last request a provider
accepted before it, read from the persisted anchor, which is route-validated
where it is read. A fold before that request would explain a smaller input by
itself, so it disables the comparison, as prunes, image omissions and a
shrinking tool set already do.
The note is now reported once per session rather than once per send. The
condition persists once a provider starts truncating, so a note on every later
turn would repeat one fact the user has already been told.
Refs apache#4559, apache#4623
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 82e454e to cb7a4ddCompareSeptember 3, 2026 16:36
@Joob1n

Copy link
Copy Markdown
ContributorAuthor

All four are addressed on cb7a4dd0d, and both P2s were right.

The false positive. Your one-line fix is the one I took: equality across the boundary, <= inside a send. The reasoning that convinced me is that inside a send Maka knows it only appended, and across the boundary it does not — and none of the three legitimate shrinks lands on exactly the same count, while truncation to a fixed window lands on it every time. You were also right that toolSchemaShrank is dead across the boundary: lastStepActiveToolCount is send-local and undefined at step 0. Equality covers that case too, so I left the exclusion where it is rather than making it reach across sends for no additional cover.

The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to <= fails the shrink case. The positive one asserts the note's data rather than its kind.

The numbers.inputTokens and priorInputTokens are in data now, rendered in both locales and in the CLI transcript.

The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and session-manager.ts disposes the backend in a dozen places. I had also displaced the Session-scoped running total comment from cumulativeUsageCheckpoint; it is back on its field.

Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.

The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.

The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.

Approving. Merging when the checks land.

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what cb7a4dd0d changed and you are the one who can tell whether the fix matches what you meant.

The two P2s in particular:

  • Equality across the boundary is your one-line fix, taken as given. It also subsumes the dead toolSchemaShrank exclusion you found, so I left that exclusion send-local rather than making it reach across for no additional cover. Worth a look at whether you agree with leaving it.
  • The regression protection is rebuilt on single-request sends, so only the step-0 comparison can write the note, plus two mutation checks: removing the cross-boundary comparison fails the positive case, relaxing equality to <= fails the shrink case. This was a process failure on my side — I ran the gates and skipped the mutation check I normally run — so it is the part I would most like a second pair of eyes on.

P3s: the note carries inputTokens and priorInputTokens now, rendered in both locales and the CLI; the scope wording is "once per backend" everywhere; and the Session-scoped running total comment is back on cumulativeUsageCheckpoint.

@likun666661 — your approval predates all of this, so it should not carry: the comparison changed from <= to equality across the boundary, and the note now carries data. The send-module invariants you set on #4653 are unchanged.

test is still running on this head; I will confirm when it lands.

@Astro-Han
Astro-Han merged commit 3cb79b2 into apache:mainSep 3, 2026
1 check passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Joob1n@Astro-Han@likun666661
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): report a provider dropping context across the send boundary - #4669

Merged
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping
Sep 3, 2026
Merged

feat(runtime): report a provider dropping context across the send boundary#4669
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping

Conversation

@Joob1n

@Joob1nJoob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.

A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.

The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.

Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.

The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.

The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.

Refs #4559, #4623

Not in this PR

"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.

Verification

mid-turn-capacity-backend 76/76, overflow-reactive-recovery 50/50, context-budget 7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.

Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to <= fails the shrink case.

  • "reports provider context dropping across the send boundary" — a persisted anchor at 3,716 input tokens and a request the provider also counts at 3,716; asserts the note's data, not just its kind.
  • "does not report dropping across the boundary when the input grew" — the ordinary case.
  • "does not report dropping across the boundary when the input merely shrank" — the manual-compaction, smaller-tool-set and edited-history cases in one shape.
  • "does not report dropping across the boundary when this send folded first" — the fold explains the smaller input, so the comparison is not made.

Self-review

  • The cross-turn comparison is input against input, never against the baseline: input + output is not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.
  • The first version of this PR compared with <= across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.
  • Its tests also passed on the pre-change code, because a two-step send let the in-send comparison write the same note. Single-request sends and two mutation checks fix that; it is the reason the second round exists.
  • Backend-scoped rather than per-send is a behaviour change to the in-send note as well. That is deliberate: the two now share one latch because they report one condition.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.

One change I would make before merging, and it is one line.

Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.

The third is the one in your self-review, editing or branching history between turns.

The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

The three new tests pass on the old code

All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.

Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

Two smaller ones

The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.

contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

Shape

This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.

Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.

Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.

AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.

简体中文

追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。

合并前我想改一处,一行的事。

跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。

我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumberpruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226<= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。

第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。

第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。

#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。

三个新测试在旧代码上都会通过

三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 40003200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。

改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。

两条小的

这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokenspriorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。

contextProviderDroppingReportedAiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。

形状

这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。

把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

Ready for review. CI is green.

@me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too.

@likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself.

@Astro-Han

Copy link
Copy Markdown
Contributor

Severities for the review above, which I should have included with it.

P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual /compact then the next message (the reverse anchor scan skips the synthetic no-anchor record and returns the pre-compaction anchor); a smaller tool set in the next turn (toolSchemaShrank reads lastStepActiveToolCount, send-local and always undefined at step 0, so that exclusion is dead across the boundary); and the edit-or-branch case in your self-review. Consequence is bounded but asymmetric: a persisted note the user cannot dismiss, asserting something untrue, and the session latch then silences the real one. ai-sdk-backend.ts:2226. One line: compare for equality across the boundary, keep <= inside a send.

P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step takes the fixture default usage(120, 10) and the old in-send comparison writes the same note from 120 <= 3716; the assertion only checks kind. The two negative cases hold on the old code as well. Single-request sends leave only the step-0 comparison able to write it.

P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. inputTokens and priorInput in data.

P3 — once per backend activation, not once per session.contextProviderDroppingReported is an instance field and the backend is disposed in a dozen places in session-manager.ts. Behavior is fine; the CHANGELOG and the comment say session. The Session-scoped running total comment also now sits between cumulativeUsageCheckpoint and its target.

No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets compactionAppliedThisSend, and no textual overlap), and on scope or ownership.

No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small.

简体中文

上面那份 review 的分级,应该跟着一起给的。

P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 /compact 之后发下一条(倒序扫描跳过不带 anchor 的合成记录,拿回压缩前的 anchor);下一轮工具集变小(toolSchemaShrank 读的 lastStepActiveToolCount 是 send 内局部变量,step 0 恒 undefined,这条排除项跨边界是死的);以及你 self-review 里写的编辑或分支那种。后果有界但不对称:一条持久化、用户无法撤销、且事实错误的提示,随后 latch 让真正的那一次永远沉默。ai-sdk-backend.ts:2226。一行:跨边界判等,send 内维持 <=

P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 firstStepUsage: { input: 3_716 } 却没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧的 send 内比较用 120 <= 3716 就写出了同一条 note,而断言只查 kind。两条否定断言在旧代码上同样成立。改成单请求 send,就只剩 step 0 的比较能写出它。

P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 inputTokenspriorInput 放进 data

P3 —— 实际是每次 backend 激活一次,不是每会话一次。contextProviderDroppingReported 是实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉。行为没问题,但 CHANGELOG 和注释写的是 session。那句 Session-scoped running total 现在也夹在 cumulativeUsageCheckpoint 和它要描述的目标之间。

无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 compactionAppliedThisSend,也没有文本重叠)、以及范围与归属。

没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the local ones inline with their grades. Reasoning is in my review above.

Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens <= lastStepInputTokens
stepUsage.inputTokens <= priorInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.

The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

const toolSchemaShrank =
lastStepActiveToolCount !== undefined &&
activeToolsForRequest.length < lastStepActiveToolCount;
// Across the send boundary the comparison is the same one,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.

The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.

/**
* The provider has been reported dropping context in this session.
*
* Session-scoped, not per send: the condition persists once a provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.

Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

await runFixtureTurn(fixture, consumer);

const note = fixture.messages.find(
(message): message is { type: 'system_note'; kind: string } =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.

This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.

Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

if (persisted) {
state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0);
state.lastAcceptedTotalTokens = state.baselineTokens;
state.priorAcceptedInputTokens = persisted.inputTokens;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.

Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.

@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 94b4032 to 82e454eCompareSeptember 3, 2026 16:35
…ndary
The provider-dropping note compared each step's input against the previous
step's, so it only saw a provider evicting context from inside one send. The
shape it exists for is not visible there: a provider that truncates to a fixed
window reports the same input on every later request while the user keeps
adding turns, and a send of one or two steps has no earlier step to compare
with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight
turns with nothing reported, which is the case the note was written for.
The first request of a send now compares against the last request a provider
accepted before it, read from the persisted anchor, which is route-validated
where it is read. A fold before that request would explain a smaller input by
itself, so it disables the comparison, as prunes, image omissions and a
shrinking tool set already do.
The note is now reported once per session rather than once per send. The
condition persists once a provider starts truncating, so a note on every later
turn would repeat one fact the user has already been told.
Refs apache#4559, apache#4623
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 82e454e to cb7a4ddCompareSeptember 3, 2026 16:36
@Joob1n

Copy link
Copy Markdown
ContributorAuthor

All four are addressed on cb7a4dd0d, and both P2s were right.

The false positive. Your one-line fix is the one I took: equality across the boundary, <= inside a send. The reasoning that convinced me is that inside a send Maka knows it only appended, and across the boundary it does not — and none of the three legitimate shrinks lands on exactly the same count, while truncation to a fixed window lands on it every time. You were also right that toolSchemaShrank is dead across the boundary: lastStepActiveToolCount is send-local and undefined at step 0. Equality covers that case too, so I left the exclusion where it is rather than making it reach across sends for no additional cover.

The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to <= fails the shrink case. The positive one asserts the note's data rather than its kind.

The numbers.inputTokens and priorInputTokens are in data now, rendered in both locales and in the CLI transcript.

The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and session-manager.ts disposes the backend in a dozen places. I had also displaced the Session-scoped running total comment from cumulativeUsageCheckpoint; it is back on its field.

Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.

The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.

The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.

Approving. Merging when the checks land.

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what cb7a4dd0d changed and you are the one who can tell whether the fix matches what you meant.

The two P2s in particular:

  • Equality across the boundary is your one-line fix, taken as given. It also subsumes the dead toolSchemaShrank exclusion you found, so I left that exclusion send-local rather than making it reach across for no additional cover. Worth a look at whether you agree with leaving it.
  • The regression protection is rebuilt on single-request sends, so only the step-0 comparison can write the note, plus two mutation checks: removing the cross-boundary comparison fails the positive case, relaxing equality to <= fails the shrink case. This was a process failure on my side — I ran the gates and skipped the mutation check I normally run — so it is the part I would most like a second pair of eyes on.

P3s: the note carries inputTokens and priorInputTokens now, rendered in both locales and the CLI; the scope wording is "once per backend" everywhere; and the Session-scoped running total comment is back on cumulativeUsageCheckpoint.

@likun666661 — your approval predates all of this, so it should not carry: the comparison changed from <= to equality across the boundary, and the note now carries data. The send-module invariants you set on #4653 are unchanged.

test is still running on this head; I will confirm when it lands.

@Astro-Han
Astro-Han merged commit 3cb79b2 into apache:mainSep 3, 2026
1 check passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Joob1n@Astro-Han@likun666661
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat(runtime): report a provider dropping context across the send boundary - #4669

Merged
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping
Sep 3, 2026
Merged

feat(runtime): report a provider dropping context across the send boundary#4669
Astro-Han merged 1 commit into
apache:mainfrom
Joob1n:feat/context-cross-turn-dropping

Conversation

@Joob1n

@Joob1nJoob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The provider-dropping note compared each step's input against the previous step's, so it only saw a provider evicting context from inside one send. The shape it exists for is not visible there.

A provider that truncates to a fixed window reports the same input on every later request while the user keeps adding turns. A send of one or two steps has no earlier step to compare against, so nothing fires. The live evidence on #4623 is a local Ollama session with no declared window: input climbs 745 → 1,483 → 2,221 → 2,960 → 3,700 and then plateaus at 3,716 for the remaining turns while the user keeps writing. The model is losing its oldest messages every turn and the transcript says nothing.

The first request of a send now compares against the last request a provider accepted before it, read from the persisted anchor. That anchor is route-validated where it is read, so this does not pair one model's counts with another's.

Across the boundary the test is equality, not "did not grow." Inside a send Maka knows it only appended, so any shortfall is the provider's. Across the boundary it does not know that: a manual compaction leaves the pre-compaction anchor behind, a turn can carry a smaller tool set, and a user can edit or branch history. All three shrink the input legitimately, and none lands on exactly the same count. A provider truncating to a fixed window does, on every later request. A fold before the request disables the comparison outright, since it explains a smaller input by itself.

The note carries the two counts it compared, like its four siblings, so a claim the user cannot dismiss is one they can check.

The note is now reported once per backend rather than once per send. The condition persists once a provider starts truncating, so a note on every later turn would repeat one fact the user has already been told; the scope is this backend's lifetime rather than the Session's, so a backend that is disposed and rebuilt may say it once more.

Refs #4559, #4623

Not in this PR

"Compact and retry" for a rejection the classifier does not recognise, the last item on #4623.

Verification

mid-turn-capacity-backend 76/76, overflow-reactive-recovery 50/50, context-budget 7/7; typecheck, lint and format clean; the epoch guard confirms no protocol change.

Four cases, each on a single-request send so that only the step-0 comparison can write the note. Two mutations verify they carry their obligation: removing the cross-boundary comparison fails the positive case, and relaxing equality back to <= fails the shrink case.

  • "reports provider context dropping across the send boundary" — a persisted anchor at 3,716 input tokens and a request the provider also counts at 3,716; asserts the note's data, not just its kind.
  • "does not report dropping across the boundary when the input grew" — the ordinary case.
  • "does not report dropping across the boundary when the input merely shrank" — the manual-compaction, smaller-tool-set and edited-history cases in one shape.
  • "does not report dropping across the boundary when this send folded first" — the fold explains the smaller input, so the comparison is not made.

Self-review

  • The cross-turn comparison is input against input, never against the baseline: input + output is not the floor of the next input on wires that do not resend reasoning, so a baseline comparison would report every such turn as provider dropping.
  • The first version of this PR compared with <= across the boundary and had three false positives on ordinary paths, found by @Astro-Han. Equality removes all three without weakening the signal, because truncation to a fixed window is exactly what produces an identical count.
  • Its tests also passed on the pre-change code, because a two-step send let the in-send comparison write the same note. Single-request sends and two mutation checks fix that; it is the reason the second round exists.
  • Backend-scoped rather than per-send is a behaviour change to the in-send note as well. That is deliberate: the two now share one latch because they report one condition.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silent case is the right one to chase, and the transcript is the right surface for it: this is a fact about the conversation being damaged, so it belongs in the conversation rather than in a log or a settings page. The #4623 evidence is a real plateau, and nothing else covers "no declared window plus a provider that truncates quietly". I also checked the protocol claim and it holds: context_provider_dropping has been in all three closed allowlists since epoch 106, this does not touch the kind or add data, and frequency is not part of the wire vocabulary, so the epoch should not move. No conflict with #4667 either, and a fold produced by its retreat still sets compactionAppliedThisSend, so the exclusion catches it.

One change I would make before merging, and it is one line.

Compare for equality across the send boundary, not <=. Keep <= inside a send. That single change removes three false positives at once, including the one you already documented.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins, and the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. The <= at ai-sdk-backend.ts:2226 hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas. toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send (ai-sdk-backend.ts:1565) and is always undefined at step 0, so that exclusion is dead across the boundary. Your own comment says the exclusion exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately. The anchor does not record a tool count, so there is no cheap way to check it, but equality sidesteps it entirely.

The third is the one in your self-review, editing or branching history between turns.

The evidence in #4623 is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

The three new tests pass on the old code

All three set finalAtSecondCall: true, so each turn has two steps and the pre-existing in-send comparison runs. The first case sets firstStepUsage: { input: 3_716 } and no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the old code writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false. So the checklist item does not hold yet and the new cross-boundary path has no regression protection.

Making them single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

Two smaller ones

The note carries no numbers, while the four sibling notes in this family do. It asserts that the provider is dropping context and tells the user to declare a window, and they have no way to check it or judge how bad it is. Putting inputTokens and priorInput in data is a few lines and makes the claim falsifiable. That matters more here than elsewhere because the failure is asymmetric: the note is persisted, the user cannot dismiss it, and a wrong one silences the real one for the rest of the session.

contextProviderDroppingReported is an instance field on AiSdkBackend, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart. So it is once per backend activation, not once per session. The behavior is fine, arguably better, but the CHANGELOG and the comment both say session, and someone will read that literally. Also the /** Session-scoped running total, deliberately accumulated across turns. */ comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

Shape

This is a small change in the right place. The note sits beside its four siblings in the step-finish handler, which is the existing seam for saying something based on reported usage, and there is no better home for it. Nothing downstream acts on it beyond appendMessage, so a wrong one costs a line of text and this session's future warning, nothing else. That is also why I do not think it should hold the release: what it fixes, eight silent turns of a model quietly forgetting, is worth more than one wrong line.

Ablating the session latch leaves the #4623 spec satisfied, just noisier, and you have a good reason to keep it, so I am not asking for it back.

Evidence boundary: read at 94b40323 against mainb9748a77; the anchor read and its route check, the fold exclusions, the three allowlists, every consumer of the note, and the fixture default behind the three new tests verified from source. No build, no test run, no runtime. The manual-compaction path is traced rather than observed, and one /compact followed by a message settles it. test was still running when I read this.

AI-assisted review: drafted with Maka. I verified the anchor scan, the fixture default and the epoch claim myself.

简体中文

追这个静默场景是对的,transcript 也是对的落点:这是一个「你此刻这条对话正在损坏」的事实,属于对话本身,不属于日志或设置页。#4623 的证据是真实的平台期,而「未声明窗口 + provider 静默截断」这个场景确实没有别的信号覆盖。协议那条 claim 我也核了,成立:context_provider_dropping 从 epoch 106 起就在三处封闭 allowlist 里,这次不改 kind、不加 data,频率本来就不参与 decode,epoch 不该动。和 #4667 也不冲突,它 retreat 产生的 fold 一样会置 compactionAppliedThisSend,排除项接得住。

合并前我想改一处,一行的事。

跨 send 边界改成判等,不用 <=,send 内维持 <=。这一处改动同时消掉三种误报,包括你自己已经写在正文里的那种。

我最在意的是手动压缩。persistedRequestAnchor 倒序扫描、第一条带 anchor 的记录说了算,而手动 /compact 写的合成 token_usage 不带 anchor,于是扫描跳过它继续往前,拿到的是压缩之前那个真实的大 anchor。下一个 send 的首请求跑在压缩后的历史上,input 小得多。compactionAppliedThisSend 是 per-send 状态,而压缩发生在上一次操作里,所以为 false;replacedStepNumberpruneAppliedAtStep 是 undefined,omittedImageToolResults 为空。ai-sdk-backend.ts:2226<= 命中,一个刚刚主动压缩完的用户被告知他的 provider 在丢上下文、应该去声明窗口。又因为 latch 是会话级的,这一次错报还会永久吃掉这个会话真正的那一次。

第二种是工具 schema。toolSchemaShrank 读的 lastStepActiveToolCount 声明在 send 内(ai-sdk-backend.ts:1565),step 0 恒为 undefined,所以这条排除项跨边界是死的。你自己的注释写着这条排除存在是因为「请求是 Maka 塑造的,provider 没丢东西」,这个理由跨轮同样成立:切换权限模式、断开一个 MCP server、subagent 换了工具集,都会合法地让 schema 变小。anchor 里没有记工具数,没有便宜的校验办法,但判等能直接绕开。

第三种就是你 self-review 里写的那种,两个 turn 之间编辑或分支历史。

#4623 的证据形态本身就是 3,716 上的精确平台,判等正是这个形态需要的。放弃的是「跨边界 input 显著下降」这类信号,而跨边界的显著下降几乎总是 Maka 自己造成的,不是 provider。

三个新测试在旧代码上都会通过

三条都设了 finalAtSecondCall: true,所以每个 turn 有两步,既有的 send 内比较会跑。第一条设 firstStepUsage: { input: 3_716 } 但没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧代码用 120 <= 3716 就已经写出同一条 note 了,而断言只查 kind,分不出是哪次比较产生的。两条否定断言在旧代码上同样成立,4200 <= 40003200 <= 3000 都为假。所以那个勾选项目前不成立,跨边界这条新路径没有回归保护。

改成单请求 send(不产生 tool call,也就没有 send 内比较),就只剩 step 0 的比较能写出 note。顺手补一条误报守卫:有 anchor、首请求小很多、没有 fold,断言不出 note。

两条小的

这条 note 不带任何数字,而同族另外四条都带。它断言 provider 在丢上下文并建议用户去声明窗口,用户既无法核对也判断不了严重程度。把 inputTokenspriorInput 放进 data 是几行的事,能让这句断言变得可证伪。这里比别处更要紧,因为失败是不对称的:note 会持久化、用户不能撤销,而一次错报会让这个会话后续真正的那一次永远沉默。

contextProviderDroppingReportedAiSdkBackend 的实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉重建(plan 的 revise、abandon、approve、resume,lineage 处理),加上进程重启。所以它实际是「每次 backend 激活一次」,不是「每会话一次」。行为本身没问题,甚至更好,但 CHANGELOG 和注释都写的是 session,会有人照字面理解。另外上面那句 /** Session-scoped running total, deliberately accumulated across turns. */ 现在夹在 cumulativeUsageCheckpoint 和它要描述的目标之间,读起来像在说新字段。

形状

这是一个落在对的地方的小改动。这条 note 和它的四个同族并排在 step-finish 处理器里,那就是「拿 provider 报的 usage 说事」的现有接缝,没有更合适的落点。除了 appendMessage 之外没有任何下游对它采取动作,所以错报的代价就是一行字加这个会话未来的那次告警,仅此而已。这也是我认为它不该卡发版的原因:它修的那个八轮静默,比它引入的误报值钱。

把会话级 latch 烧蚀掉之后 #4623 的规格仍然满足,只是更吵,而你保留它的理由站得住,所以我不要求改回去。

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

Ready for review. CI is green.

@me2seeks — this is the cross-turn half of the provider-dropping note, and the case it exists for is the one your production data described: usage that stops growing while the user keeps adding turns. The in-send comparison could never see it. You also asked, on the reported-window note, that a persistent condition be reported once rather than repeatedly; this note is now once per session for the same reason, which changes the in-send behaviour too.

@likun666661 — the send-module invariants are yours from #4653: the comparison is input against input across the boundary, never against the baseline, and a fold before the request disables it because the fold explains the smaller input by itself.

@Astro-Han

Copy link
Copy Markdown
Contributor

Severities for the review above, which I should have included with it.

P2 — a false positive across the send boundary, on three ordinary paths. Normal user path. Manual /compact then the next message (the reverse anchor scan skips the synthetic no-anchor record and returns the pre-compaction anchor); a smaller tool set in the next turn (toolSchemaShrank reads lastStepActiveToolCount, send-local and always undefined at step 0, so that exclusion is dead across the boundary); and the edit-or-branch case in your self-review. Consequence is bounded but asymmetric: a persisted note the user cannot dismiss, asserting something untrue, and the session latch then silences the real one. ai-sdk-backend.ts:2226. One line: compare for equality across the boundary, keep <= inside a send.

P2 — the new path has no regression protection. All three new cases pass on the pre-change code, so the checklist item does not hold yet. The first sets firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step takes the fixture default usage(120, 10) and the old in-send comparison writes the same note from 120 <= 3716; the assertion only checks kind. The two negative cases hold on the old code as well. Single-request sends leave only the step-0 comparison able to write it.

P3 — the note carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. inputTokens and priorInput in data.

P3 — once per backend activation, not once per session.contextProviderDroppingReported is an instance field and the backend is disposed in a dozen places in session-manager.ts. Behavior is fine; the CHANGELOG and the comment say session. The Session-scoped running total comment also now sits between cumulativeUsageCheckpoint and its target.

No finding on the protocol claim (the kind has been in all three allowlists since epoch 106, frequency is not decoded, the epoch should not move), on the #4667 interaction (its retreat still sets compactionAppliedThisSend, and no textual overlap), and on scope or ownership.

No P0 and no P1, so nothing here blocks the release. The two P2s are what I would want before merge, and both are small.

简体中文

上面那份 review 的分级,应该跟着一起给的。

P2 —— 跨 send 边界的误报,三条普通路径都能触发。 正常用户路径。手动 /compact 之后发下一条(倒序扫描跳过不带 anchor 的合成记录,拿回压缩前的 anchor);下一轮工具集变小(toolSchemaShrank 读的 lastStepActiveToolCount 是 send 内局部变量,step 0 恒 undefined,这条排除项跨边界是死的);以及你 self-review 里写的编辑或分支那种。后果有界但不对称:一条持久化、用户无法撤销、且事实错误的提示,随后 latch 让真正的那一次永远沉默。ai-sdk-backend.ts:2226。一行:跨边界判等,send 内维持 <=

P2 —— 新路径没有回归保护。 三个新用例在改动前的代码上都会通过,所以那个勾选项目前不成立。第一条设了 firstStepUsage: { input: 3_716 } 却没给 finalStepUsage,第二步落到 fixture 默认的 usage(120, 10),旧的 send 内比较用 120 <= 3716 就写出了同一条 note,而断言只查 kind。两条否定断言在旧代码上同样成立。改成单请求 send,就只剩 step 0 的比较能写出它。

P3 —— 这条 note 不带数字,而同族另外四条都带,于是一个用户撤销不了的断言,他也核对不了。把 inputTokenspriorInput 放进 data

P3 —— 实际是每次 backend 激活一次,不是每会话一次。contextProviderDroppingReported 是实例字段,而 backend 在 session-manager.ts 里有十来处会被拆掉。行为没问题,但 CHANGELOG 和注释写的是 session。那句 Session-scoped running total 现在也夹在 cumulativeUsageCheckpoint 和它要描述的目标之间。

无发现:协议那条 claim(kind 从 epoch 106 起就在三处 allowlist 里,频率不参与 decode,epoch 不该动)、和 #4667 的交互(它的 retreat 一样会置 compactionAppliedThisSend,也没有文本重叠)、以及范围与归属。

没有 P0、没有 P1,所以这里没有东西阻塞发版。两条 P2 是我希望合并前处理的,都很小。

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting the local ones inline with their grades. Reasoning is in my review above.

Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens <= lastStepInputTokens
stepUsage.inputTokens <= priorInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, and the one change I would make before merge. Compare for equality across the send boundary, keep <= inside a send. One line, and it removes three false positives at once.

The one that worries me most is manual compaction. persistedRequestAnchor scans in reverse and the first anchor-bearing record wins; the synthetic token_usage a manual /compact writes carries no anchor, so the scan skips it and keeps going back to the real anchor from before the compaction. The next send's first request runs on the compacted history, so its input is much smaller. compactionAppliedThisSend is per-send state and the compaction happened in a previous operation, so it is false; replacedStepNumber and pruneAppliedAtStep are undefined and omittedImageToolResults is empty. This <= hits, and a user who just compacted on purpose is told their provider is dropping context and should declare a window. Because the latch is session scoped, that one wrong note also permanently eats the session's real one.

The second is tool schemas, see the note on the exclusion list above. The third is the edit-or-branch case in your self-review.

The #4623 evidence is an exact plateau at 3,716, so equality is what the shape actually calls for. What you give up is "input dropped a lot across the boundary", and across a boundary a large drop is almost always something Maka did rather than something the provider did.

const toolSchemaShrank =
lastStepActiveToolCount !== undefined &&
activeToolsForRequest.length < lastStepActiveToolCount;
// Across the send boundary the comparison is the same one,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2toolSchemaShrank reads lastStepActiveToolCount, which is declared inside the send at :1565 and is therefore always undefined at step 0, so this exclusion is dead across the boundary. Your own comment says it exists because Maka shaped the request and the provider dropped nothing, and that reasoning holds across turns just as well: switching permission mode, disconnecting an MCP server, or a subagent with a different tool set all shrink the schema legitimately, by thousands of tokens.

The anchor does not record a tool count, so there is no cheap check here; widening it would change the token_usage shape and cost an epoch, which is not worth it. Equality on the cross-boundary comparison sidesteps it entirely.

/**
* The provider has been reported dropping context in this session.
*
* Session-scoped, not per send: the condition persists once a provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 This latch is an instance field, and the backend is disposed and rebuilt in a dozen places in session-manager.ts (plan revise, abandon, approve, resume, lineage handling) as well as on restart, so it is once per backend activation rather than once per session. The behaviour is fine, arguably better; the CHANGELOG and this comment both say session and someone will read that literally.

Also worth moving the field: the Session-scoped running total, deliberately accumulated across turns comment above now sits between cumulativeUsageCheckpoint and its target, so it reads as describing the new field.

await runFixtureTurn(fixture, consumer);

const note = fixture.messages.find(
(message): message is { type: 'system_note'; kind: string } =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All three new cases pass on the pre-change code, so the checklist line does not hold yet and the cross-boundary path has no regression protection.

This one sets finalAtSecondCall: true and firstStepUsage: { input: 3_716 } with no finalStepUsage, so the second step falls to the fixture default usage(120, 10) and the pre-existing in-send comparison writes the same note from 120 <= 3716. The assertion only checks kind, so it cannot tell which comparison produced it. The two negative cases hold on the old code too, since 4200 <= 4000 and 3200 <= 3000 are both false.

Making these single-request sends (no tool call, so no in-send comparison exists) leaves only the step-0 comparison able to write the note. Worth adding a false-positive guard in the same pass: an anchor, a much smaller first request, and no fold, asserting silence.

if (persisted) {
state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0);
state.lastAcceptedTotalTokens = state.baselineTokens;
state.priorAcceptedInputTokens = persisted.inputTokens;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Related to the equality change: this is where the pre-compaction anchor gets carried into the next send. If you would rather not touch the comparison itself, the alternative fix lives here, in persistedRequestAnchor's reverse scan: treat an anchorless usage record as a barrier rather than skipping it, at least for this field, so a manual /compact produces a cold start instead of a stale boundary. Same one-boolean cost. I prefer the equality change because it also covers the tool-schema and edit-or-branch cases.

Separately, P3 on the note itself: it carries no numbers while its four siblings do, so a claim the user cannot dismiss is also one they cannot check. Putting inputTokens and priorInput in data is a few lines and makes it falsifiable, which matters here because a wrong note is persisted and silences the real one for the rest of the session.

@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 94b4032 to 82e454eCompareSeptember 3, 2026 16:35
…ndary
The provider-dropping note compared each step's input against the previous
step's, so it only saw a provider evicting context from inside one send. The
shape it exists for is not visible there: a provider that truncates to a fixed
window reports the same input on every later request while the user keeps
adding turns, and a send of one or two steps has no earlier step to compare
with. The live evidence in apache#4623 plateaus at 3,716 input tokens across eight
turns with nothing reported, which is the case the note was written for.
The first request of a send now compares against the last request a provider
accepted before it, read from the persisted anchor, which is route-validated
where it is read. A fold before that request would explain a smaller input by
itself, so it disables the comparison, as prunes, image omissions and a
shrinking tool set already do.
The note is now reported once per session rather than once per send. The
condition persists once a provider starts truncating, so a note on every later
turn would repeat one fact the user has already been told.
Refs apache#4559, apache#4623
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1nforce-pushed the feat/context-cross-turn-dropping branch from 82e454e to cb7a4ddCompareSeptember 3, 2026 16:36
@Joob1n

Copy link
Copy Markdown
ContributorAuthor

All four are addressed on cb7a4dd0d, and both P2s were right.

The false positive. Your one-line fix is the one I took: equality across the boundary, <= inside a send. The reasoning that convinced me is that inside a send Maka knows it only appended, and across the boundary it does not — and none of the three legitimate shrinks lands on exactly the same count, while truncation to a fixed window lands on it every time. You were also right that toolSchemaShrank is dead across the boundary: lastStepActiveToolCount is send-local and undefined at step 0. Equality covers that case too, so I left the exclusion where it is rather than making it reach across sends for no additional cover.

The missing regression protection. This one was a process failure on my side, not a judgement call: I ran the gates and skipped the mutation check I normally run, so I did not notice the fixture default made the old in-send comparison write the same note. The four cases are now single-request sends, and two mutations confirm they hold their obligation — removing the cross-boundary comparison fails the positive case, and relaxing equality to <= fails the shrink case. The positive one asserts the note's data rather than its kind.

The numbers.inputTokens and priorInputTokens are in data now, rendered in both locales and in the CLI transcript.

The scope wording. Corrected to "once per backend" in the comment, the CHANGELOG and the PR body, since the field is an instance field and session-manager.ts disposes the backend in a dozen places. I had also displaced the Session-scoped running total comment from cumulativeUsageCheckpoint; it is back on its field.

Thank you for the severities, and for separating what you checked from what you read statically — the "no finding" list saved me from re-deriving the epoch question.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P2s are addressed and addressed at the right place. Equality across the boundary rather than a barrier in the anchor scan is the version I preferred, and your reasoning for it is the sharper one: none of the three legitimate shrinks lands on exactly the same count, and truncation to a fixed window lands on it every time. Leaving toolSchemaShrank where it is instead of making it reach across sends is right, since equality already covers that case and reaching would have bought nothing.

The tests now carry the obligation: single-request sends leave only the step-0 comparison able to write the note, and the shrink case is the guard that was missing. Asserting on data rather than kind also pins the payload in the same stroke.

The latch comment now says what it does, and moving the field above cumulativeUsageCheckpoint puts that Session-scoped running total line back with its own target.

Approving. Merging when the checks land.

@Joob1n

Copy link
Copy Markdown
ContributorAuthor

@Astro-Han — asking you for the second pass here, since all four findings in your severity list are what cb7a4dd0d changed and you are the one who can tell whether the fix matches what you meant.

The two P2s in particular:

  • Equality across the boundary is your one-line fix, taken as given. It also subsumes the dead toolSchemaShrank exclusion you found, so I left that exclusion send-local rather than making it reach across for no additional cover. Worth a look at whether you agree with leaving it.
  • The regression protection is rebuilt on single-request sends, so only the step-0 comparison can write the note, plus two mutation checks: removing the cross-boundary comparison fails the positive case, relaxing equality to <= fails the shrink case. This was a process failure on my side — I ran the gates and skipped the mutation check I normally run — so it is the part I would most like a second pair of eyes on.

P3s: the note carries inputTokens and priorInputTokens now, rendered in both locales and the CLI; the scope wording is "once per backend" everywhere; and the Session-scoped running total comment is back on cumulativeUsageCheckpoint.

@likun666661 — your approval predates all of this, so it should not carry: the comparison changed from <= to equality across the boundary, and the note now carries data. The send-module invariants you set on #4653 are unchanged.

test is still running on this head; I will confirm when it lands.

@Astro-Han
Astro-Han merged commit 3cb79b2 into apache:mainSep 3, 2026
1 check passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Joob1n@Astro-Han@likun666661