Skip to content

Runtime replay can invert earlier assistant text behind later client tool calls #4749

Description

@hqhq1025

Summary

AiSdkBackend.materializeRuntimeReplayPlan() can replay an earlier assistant text step after a later client tool call and result. This changes durable ledger chronology before the next provider request.

Observed behavior

For a durable sequence such as:

assistant text (step A) -> client tool call (step B) -> tool result

the provider request can contain:

client tool call (step B) -> tool result -> assistant text (step A)

The defect is especially visible when Responses text item boundaries give progress text its own provider step identity and the following client tool call has a different step identity.

Root cause

The replay materializer parks assistant text and reasoning by stepId, but its flush path emits buffered calls before pending text/reasoning steps. That ordering was introduced with the per-step replay materializer in #1425 (a82d4fc090defa3b958ca394f6a1bd5ea044a1a1) and retained when #1501 (b76b60f3a4c14b9e078823b87ec7668cd1e1b513) added intermediate flushPendingSteps() boundaries. #4270 increased exposure by adding model-authored progress updates and preserving Responses text item boundaries; it did not introduce the underlying inversion.

Impact

The next provider request receives a history that disagrees with the durable Runtime event ledger. In tool-heavy turns, a previously completed action can appear to precede the earlier statement that introduced it, which can confuse continuation state and execution phase.

Acceptance criteria

  • Pending assistant text or reasoning from an earlier distinct step is emitted before a later client tool call.
  • Multiple pending assistant steps retain ledger order.
  • Text/reasoning and client tools remain grouped when they are contiguous and share the same step identity.
  • A matching tool call that arrives after another pending step does not jump backward across that intervening step.
  • Provider-executed tool chronology and legacy events without a step identity remain unchanged.
  • Focused regression tests fail on the old materializer and pass on the repaired implementation.

Controlled experiments

All runs used independent Maka CLI sessions with GPT-5.6 Sol. Replay order was recomputed from each session's durable runtime.sqlite and captured provider-request artifacts.

ArmStrict task completionComparable replay pairs inverted
Baseline6/10789/789
Patched replay8/100/563
Broken replay, progress prompt removed3/10138/138

A neutral five-task comparison retained the broken materializer:

ArmStrict task completionComparable replay pairs inverted
Progress enabled4/597/97
Progress removed5/571/71

Additional patched stress test:

  • 20/20 independent sessions produced and read back the requested artifact.
  • 50 client tool calls and 50 matching results were persisted.
  • 119 comparable earlier-text/later-tool pairs were replayed with 0 inversions.
  • 0/20 sessions terminated without a tool call.

These results establish the chronology defect and its repair. They do not establish that replay repair prevents every model from ending its first provider response with stop. Prompt policy and completion/fulfillment guards are intentionally out of scope for this issue.

Scope

This issue is limited to faithful Runtime event replay ordering. It does not change the main-session prompt, provider finish-reason mapping, CLI success classification, or add a semantic task-fulfillment guard.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions