Skip to content

fix: merge streaming tool calls by logical index - #3377

Closed
he-yufeng wants to merge 1 commit into
openai:mainfrom
he-yufeng:fix/stream-tool-call-index-merge
Closed

he-yufeng wants to merge 1 commit into
openai:mainfrom
he-yufeng:fix/stream-tool-call-index-merge

Conversation

@he-yufeng

Copy link
Copy Markdown
Contributor

Summary

Fix the streaming delta accumulator so indexed lists are merged by their logical index field even when the first chunk already contains duplicate entries for the same index.

This keeps OpenAI-compatible streaming tool-call chunks like this from being stranded as two physical list entries:

  • first entry: tool id/name at index: 0
  • second entry: first argument fragment at index: 0
  • later chunks: more argument fragments at index: 0

The chat-completions initial snapshot now runs the initial delta through the same accumulator path as later chunks, and the Assistants accumulator gets the same first-chunk handling.

Closes #3201.

To verify

$env:PYTHONPATH=(Resolve-Path .\src).Path; python -m pytest tests\lib\test_streaming_deltas.py tests\lib\chat\test_completions_streaming.py -q -o addopts=''
python -m py_compile src\openai\lib\streaming\_deltas.py src\openai\lib\streaming\_assistants.py src\openai\lib\streaming\chat\_completions.py tests\lib\chat\test_completions_streaming.py tests\lib\test_streaming_deltas.py
python -m ruff check src\openai\lib\streaming\_deltas.py src\openai\lib\streaming\_assistants.py src\openai\lib\streaming\chat\_completions.py tests\lib\chat\test_completions_streaming.py tests\lib\test_streaming_deltas.py
python -m ruff format --check src\openai\lib\streaming\_deltas.py src\openai\lib\streaming\_assistants.py src\openai\lib\streaming\chat\_completions.py tests\lib\chat\test_completions_streaming.py tests\lib\test_streaming_deltas.py
python -m pyright src\openai\lib\streaming\_deltas.py src\openai\lib\streaming\_assistants.py src\openai\lib\streaming\chat\_completions.py tests\lib\test_streaming_deltas.py
git diff --check

I also ran python -m pyright tests\lib\chat\test_completions_streaming.py; it still reports existing inline_snapshot unknown-type noise in that file, unrelated to this patch.

@he-yufeng
he-yufeng requested a review from a team as a code owner June 7, 2026 16:24

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e69bf216fe

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +1042 to +1043
if acc_entry.get("index") == index:
acc_value[acc_index] = accumulate_delta(acc_entry, delta_entry)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve positional fallback for unindexed assistant snapshots

For Assistants run steps, run_step_snapshots are seeded from thread.run.step.created with final RunStep tool-call objects, whose generated types do not include an index; only subsequent delta objects carry one. With this new lookup, an existing unindexed entry never matches delta_entry["index"], so the first delta for tool call 0 is appended as a duplicate instead of being merged into the existing slot. The event handler later reads step_snapshot.step_details.tool_calls[tool_call_delta.index], so code-interpreter/function deltas for created run steps can surface stale or empty snapshots rather than the accumulated arguments/input.

Useful? React with 👍 / 👎.

@he-yufeng

Copy link
Copy Markdown
Contributor Author

Still valid on current main — when a provider interleaves deltas from several parallel tool calls, the stream merger keys on arrival order rather than the tool-call index, so argument fragments land on the wrong call. This one merges by logical index instead and pins the interleaved-delta case in tests. CI is green; happy to rebase if that helps review.

@he-yufeng
he-yufeng force-pushed the fix/stream-tool-call-index-merge branch from e69bf21 to b234505 Compare August 16, 2026 23:58

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b234505c7c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

choices[choice.index] = {
**choice.model_dump(exclude_unset=True, exclude={"delta"}),
"message": choice.delta.to_dict(),
"message": accumulate_delta({}, cast("dict[object, object]", choice.delta.to_dict())),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Normalize duplicate indexes for later-arriving choices

This normalization only runs for choices present in the stream's first chunk. When streaming multiple choices, a choice may first appear in a later chunk; the except IndexError path in _accumulate_chunk still constructs its message directly from choice.delta.to_dict(). If that choice's first delta contains duplicate tool-call entries for one logical index, they remain separate, subsequent events read only the first physical entry, and the final completion retains a stale duplicate. Apply the same accumulator initialization when constructing a newly encountered choice.

Useful? React with 👍 / 👎.

@marcuswood-oai

Copy link
Copy Markdown
Contributor

Thanks for taking the time to work on this! We’ve merged the fix for duplicate-index tool-call deltas in #3425, so I’m closing this as a duplicate. We appreciate the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming tool_call deltas with duplicate indexes in first chunk are accumulated incorrectly

2 participants