Skip to content

Python: Fix streamed parallel tool calls merging into the wrong call - #8337

Open
Sadok Barbouche (cr-sbarbouche) wants to merge 2 commits into
microsoft:mainfrom
cr-sbarbouche:fix-interleaved-parallel-tool-call-streaming
Open

Python: Fix streamed parallel tool calls merging into the wrong call#8337
Sadok Barbouche (cr-sbarbouche) wants to merge 2 commits into
microsoft:mainfrom
cr-sbarbouche:fix-interleaved-parallel-tool-call-streaming

Conversation

@cr-sbarbouche

Copy link
Copy Markdown
Contributor

Motivation & Context

Parallel tool calling is a normal thing for a model to do — "get the weather in
NYC and the time in Boston" is one turn, two tool calls. When those two calls
stream, their argument fragments don't have to arrive one call fully at a time;
OpenAIResponsesClient already tags every response.function_call_arguments.delta
event with its call's real call_id specifically because it can't assume that
ordering. The shared response-aggregation code in _types.py didn't honor that:
it only ever merged an incoming function_call chunk into whatever the last
content item happened to be, regardless of whether that item was actually the
call the chunk belonged to. The result is silent corruption — one call's
arguments end up empty, the other's end up as an invalid-JSON mash of both
calls' fragments, or both calls get split across several duplicate, never-
reassembled content items. None of this raises an exception, so it surfaces
downstream as a confusing JSONDecodeError in parse_arguments() or, worse, a
tool invoked with the wrong arguments.

Description & Review Guide

  • What are the major changes? _process_update in
    agent_framework/_types.py now merges a streamed function_call chunk into
    the specific in-progress call it belongs to. _merge_function_call_content
    matches by call_id across all of the message's existing content (not just
    the trailing item) when the chunk carries one, and only falls back to
    merging with the trailing function_call item for chunks that don't carry a
    call_id at all — which some providers only send on the first chunk of a
    call, so that fallback preserves today's single-call-at-a-time behavior.
  • What is the impact of these changes? Streaming responses with two or
    more concurrent tool calls now aggregate correctly regardless of how their
    argument deltas interleave. This is provider-agnostic: it fixes any client
    that (like OpenAIResponsesClient already does) tags every delta with the
    call's real call_id, with no client-specific changes required.
  • What do you want reviewers to focus on? Whether the fallback-to-trailing-item
    path for call_id-less continuation chunks is the right compromise — it's
    needed to keep existing single-call streaming (e.g. the legacy OpenAI Chat
    Completions API, which only sends call_id on a tool call's first delta)
    working exactly as before, but by construction it can't disambiguate two
    concurrent untagged calls. Fixing that fully means teaching that one
    client to track its own indexcall_id mapping, which felt like a
    separate, client-specific change rather than something the shared
    aggregation layer should own — noted as follow-up context in the issue.

Related Issue

Fixes #8336

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

_process_update only ever merged an incoming function_call chunk into
message.contents[-1], so once two tool calls are in flight at the same
time their interleaved argument deltas got mashed together or split
into duplicate, unmergeable fragments instead of each call ending up
complete. Match the merge target by call_id across all in-progress
calls, falling back to the trailing item for continuation deltas that
never repeat their call_id.

Fixes microsoft#8336

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Tagged chunks can still merge into an unrelated untagged trailing call when no matching call ID is found.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Fixes aggregation of interleaved streamed parallel function calls by matching their call IDs.

Changes:

  • Adds call-ID-aware function-call merging.
  • Adds regression and fallback tests.
File summaries
File Description
python/packages/core/agent_framework/_types.py Adds targeted function-call chunk aggregation.
python/packages/core/tests/core/test_types.py Tests interleaved calls and ID-less continuations.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +2083 to +2099
call_id = getattr(content, "call_id", None)
if call_id:
for index in range(len(message.contents) - 1, -1, -1):
existing = message.contents[index]
if existing.type == "function_call" and getattr(existing, "call_id", None) == call_id:
try:
message.contents[index] = existing + content
except (AdditionItemMismatch, ContentError):
break
return
if message.contents and message.contents[-1].type == "function_call":
try:
message.contents[-1] += content
return
except (AdditionItemMismatch, ContentError):
pass
message.contents.append(content)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — you're right that _add_function_call_content only rejects a merge when both sides carry a call_id, so an untagged trailing item would have silently absorbed a differently-tagged new call. Fixed in 3467e10: a tagged chunk with no matching in-progress call now always gets appended instead of falling into the trailing-item fallback, which is reserved for call-ID-less continuation chunks only. Added a regression test for this exact case.

Content.__add__ only rejects a function_call merge when both sides
carry a call_id and they differ, so a chunk tagged with a brand-new
call_id could still fall into the trailing-item fallback and get
silently absorbed by an untagged trailing call, adopting its id and
mashing two unrelated calls' arguments together. Reserve that fallback
for chunks that carry no call_id at all; a tagged chunk with no
matching in-progress call is always a new call and should be appended.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: Streamed parallel tool calls get merged into the wrong call when their argument deltas interleave

2 participants