Python: Fix streamed parallel tool calls merging into the wrong call - #8337
Conversation
_process_update only ever merged an incoming function_call chunk into message.contents[-1], so once two tool calls are in flight at the same time their interleaved argument deltas got mashed together or split into duplicate, unmergeable fragments instead of each call ending up complete. Match the merge target by call_id across all in-progress calls, falling back to the trailing item for continuation deltas that never repeat their call_id. Fixes microsoft#8336
There was a problem hiding this comment.
🟡 Changes recommended
Tagged chunks can still merge into an unrelated untagged trailing call when no matching call ID is found.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Fixes aggregation of interleaved streamed parallel function calls by matching their call IDs.
Changes:
- Adds call-ID-aware function-call merging.
- Adds regression and fallback tests.
File summaries
| File | Description |
|---|---|
python/packages/core/agent_framework/_types.py |
Adds targeted function-call chunk aggregation. |
python/packages/core/tests/core/test_types.py |
Tests interleaved calls and ID-less continuations. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced (auto)
Note
Copilot is running an experiment and ran this review at Balanced.
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
| call_id = getattr(content, "call_id", None) | ||
| if call_id: | ||
| for index in range(len(message.contents) - 1, -1, -1): | ||
| existing = message.contents[index] | ||
| if existing.type == "function_call" and getattr(existing, "call_id", None) == call_id: | ||
| try: | ||
| message.contents[index] = existing + content | ||
| except (AdditionItemMismatch, ContentError): | ||
| break | ||
| return | ||
| if message.contents and message.contents[-1].type == "function_call": | ||
| try: | ||
| message.contents[-1] += content | ||
| return | ||
| except (AdditionItemMismatch, ContentError): | ||
| pass | ||
| message.contents.append(content) |
There was a problem hiding this comment.
Good catch — you're right that _add_function_call_content only rejects a merge when both sides carry a call_id, so an untagged trailing item would have silently absorbed a differently-tagged new call. Fixed in 3467e10: a tagged chunk with no matching in-progress call now always gets appended instead of falling into the trailing-item fallback, which is reserved for call-ID-less continuation chunks only. Added a regression test for this exact case.
Content.__add__ only rejects a function_call merge when both sides carry a call_id and they differ, so a chunk tagged with a brand-new call_id could still fall into the trailing-item fallback and get silently absorbed by an untagged trailing call, adopting its id and mashing two unrelated calls' arguments together. Reserve that fallback for chunks that carry no call_id at all; a tagged chunk with no matching in-progress call is always a new call and should be appended.
Motivation & Context
Parallel tool calling is a normal thing for a model to do — "get the weather in
NYC and the time in Boston" is one turn, two tool calls. When those two calls
stream, their argument fragments don't have to arrive one call fully at a time;
OpenAIResponsesClientalready tags everyresponse.function_call_arguments.deltaevent with its call's real
call_idspecifically because it can't assume thatordering. The shared response-aggregation code in
_types.pydidn't honor that:it only ever merged an incoming
function_callchunk into whatever the lastcontent item happened to be, regardless of whether that item was actually the
call the chunk belonged to. The result is silent corruption — one call's
arguments end up empty, the other's end up as an invalid-JSON mash of both
calls' fragments, or both calls get split across several duplicate, never-
reassembled content items. None of this raises an exception, so it surfaces
downstream as a confusing
JSONDecodeErrorinparse_arguments()or, worse, atool invoked with the wrong arguments.
Description & Review Guide
_process_updateinagent_framework/_types.pynow merges a streamedfunction_callchunk intothe specific in-progress call it belongs to.
_merge_function_call_contentmatches by
call_idacross all of the message's existing content (not justthe trailing item) when the chunk carries one, and only falls back to
merging with the trailing
function_callitem for chunks that don't carry acall_idat all — which some providers only send on the first chunk of acall, so that fallback preserves today's single-call-at-a-time behavior.
more concurrent tool calls now aggregate correctly regardless of how their
argument deltas interleave. This is provider-agnostic: it fixes any client
that (like
OpenAIResponsesClientalready does) tags every delta with thecall's real
call_id, with no client-specific changes required.path for call_id-less continuation chunks is the right compromise — it's
needed to keep existing single-call streaming (e.g. the legacy OpenAI Chat
Completions API, which only sends
call_idon a tool call's first delta)working exactly as before, but by construction it can't disambiguate two
concurrent untagged calls. Fixing that fully means teaching that one
client to track its own
index→call_idmapping, which felt like aseparate, client-specific change rather than something the shared
aggregation layer should own — noted as follow-up context in the issue.
Related Issue
Fixes #8336
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.