Skip to content

fix: mark interrupted tool replies - #6859

Open
chenghao-mou wants to merge 5 commits into
mainfrom
chenghao/fix/AGT-3295-interrupted-tool-output
Open

fix: mark interrupted tool replies#6859
chenghao-mou wants to merge 5 commits into
mainfrom
chenghao/fix/AGT-3295-interrupted-tool-output

Conversation

@chenghao-mou

@chenghao-mou chenghao-mou commented Aug 14, 2026

Copy link
Copy Markdown
Member

Why

Tool outputs remain in chat history after execution. If the user interrupts their spoken reply, the next LLM turn can treat that explanation as delivered and return no reply.

Behavior

Expose incomplete delivery as {"output":"...","reply_interrupted":true}. Normal tool outputs stay unchanged, and immediate commits keep tools with side effects from being replayed.

Addresses AGT-3295

Initial prompt and agent context

Model: GPT-5.6

so today, we commit the tool output after execution, but if the user interrupts the tool reply, llm might treat the tool output as spoken and skips the reply altogether in the new reply. We should try wrapping the tool output with some kind of interrupted:bool flag so it is clear to them that it is delivered or not.

The last thing I want to do is to modify the chat context as it is fairly complex already to maintain the contract between the user and the framework.

okay, let's make the field "reply_interrupted"

@chenghao-mou
chenghao-mou force-pushed the chenghao/fix/AGT-3295-interrupted-tool-output branch from a202b70 to f644561 Compare August 14, 2026 16:43
@chenghao-mou chenghao-mou changed the title fix: mark tool outputs after interrupted replies fix: mark interrupted tool replies Aug 17, 2026
@chenghao-mou

chenghao-mou commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

Behavior replay

Model: openai/gpt-4.1-mini. The original basic-agent prompt and regular lookup_weather tool were used. Five trials per case were interleaved.

Exact conversation:

user: Can you help me check the weather in Dublin?
assistant: Let me check that for you.
assistant tool call: lookup_weather(location="Dublin", latitude="53.3498", longitude="-6.2603")
tool: The current weather in Dublin is sunny at 70 degrees. This afternoon will reach 75 degrees, with light west winds around 8 miles per hour and humidity near 45 percent. There is a 10 percent chance of rain. Temperatures will cool to 62 degrees after sunset. Tomorrow will be partly cloudy, with a high of 72 degrees and a low of 58 degrees.
user: Okay, take your time.

No assistant message appears between the tool result and final user message. The before case sends the raw tool text. The after case wraps the same text as {"output": "...", "reply_interrupted": true}.

Tool content Weather result relayed Weather result skipped Avg. covered forecast categories
Raw 2/5 3/5 1.6/7
reply_interrupted: true 5/5 0/5 5.2/7

A result counts as relayed when the reply covers at least two of these categories: current conditions, afternoon temperature, wind, humidity, rain, evening temperature, and next-day forecast.

@chenghao-mou
chenghao-mou marked this pull request as ready for review August 17, 2026 13:38
@chenghao-mou
chenghao-mou requested a review from a team as a code owner August 17, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant