Skip to content

Fix tool-call fallback for structured completion content - #2278

Open
Slayer (suger-m) wants to merge 2 commits into
microsoft:mainfrom
suger-m:fix/issue-2263-tool-call-json-fallback
Open

Slayer (suger-m) wants to merge 2 commits into
microsoft:mainfrom
suger-m:fix/issue-2263-tool-call-json-fallback

Conversation

@suger-m

Copy link
Copy Markdown

Summary

  • Fix structured response parsing when provider responses return empty assistant content and place JSON in function tool-call arguments.
  • Add fallback in LLMCompletionResponse.content to read tool_calls[].function.arguments when message content is empty.
  • Update gather_completion_response and gather_completion_response_async to use response.content.
  • Add unit regression tests for content fallback, gather helpers, and structured parsing.

Issue

Fixes #2263

Validation

  • uvx ruff check packages/graphrag-llm/graphrag_llm/types/types.py packages/graphrag-llm/graphrag_llm/utils/gather_completion_response.py tests/unit/utils/test_completion_response.py
  • python -m pytest tests/unit/utils/test_completion_response.py -q

Handles LiteLLM structured-output fallbacks that return JSON in function tool-call arguments with empty assistant content.\n\n- Add fallback in LLMCompletionResponse.content to return function tool-call arguments\n- Update gather_completion_response helpers to use response.content\n- Add regression tests for content gathering and structured parsing via tool calls\n\nFixes microsoft#2263
@suger-m
Slayer (suger-m) requested a review from a team as a code owner March 12, 2026 08:58
@suger-m

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@microsoft-github-policy-service

Copy link
Copy Markdown

Slayer (@suger-m) the command you issued was incorrect. Please try again.

Examples are:

@microsoft-github-policy-service agree

and

@microsoft-github-policy-service agree company="your company"

@suger-m

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="your company"

@suger-m

Copy link
Copy Markdown
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: model_supports_json is no longer supported on v3, breaking custom model setups

1 participant