Skip to content

.NET: Preserve cached and reasoning token counts in Foundry Hosting - #8334

Open
Manjunath Janardhan (manjunathshiva) wants to merge 1 commit into
microsoft:mainfrom
manjunathshiva:fix/6823-foundry-usage-counters
Open

.NET: Preserve cached and reasoning token counts in Foundry Hosting#8334
Manjunath Janardhan (manjunathshiva) wants to merge 1 commit into
microsoft:mainfrom
manjunathshiva:fix/6823-foundry-usage-counters

Conversation

@manjunathshiva

Copy link
Copy Markdown
Contributor

Motivation & Context

Foundry Hosting reports cached input and reasoning tokens as zero when providers populate the dedicated UsageDetails properties. For example, CachedInputTokenCount = 34304 and ReasoningTokenCount = 128 become zero in the completed response, while input/output/total counts remain correct.

Description & Review Guide

  • What are the major changes? Read CachedInputTokenCount and ReasoningTokenCount first, falling back to each existing AdditionalCounts key only when the dedicated property is absent. Add nine regression cases covering missing values, explicit zero, conflicting and mixed representations, and accumulation into completed and failed responses.
  • What is the impact of these changes? Foundry response usage preserves the provider's cached/reasoning counts. Legacy dictionary-only callers retain their behavior. No public API or response-schema changes.
  • What do you want reviewers to focus on? The per-counter fallback and precedence rules, especially explicit zero and avoiding double counting.

Verified with .NET SDK 10.0.401 on macOS arm64: seven regression cases failed before the fix; all nine pass afterward. All 473 Foundry Hosting tests pass. The full solution builds with zero warnings/errors; the enabled .NET 10 Debug unit suite passes 7,322 tests with 65 skipped and no failures, preserving the solution's existing Feature Registry Debug exclusion. Formatting checks pass. Reproduction uses the actual in-process converter; no live Azure/model calls were required.

Related Issue

Fixes #6823

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused implementation correctly preserves compatibility and is comprehensively covered by regression tests.

Pull request overview

Fixes #6823 by preserving provider-reported cached-input and reasoning-token counts in Foundry Hosting.

Changes:

  • Prefer dedicated usage counters while retaining legacy dictionary fallbacks.
  • Add regression coverage for precedence, explicit zeroes, fallback, and terminal-event accumulation.
File summaries
File Description
dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/OutputConverter.cs Corrects usage-counter conversion and fallback behavior.
dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/OutputConverterTests.cs Covers counter representations, precedence, and accumulation.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Usage: [Issues, PRs], Target: .Net

Projects

None yet

2 participants