fix(models): resolve nested provider behind litellm_proxy prefix

[vietnamesekid]

Link to Issue or Description of Change

• Closes: [Bug] LiteLlm sends invalid PDF content for Azure-backed models through litellm_proxy #6538

Problem:

_get_provider_from_model splits the model string on the first / and treats
that segment as the provider. That assumption breaks for nested LiteLLM Proxy
identifiers, because litellm_proxy names the transport, not the model family:

_get_provider_from_model("litellm_proxy/azure/my-deployment")  # -> "litellm_proxy"

litellm_proxy is not in _FILE_ID_REQUIRED_PROVIDERS ({"openai", "azure"}),
so the Azure/OpenAI file-upload path in _get_content never runs. The PDF goes
out as an inline file_data block instead of an uploaded file_id. When the
proxy translates that for the Azure Responses API, Azure rejects the content
item before inference:

BadRequestError: AzureException BadRequestError
Missing required parameter: 'input[0].content[3]'

While tracing this I found the same first-segment assumption in four more
helpers, so the blast radius is wider than the PDF case in the issue. Every
provider-specific behavior silently degrades once a model is reached through
the proxy:

Helper	litellm_proxy/... input	Before	Expected
_get_provider_from_model	litellm_proxy/azure/gpt-4	litellm_proxy	azure
_is_anthropic_model	litellm_proxy/anthropic/claude-4	False	True
_is_litellm_vertex_model	litellm_proxy/vertex_ai/gemini-2.5-flash	False	True
_is_litellm_gemini_model	litellm_proxy/vertex_ai/gemini-2.5-flash	False	True
_extract_gemini_model_from_litellm	litellm_proxy/vertex_ai/gemini-2.5-pro	vertex_ai/gemini-2.5-pro	gemini-2.5-pro

Practical effect beyond the PDF bug: proxied Anthropic models lose thinking
block formatting, and proxied Gemini models are not recognized as Vertex or
Gemini routes.

Worth noting that litellm.get_llm_provider() also returns litellm_proxy
for these strings. That is correct for LiteLLM, which only needs to know where
to send the request. ADK uses the value for something different, namely how to
shape the payload, and that has to follow the provider that actually serves
the model. So the fix belongs here rather than upstream.

Solution:

Add _strip_proxy_prefix() and call it before provider and model family
detection. A proxied model is then shaped exactly like its direct equivalent.

Three things I deliberately kept intact:

1. Model strings without the prefix take the same code path as before. The
   helper returns the input unchanged when there is nothing to strip.
2. A bare litellm_proxy/<deployment> has no nested provider, so the
   remainder falls through to the existing model name heuristics.
   litellm_proxy/azure-gpt-4 resolves to azure; an opaque
   litellm_proxy/my-deployment resolves to "".
3. The prefix match is case insensitive, so LiteLLM_Proxy/azure/gpt-4 works.

One behavior change worth calling out for review: an opaque
litellm_proxy/my-deployment now returns "" rather than "litellm_proxy".
I grepped for consumers of that literal and there are none. "" is already
the established "provider not determinable" value in this function, so
unknown deployments now take the generic path, which is the honest answer when
the backing provider cannot be known from the string alone.

Testing Plan

Unit Tests:

• I have added or updated unit tests for my change.
• All unit tests pass locally.

Extended the existing test_get_provider_from_model table with the nested
proxy forms, the case insensitive variant, and both bare deployment cases.

Added test_model_family_detection_through_litellm_proxy, which pins all four
model family helpers across proxied and direct strings so the two stay in
lockstep.

Added test_get_content_pdf_proxied_azure_uses_file_id as the regression test
for the reported symptom. It drives _get_content from the model string and
asserts the upload actually happens with custom_llm_provider="azure", which
is the part that was silently skipped.

$ pytest tests/unittests/models/test_litellm.py -q
376 passed in 2.77s

$ pytest tests/unittests/models/ -q
957 passed, 34 warnings in 12.12s

I checked that the new tests actually fail without the fix rather than passing
by construction. Reverting just the _strip_proxy_prefix call in
_get_provider_from_model and rerunning:

8 failed, 19 passed, 349 deselected
FAILED test_get_provider_from_model[litellm_proxy/azure/my-deployment-azure]
FAILED test_get_provider_from_model[litellm_proxy/openai/gpt-4o-openai]
FAILED test_get_provider_from_model[litellm_proxy/anthropic/claude-3-anthropic]
FAILED test_get_provider_from_model[litellm_proxy/vertex_ai/gemini-pro-vertex_ai]
FAILED test_get_provider_from_model[LiteLLM_Proxy/azure/gpt-4-azure]
FAILED test_get_provider_from_model[litellm_proxy/azure-gpt-4-azure]
FAILED test_get_provider_from_model[litellm_proxy/my-deployment-]
FAILED test_get_content_pdf_proxied_azure_uses_file_id

Formatted with pyink 25.12.0 and isort 8.0.1, matching the pinned versions
in .pre-commit-config.yaml.

Manual End-to-End (E2E) Tests:

Verified the conversion decision without network access, mocking the upload so
the emitted content block is visible. This is the assertion that matters,
since it is the payload Azure rejects:

import asyncio
from unittest import mock
from google.adk.models import lite_llm
from google.adk.models.lite_llm import (
    _content_to_message_param, _get_provider_from_model, _ensure_litellm_imported)
from google.genai import types

_ensure_litellm_imported()

pdf = types.Content(role="user", parts=[
    types.Part(text="summarize"),
    types.Part(inline_data=types.Blob(mime_type="application/pdf", data=b"%PDF-1.4 fake")),
])

async def show(model):
    provider = _get_provider_from_model(model)
    with mock.patch.object(lite_llm.litellm, "acreate_file",
                           new=mock.AsyncMock(return_value=mock.Mock(id="file-abc"))):
        msg = await _content_to_message_param(pdf, provider=provider, model=model)
    print(f"{model:36} provider={provider!r:12} -> {msg['content'][1]}")

for m in ["azure/gpt-4", "litellm_proxy/azure/my-deployment", "litellm_proxy/my-deployment"]:
    asyncio.run(show(m))

Before:

azure/gpt-4                          provider='azure'      -> {'type': 'file', 'file': {'file_id': 'file-abc', 'format': 'application/pdf'}}
litellm_proxy/azure/my-deployment    provider='litellm_proxy' -> {'type': 'file', 'file': {'file_data': 'data:application/pdf;base64,JVBERi0xLjQgZmFrZQ=='}}
litellm_proxy/my-deployment          provider='litellm_proxy' -> {'type': 'file', 'file': {'file_data': 'data:application/pdf;base64,JVBERi0xLjQgZmFrZQ=='}}

After:

azure/gpt-4                          provider='azure'      -> {'type': 'file', 'file': {'file_id': 'file-abc', 'format': 'application/pdf'}}
litellm_proxy/azure/my-deployment    provider='azure'      -> {'type': 'file', 'file': {'file_id': 'file-abc', 'format': 'application/pdf'}}
litellm_proxy/my-deployment          provider=''           -> {'type': 'file', 'file': {'file_data': 'data:application/pdf;base64,JVBERi0xLjQgZmFrZQ=='}}

The proxied Azure model now produces a payload byte for byte identical to the
direct Azure model. The opaque deployment still uses file_data, which is the
correct fallback when the backing provider is unknowable from the string.

I did not run this against a live Azure backed proxy, since I do not have a
deployment to test with. The reporter in #6538 is set up for that and could
confirm on a real endpoint.

Checklist

• I have read the CONTRIBUTING.md document.
• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.
• I have manually tested my changes end-to-end.
• Any dependent changes have been merged and published in downstream modules.

Additional context

Scope note: I limited this to reading the nested provider out of the model
string. Routing, credentials, and how LiteLLM itself resolves the proxy are
untouched.

If maintainers would rather keep the blast radius to the reported PDF bug, the
change to _get_provider_from_model alone fixes #6538 and the four model
family helpers can be split into a follow up. I kept them together because
they share one root cause, and fixing only the provider lookup leaves the same
bug reachable through the Anthropic and Gemini paths.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[vietnamesekid]

@llalitkumarrr @ross-p I verified this branch against the reproduction in #6538. It fixes what the issue diagnoses, but while testing I found a second layer that this PR does not cover. Details below, and I would like your call on scope before I push anything further.

Everything below is reproducible. The layer 1 output comes from the commit currently on this PR (ebc87873). The layer 2 output comes from a follow-up commit I deliberately did not push to this PR, so you can read it without it landing here: vietnamesekid/adk-python@3b96796

Layer 1: payload shape (what this PR fixes)

Running the repro from the issue against a local HTTP endpoint standing in for the upload target, so these are real requests rather than assertions about mocks.

On main:

provider: litellm_proxy
file keys: dict_keys(['file_data'])

On this PR, with AZURE_API_BASE / AZURE_API_KEY / AZURE_API_VERSION set:

provider: azure
    [SERVER HIT] POST /openai/files?api-version=2024-07-01-preview  (293 bytes)
result: {'file_id': 'file-fromserver', 'format': 'application/pdf'}

The upload happens and a real file_id comes back. file keys goes from ['file_data'] to ['file_id', 'format'], matching the reported symptom, and the payload is now identical to what a direct azure/gpt-4 model produces.

One note on the script as written in the issue: it no longer prints, it raises.

openai.OpenAIError: Missing credentials. Please pass one of `api_key`,
`azure_ad_token`, `azure_ad_token_provider`, or the `AZURE_OPENAI_API_KEY`
or `AZURE_OPENAI_AD_TOKEN` environment variables.

That is the fix working. The traceback goes through litellm.acreate_file into litellm/llms/azure/files/handler.py, which is the Azure upload path that was being skipped. It only fails because no Azure credentials are set. On main the script never reaches that call at all.

Which leads to the part I want to raise.

Layer 2: the upload does not go through the proxy

litellm.acreate_file resolves its endpoint independently of the completion call. ADK passes only custom_llm_provider, so the upload falls back to the underlying provider's environment variables. In litellm/files/main.py, the custom_llm_provider == "azure" branch calls get_azure_credentials(api_base=None, api_key=None, ...), which resolves from AZURE_API_BASE / AZURE_API_KEY.

So after this PR the request splits in two:

• completion goes to the proxy
• file upload goes straight to Azure

That is invisible if you happen to have AZURE_* set, which is why layer 1 above looks clean. It is fatal if you do not, which is the normal reason to front a provider with a proxy in the first place.

Same PR commit, same script, only difference is AZURE_* unset and proxy credentials configured instead:

provider: azure
ERROR: OpenAIError | Missing credentials. Please pass one of `api_key`, ...
server hits: []

Nothing is sent. So for a proxy-only setup this PR turns "Azure rejects the request" into "the client cannot upload at all". Better in that it fails fast and locally, but still broken.

Proposed fix for layer 2

Forward api_base, api_key, and api_version from the completion arguments to acreate_file when the model is proxied. Direct models forward nothing and keep their current environment-variable resolution, since their api_base already points at the provider.

I have this working in vietnamesekid/adk-python@3b96796 (2 files, +182/-9). Same proxy-only scenario, AZURE_* unset:

upload_params: {'api_base': 'http://127.0.0.1:8799', 'api_key': 'proxy-key'}
    [SERVER HIT] POST /openai/files?api-version=2025-02-01-preview  (293 bytes)
content: {'type': 'file', 'file': {'file_id': 'file-fromserver', 'format': 'application/pdf'}}

The api-version differs from the layer 1 output above because nothing sets it in this scenario, so LiteLLM falls back to its own AZURE_DEFAULT_API_VERSION (2025-02-01-preview). If the caller passes api_version it is forwarded like the other two. Flagging it so the difference does not look like drift if you rerun this.

For the record, I first tried passing custom_llm_provider="litellm_proxy" and letting LiteLLM route the upload itself. That does not work: the files API has no litellm_proxy branch and raises LiteLLM doesn't support litellm_proxy for 'create_file'. Only ['openai', 'azure', 'vertex_ai', 'manus', 'anthropic'] are supported. Keeping the real provider and overriding the endpoint is the only path I found.

Known limit of that fix

It only covers proxy settings passed to the constructor:

LiteLlm(model="litellm_proxy/azure/my-deployment", api_base=..., api_key=...)

If the proxy is configured through LITELLM_PROXY_API_BASE / LITELLM_PROXY_API_KEY instead, which litellm/llms/litellm_proxy/chat/transformation.py reads for the completion call, there is nothing in the completion arguments to forward and the upload still fails:

upload_params: {}
ERROR: OpenAIError | Missing credentials...
server hits: []

Covering that means reading those env vars in ADK as a fallback. I have not done it, since it is a judgment call about how much LiteLLM configuration ADK should reimplement, and I would rather ask than guess.

What I would like decided

Three options, and I am happy with any of them:

1. I push vietnamesekid/adk-python@3b96796 into this PR. Larger diff, threads an optional upload_params through _get_completion_inputs and _content_to_message_param down to _get_content, but [Bug] LiteLlm sends invalid PDF content for Azure-backed models through litellm_proxy #6538 is genuinely fixed for constructor-configured proxy setups.
2. This PR merges as is for the payload shape, and I open a separate issue plus PR for the upload routing.
3. Something else you prefer.

I lean towards 1, since a proxy-only setup is the common case and layer 1 alone does not make the reported scenario work end to end.

Also worth flagging

The same first-segment assumption affects four other helpers, already included in this PR:

Helper	Input	main	This branch
_is_anthropic_model	litellm_proxy/anthropic/claude-4	False	True
_is_litellm_vertex_model	litellm_proxy/vertex_ai/gemini-2.5-flash	False	True
_is_litellm_gemini_model	litellm_proxy/vertex_ai/gemini-2.5-flash	False	True
_extract_gemini_model_from_litellm	litellm_proxy/vertex_ai/gemini-2.5-pro	vertex_ai/gemini-2.5-pro	gemini-2.5-pro

In practice proxied Anthropic models lose thinking block formatting and proxied Gemini models are not recognized as Vertex or Gemini routes. Same root cause, so I bundled them, but they can also be split out if you would rather keep this PR narrow.

What I could not verify

I do not have an Azure backed proxy deployment, so none of this is confirmed against a live endpoint. Everything above is the request ADK builds and where it sends it, tested locally against a stub server.

@ross-p, if you can run this branch against your setup that would close the gap. One specific question that decides whether layer 2 affects you today: do you have AZURE_API_BASE and AZURE_API_KEY set alongside your proxy config, or only the proxy credentials?

Test status: on the PR commit, 376 passed in tests/unittests/models/test_litellm.py and 957 across tests/unittests/models/. With vietnamesekid/adk-python@3b96796 applied, 385 and 966. In both cases the new tests fail when the fix is reverted, so they pin real behavior rather than passing by construction.