Skip to content

docs: rebuild the unified HF deployment support matrix from the deploy test suite (NVBug 6550792) #2087

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kevalmorabia97 merged 4 commits into from
Aug 12, 2026
Merged

docs: rebuild the unified HF deployment support matrix from the deploy test suite (NVBug 6550792) #2087

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ac3e582
docs: rebuild the unified HF deployment support matrix from the deplo…
Edwardf0t1 Aug 5, 2026
d2727e8
docs: address review — TRT-LLM v1.2.0 floor, checkmarks, conditional …
Edwardf0t1 Aug 7, 2026
f5948d8
docs: drop the Phi series from the deployment matrix
Edwardf0t1 Aug 12, 2026
ee7f5b4
docs: reframe the matrix as declared test scope, not CI-validated sup…
Edwardf0t1 Aug 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
207 changes: 168 additions & 39 deletions docs/source/deployment/3_unified_hf.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,48 +51,175 @@ The unified HF export API supports the following quantization formats:
5. INT4_AWQ - 4-bit integer with AWQ optimization
6. W4A8_AWQ - 4-bit weights and 8-bit activations with AWQ optimization

Framework-Specific Support
Minimum Framework Versions
--------------------------

TensorRT-LLM
~~~~~~~~~~~~
=============== =================
Framework Minimum version
=============== =================
TensorRT-LLM v1.2.0

@cjluo-nv cjluo-nv Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot comment.

This bumps the stated TRT-LLM floor from v0.17.0 to v1.2.0 for unified-HF checkpoints generally (and the same change lands in examples/hf_ptq/README.md and examples/diffusers/README.md, where the footnote scopes it to NVFP4). Is v1.2.0 really the minimum for FP8 as well, or is this the CI/NVFP4 version being generalized? Raising a documented minimum invalidates working user setups, so it'd help to say what the constraint actually is (e.g. "NVFP4 requires v1.2.0+; FP8 works from vX.Y").

@Edwardf0t1 Edwardf0t1 Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair challenge — I generalized a version I only have evidence for at the top end, and you are right that raising a documented floor can invalidate working setups.

What I can actually support: the deploy tests use EagleDecodingConfig and CudaGraphConfig from tensorrt_llm.llmapi, which do not exist in 0.17, and CI containers are on 1.3.0rc20. That establishes what we exercise, not the true FP8 minimum, which I do not have data for.

So rather than assert a hard floor I qualified it: v1.2.0 is stated as the oldest version we list, with an explicit note that older TensorRT-LLM releases may still serve FP8 and are simply not exercised. If you know the real FP8 floor I will put the number in. The two footnote changes in examples/hf_ptq/README.md and examples/diffusers/README.md are scoped to NVFP4, where v1.2 is the safer statement.

vLLM v0.10.1
SGLang v0.4.10
=============== =================

These are the oldest versions expected to load a unified HF checkpoint. The deployment suite itself
targets newer ones — TensorRT-LLM containers in ``.github/workflows/`` are on the 1.3.x line. Older
TensorRT-LLM releases may still serve FP8 checkpoints; that is simply not exercised, so v1.2.0 is
the oldest version stated here rather than the oldest that works.

.. _unified-hf-support-matrix:

Model Support Matrix
--------------------

What this matrix is based on
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Entries are drawn from the release deployment suite,
`tests/examples/hf_ptq/test_deploy.py <https://github.com/NVIDIA/Model-Optimizer/blob/main/tests/examples/hf_ptq/test_deploy.py>`_.
For each entry it loads the exported checkpoint in the framework and generates from four short text
prompts, asserting that each returns non-empty output.

Two limits are worth stating plainly, because they bound what any ✅ below can mean:

* **These are declared cases, not PR-gated coverage.** The suite is marked ``release`` and collects
only when pytest is given ``--run-release``, which no workflow in ``.github/workflows/`` currently
passes. A green check on a pull request does not mean these cases ran.
* **Each case is a load-and-generate smoke check on the text path.** It does not verify accuracy,
image or audio inputs, diffusion output, or that speculative decoding actually engages.

Legend:

* ✅ — declared in the release deployment suite, subject to the two limits above.
* ⚠ — expected to work, but not a suite entry: either carried over from earlier documentation, or
present as a case that does not exercise the feature the row names.
* ``-`` — not in the suite. It may still work; see `Models not listed here`_.

Language models
~~~~~~~~~~~~~~~

============================================ ============== ============ ====== ========
Model Quant format TensorRT-LLM vLLM SGLang
============================================ ============== ============ ====== ========
Llama 3.1, 3.3 FP8, NVFP4 ✅ ✅ ✅
Llama 4 Scout, Maverick FP8 ✅ ✅ ✅
Llama 4 Scout NVFP4 ✅ ✅ ✅

@cjluo-nv cjluo-nv Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot comment.

Same issue as the DeepSeek FP8 rows: the old examples/hf_ptq/README.md matrix had LLAMA 4 | FP4 | ✅ for TRT-LLM, and test_deploy.py only covers Scout NVFP4. Maverick NVFP4 is dropped rather than marked ⚠, which silently narrows documented support. Consider Llama 4 Maverick | NVFP4 | ⚠ | ⚠ | ⚠ (or - / - / - with an explicit note) so the removal is visible.

@Edwardf0t1 Edwardf0t1 Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same fix — added Llama 4 Maverick | NVFP4 | ⚠ | - | -, matching the old matrix (TRT-LLM only) rather than dropping it. The Llama 4 Scout NVFP4 row stays ✅ since that is what the suite actually declares.

Llama 4 Maverick NVFP4 ⚠ \- \-
Llama Nemotron Super 49B v1, v1.5 FP8 ✅ ✅ ✅
Llama Nemotron Ultra 253B v1 FP8 ✅ ✅ ✅
Nemotron 3 Nano 30B-A3B FP8, NVFP4 ✅ ✅ ✅
Nemotron 3 Super 120B-A12B FP8, NVFP4 ✅ ✅ ✅
Nemotron 3 Ultra 550B-A55B NVFP4 ✅ ✅ ✅
DeepSeek R1, R1-0528 NVFP4 ✅ ✅ ✅

@cjluo-nv cjluo-nv Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot comment.

The old matrices listed DS-R1 FP8 and DS-V3 FP8 as ✅ on all three frameworks; the new table has NVFP4-only rows and no FP8 row at all — not even ⚠. Per the PR's own rationale ("entries carried over from the old matrix are not silently promoted"), the symmetric case should hold too: previously-documented combinations that left the suite should become ⚠, not vanish. As written this reads to a user as "DeepSeek FP8 is no longer supported".

@Edwardf0t1 Edwardf0t1 Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — the asymmetry was mine, and it contradicted the rationale I wrote in the PR description. Added DeepSeek R1, V3 | FP8 | ⚠ | ⚠ | ⚠, carrying over the old matrix's all-three coverage. Nothing that was previously documented now silently disappears.

DeepSeek R1, V3 FP8 ⚠ ⚠ ⚠
DeepSeek V3, V3.1, V3.2 NVFP4 ✅ ✅ ✅
DeepSeek V4 Flash NVFP4 ✅ ✅ ✅
DeepSeek V4 Pro NVFP4 \- ✅ ✅
Qwen 3 8B, 14B FP8, NVFP4 ✅ ✅ ✅
Qwen 3 32B NVFP4 ✅ ✅ ✅
Qwen 3 MoE 235B-A22B FP8, NVFP4 ✅ ✅ ✅
Qwen 3 MoE 30B-A3B NVFP4 ✅ ✅ ✅
Qwen 3 Coder 480B-A35B NVFP4 ✅ ✅ ✅
Qwen 3-Next 80B-A3B NVFP4 ✅ ✅ ✅
Qwen 3.5 397B-A17B NVFP4 ✅ ✅ ✅
Qwen 3.5 122B-A10B, Qwen 3.6 35B-A3B NVFP4 \- ✅ \-
Qwen 2.5 FP8 ⚠ ⚠ ⚠
Qwen 2.5 NVFP4 ⚠ ⚠ \-
QwQ-32B FP8 ⚠ ⚠ ⚠
QwQ-32B NVFP4 ⚠ ⚠ \-
Gemma 4 31B NVFP4 ✅ ✅ ✅
Gemma 4 26B-A4B NVFP4 \- ✅ \-
GLM-4.7, GLM-5, GLM-5.2 NVFP4 ✅ ✅ ✅
GLM-5.1 NVFP4 \- ✅ ✅
Kimi K2-Thinking, K2.5 NVFP4 ✅ ✅ ✅
Kimi K2.6 NVFP4 \- ✅ \-
MiniMax M2.5, M3 NVFP4 ✅ ✅ ✅
Mixtral 8x7B FP8 ⚠ ⚠ ⚠
Mixtral 8x7B NVFP4 ⚠ \- \-
============================================ ============== ============ ====== ========

Vision-language and multimodal models
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For VLMs, modelopt quantizes the language model only; the vision encoder is kept in high precision.
The exported checkpoint therefore relies on the serving framework's own multimodal support for that
architecture — see the
`TensorRT-LLM multimodal support matrix <https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/models/supported-models.md#multimodal-feature-support-matrix-pytorch-backend>`_.

.. important::
✅ in this table is **text-only smoke coverage**. The suite sends the same plain-text prompts it
uses for language models, so no image or audio input reaches the processor or vision encoder.
These entries show that the quantized checkpoint loads and that its language path generates —
they do not demonstrate multimodal serving.

============================================ ============== ============ ====== ========
Model Quant format TensorRT-LLM vLLM SGLang
============================================ ============== ============ ====== ========
Qwen 2.5-VL 7B FP8, NVFP4 ✅ ✅ ✅

@meenchen meenchen Aug 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Qualify these checkmarks as text-only or exercise the multimodal paths

Every backend helper uses the same plain-string COMMON_PROMPTS: TRT-LLM tokenizes and generates text, vLLM builds text-only chat messages, and SGLang calls generate with text. No image or audio input reaches the processor or encoder. These cases validate checkpoint loading plus the language path, not the multimodal serving support implied by this section. Please add representative media requests and assertions, or explicitly label these cells as text-only smoke coverage.

@Edwardf0t1 Edwardf0t1 Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified and fixed. All three backend helpers send the same COMMON_PROMPTS — four plain strings — so nothing reaches a processor or vision encoder.

Added an .. important:: block above the table stating that ✅ there is text-only smoke coverage: it shows the quantized checkpoint loads and its language path generates, and does not demonstrate multimodal serving. I did not add media requests to the suite; that is a test change rather than a docs one, and it belongs with the CI work in the thread above.

Qwen 3-VL 235B-A22B NVFP4 ✅ ✅ ✅
Nemotron 3 Nano Omni 30B-A3B FP8, NVFP4 ✅ ✅ ✅
============================================ ============== ============ ====== ========

Speculative decoding drafters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Drafters are deployed on top of their base checkpoint.

Two caveats specific to this table:

* **Most entries are doubly conditional.** Beyond the ``--run-release`` gate, the drafter cases in
``test_eagle`` also require ``MODELOPT_LOCAL_EAGLE_MODEL`` to point at a directory containing the
drafter, and skip otherwise. The exception is EAGLE3 for Kimi K2.6, which is declared in
``test_kimi`` without that gate — which is also why it is the one row with vLLM coverage.
* **Medusa is marked ⚠ because the case does not exercise Medusa.** The shared harness builds a
speculative-decoding configuration only when the model ID contains ``eagle``, so the Medusa entry
performs ordinary generation. It shows the checkpoint loads and serves; it does not validate
Medusa decoding.

============================================================ ============ ============ ====== ========
Drafter Quant format TensorRT-LLM vLLM SGLang
============================================================ ============ ============ ====== ========
EAGLE3 for Llama 3.3 70B, Llama 4 Maverick FP8 ✅ \- ✅
EAGLE3 for Qwen 3 235B-A22B (incl. Thinking-2507, FP4) BF16, NVFP4 ✅ \- ✅
EAGLE3 for Qwen 3 30B-A3B-Thinking-2507 BF16 ✅ \- ✅
EAGLE3 for Kimi K2-Thinking, K2.5 NVFP4 ✅ \- ✅
EAGLE3 for Kimi K2.6 NVFP4 ✅ ✅ ✅
EAGLE3 for gpt-oss-120b BF16 ✅ \- ✅
Medusa for Llama 3.1 8B FP8 ⚠ \- ⚠
============================================================ ============ ============ ====== ========

Diffusion models
~~~~~~~~~~~~~~~~

============================================ ============== ============ ====== ========
Model Quant format TensorRT-LLM vLLM SGLang
============================================ ============== ============ ====== ========
Wan 2.2 T2V A14B FP8, NVFP4 ⚠ \- ⚠
DiffusionGemma 26B-A4B NVFP4 ✅ ✅ ✅
============================================ ============== ============ ====== ========

Wan 2.2 is marked ⚠ because its cases run through the same autoregressive text helper as the
language models and assert on generated text. They never call a diffusion or video serving API, so
they do not substantiate text-to-video deployment.

Models:
* Llama 4, 3.x (FP8, NVFP4)
* Qwen 3, 2.5 (FP8, NVFP4)
* Qwen 3 MoE (FP8, NVFP4)
* Qwen 3-VL (FP8, NVFP4)
* Deepseek R1/V3 (NVFP4)
* Mixtral 8x7B (FP8, NVFP4)
* Medusa (FP8)
* Eagle (FP8)

Requirements: TensorRT-LLM v0.17.0 or later

vLLM
~~~~

Models:
* Llama 4, 3.x (FP8, NVFP4)
* Qwen 3, 2.5 (FP8, NVFP4)
* Qwen 3 MoE (FP8, NVFP4)
* Mixtral 8x7B (FP8)
* Deepseek R1/V3 (NVFP4)

Requirements: vLLM v0.10.1 or later

SGLang
~~~~~~
.. note::
NVFP4 inference requires Blackwell GPUs. Hopper can produce an NVFP4 checkpoint but cannot serve
it. On B300/GB300 (``sm_103``) use a CUDA-13 build of the serving framework; CUDA-12 builds lack
the ``sm_103`` FP4 kernels.

Models:
* Llama 4, 3.x (FP8, NVFP4)
* Qwen 3, 2.5 (FP8, NVFP4)
* Qwen 3 MoE (FP8, NVFP4)
* Deepseek R1/V3 (NVFP4)
Models not listed here
~~~~~~~~~~~~~~~~~~~~~~

Requirements: SGLang v0.4.10 or later
This matrix records the combinations modelopt validates. It is not an exhaustive list of what will
run: vLLM, SGLang, and TensorRT-LLM load unified HF checkpoints generically, so a model built from
standard ``nn.Linear`` layers with an ``hf_quant_config.json`` will often deploy without any modelopt
change. Check the serving framework's own model support list first, then try it.

Note: While other models and quantization formats may work, they have not been thoroughly tested and validated.
The exact checkpoints behind every ✅ above, including tensor-parallel size and minimum SM
version, are listed in
`tests/examples/hf_ptq/test_deploy.py <https://github.com/NVIDIA/Model-Optimizer/blob/main/tests/examples/hf_ptq/test_deploy.py>`__;
most are published under the
`NVIDIA Hugging Face organization <https://huggingface.co/nvidia>`_.


Deployment with Selected Inference Frameworks
Expand All @@ -102,7 +229,7 @@ Deployment with Selected Inference Frameworks

Follow the `TensorRT-LLM installation instructions. <https://nvidia.github.io/TensorRT-LLM/quick-start-guide.html#installation>`_

Currently we support fp8 and nvfp4 quantized models for TensorRT-LLM deployment, you need v0.17.0 or later version of TensorRT-LLM.
FP8 and NVFP4 quantized models are supported; you need v1.2.0 or later version of TensorRT-LLM.

To run modelopt quantized model from Huggingface model hub, e.g., `nvidia/Llama-3.1-8B-Instruct-FP8`_, refer to the sample code below:

Expand Down Expand Up @@ -136,7 +263,8 @@ Deployment with Selected Inference Frameworks

Follow `vLLM installation instructions. <https://github.com/vllm-project/vllm?tab=readme-ov-file#getting-started>`_

Currently we support fp8 quantized models (without fp8 kv cache) for vLLM deployment, you need v0.6.5 or later version of vLLM.
FP8 and NVFP4 quantized models are supported; you need v0.10.1 or later version of vLLM. Pass
``quantization="modelopt"`` for FP8 and ``quantization="modelopt_fp4"`` for NVFP4.

To run modelopt quantized model from Huggingface model hub, e.g., `nvidia/Llama-3.1-8B-Instruct-FP8`_, refer to the sample code below:

Expand Down Expand Up @@ -171,7 +299,8 @@ Deployment with Selected Inference Frameworks

Follow the `SGLang installation instructions. <https://docs.sglang.ai/get_started/install.html>`_

Currently we support fp8 quantized models (without fp8 kv cache) for SGLang deployment, you need to use the main branch of SGLang (since Jan 6, 2025) and build it from source.
FP8 and NVFP4 quantized models are supported; you need v0.4.10 or later version of SGLang. Pass
``quantization="modelopt"`` for FP8 and ``quantization="modelopt_fp4"`` for NVFP4.

To run modelopt quantized model from Huggingface model hub, e.g., `nvidia/Llama-3.1-8B-Instruct-FP8`_, refer to the sample code below:

Expand Down
2 changes: 1 addition & 1 deletion examples/diffusers/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ mtq.quantize(model=transformer, config=quant_config, forward_func=forward_pass)

> *<sup>1.</sup>The w4a8_awq is an experimental quantization scheme that may result in a higher accuracy penalty.*

> *<sup>2.</sup>A selective set of the popular models are internally tested. The actual model support list may be longer. NVFP4 inference requires Blackwell GPUs and TensorRT-LLM v0.17 or later*
> *<sup>2.</sup>A selective set of the popular models are internally tested. The actual model support list may be longer. NVFP4 inference requires Blackwell GPUs and TensorRT-LLM v1.2 or later*

> *<sup>3.</sup>The SVDQuant Perf in TRT might not good as the [Nunchaku: MIT-Nvidia](https://github.com/nunchaku-tech/nunchaku) at this moment.*

Expand Down
41 changes: 19 additions & 22 deletions examples/hf_ptq/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ Please reference our [framework scripts](#framework-scripts) and our [docs](http
> *<sup>2.</sup>For some models, there is only support for exporting quantized checkpoints.* \
> *<sup>3.</sup>W4A8_AWQ is only available on some models but not all* \
> *<sup>4.</sup>For some models, KV cache quantization may result in a higher accuracy penalty.* \
> *<sup>5.</sup>A selective set of the popular models are internally tested. The actual model support list may be longer. NVFP4 inference requires Blackwell GPUs and TensorRT-LLM v0.17 or later* \
> *<sup>5.</sup>A selective set of the popular models are internally tested. The actual model support list may be longer. NVFP4 inference requires Blackwell GPUs and TensorRT-LLM v1.2 or later* \
> *<sup>6.</sup>Some models currently support export to HF format only.* \
> *<sup>7.</sup>[PTQ for DeepSeek](../deepseek/README.md)* \
> *<sup>8.</sup>GLM-4.7 has MTP (Multi-Token Prediction) layers that are automatically loaded and excluded from quantization.* \
Expand Down Expand Up @@ -587,27 +587,24 @@ print(llm_fp8.generate(["What's the age of the earth? "]))

### Unified HF Checkpoint Deployment Model Support Matrix

| Model | Quant format | TRT-LLM | vLLM | SGLang |
| :---: | :---: | :---: | :---: | :---: |
| LLAMA 3.x | FP8 | ✅ | ✅ | ✅ |
| LLAMA 3.x | FP4 | ✅ | ✅ | ✅ |
| LLAMA 4 | FP8 | ✅ | - | ✅ |
| LLAMA 4 | FP4 | ✅ | - | - |
| DS-R1 | FP8 | ✅ | ✅ | ✅ |
| DS-R1 | FP4 | ✅ | ✅ | ✅ |
| DS-V3 | FP8 | ✅ | ✅ | ✅ |
| DS-V3 | FP4 | ✅ | ✅ | ✅ |
| QWen3 | FP8 | ✅ | ✅ | ✅ |
| QWen3 | FP4 | ✅ | ✅ | - |
| QWen3 MoE | FP8 | ✅ | ✅ | ✅ |
| QWen3 MoE | FP4 | ✅ | - | - |
| QWen3.5 MoE | FP4 | - | - | ✅ |
| QWen2.5 | FP8 | ✅ | ✅ | ✅ |
| QWen2.5 | FP4 | ✅ | ✅ | - |
| QwQ-32B | FP8 | ✅ | ✅ | ✅ |
| QwQ-32B | FP4 | ✅ | ✅ | - |
| Mixtral 8x7B | FP8 | ✅ | ✅ | ✅ |
| Mixtral 8x7B | FP4 | ✅ | - | - |
The deployment support matrix — which model families and quantization formats are covered on
TRT-LLM, vLLM, and SGLang, including vision-language models, speculative decoding drafters, and
diffusion models — lives in the documentation so there is a single copy to keep current:

**[Unified HF Checkpoint → Model Support Matrix](https://nvidia.github.io/Model-Optimizer/deployment/3_unified_hf.html#model-support-matrix)**

@cjluo-nv cjluo-nv Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this always synced with the latest main branch?

@Edwardf0t1 Edwardf0t1 Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not automatically, no — the docs site publishes from main, so that link tracks the merged state of 3_unified_hf.rst, but nothing enforces that the matrix itself stays in step with test_deploy.py. Today it is a human keeping one file current instead of three.

That is the remaining gap, and it is the same one this PR only partially closes: consolidating to a single copy removes the three-way drift, but not the doc-vs-test drift. A generator plus a CI check would close it properly. I would rather do that as its own change than grow this one — noted in the PR description as follow-up. If you would prefer the link point at the versioned main doc path instead of the latest published page, easy to switch.


Each entry there is drawn from [`tests/examples/hf_ptq/test_deploy.py`](../../tests/examples/hf_ptq/test_deploy.py),
which loads the exported checkpoint in each framework and generates from short text prompts. That
file is also the place to look for the exact checkpoint, tensor-parallel size, and minimum SM
version behind each entry.

> *Note: those cases are marked `release` and run out-of-band — no workflow currently passes
> `--run-release` — and each is a load-and-generate smoke check on the text path. Read the legend in
> the docs before treating an entry as verified support.*

> *Note: the matrix records what modelopt validates, not the full set of what will run. vLLM, SGLang,
> and TRT-LLM load unified HF checkpoints generically, so unlisted models frequently deploy without
> any modelopt change — check the serving framework's own model support list and try it.*

### (Legacy) TensorRT-LLM Checkpoints

Expand Down
Loading
Loading