[Cherry-pick] PRs #2172 #2087 #2152 #2060 #2008 #2194

.agents/skills/deployment/references/support-matrix.md

@@ -2,26 +2,18 @@
 
 ## Unified HF Checkpoint — Framework Compatibility
 
-| Model | Quant Format | TRT-LLM | vLLM | SGLang |
-|-------|-------------|---------|------|--------|
-| Llama 3.x | FP8 | yes | yes | yes |
-| Llama 3.x | FP4 | yes | yes | yes |
-| Llama 4 | FP8 | yes | — | yes |
-| Llama 4 | FP4 | yes | — | — |
-| DeepSeek R1 | FP8 | yes | yes | yes |
-| DeepSeek R1 | FP4 | yes | yes | yes |
-| DeepSeek V3 | FP8 | yes | yes | yes |
-| DeepSeek V3 | FP4 | yes | yes | yes |
-| Qwen 3 | FP8 | yes | yes | yes |
-| Qwen 3 | FP4 | yes | yes | — |
-| Qwen 3 MoE | FP8 | yes | yes | yes |
-| Qwen 3 MoE | FP4 | yes | — | — |
-| Qwen 2.5 | FP8 | yes | yes | yes |
-| Qwen 2.5 | FP4 | yes | yes | — |
-| QwQ-32B | FP8 | yes | yes | yes |
-| QwQ-32B | FP4 | yes | yes | — |
-| Mixtral 8x7B | FP8 | yes | yes | yes |
-| Mixtral 8x7B | FP4 | yes | — | — |
+**Do not maintain a copy of the matrix here.** The single source of truth is
+`docs/source/deployment/3_unified_hf.rst` ("Model Support Matrix"), and every entry in it is drawn
+from `tests/examples/hf_ptq/test_deploy.py`.
+
+Read that doc's legend before reporting a model as supported: the cases are marked `release` and do
+not run on PR CI, and each is a load-and-generate smoke check on the text path — so an entry is
+declared coverage, not proof the combination serves correctly.
+
+To answer "is model X supported on framework Y", read one of those two files — `test_deploy.py` is
+the more precise answer, since it also carries the exact checkpoint, tensor-parallel size, and
+minimum SM version per entry. It covers language models, VLMs (Qwen2.5-VL, Qwen3-VL,
+Nemotron Omni), EAGLE3/Medusa drafters, and diffusion models.
 
 ## Supported Quantization Formats
 
@@ -50,18 +42,19 @@
 | SGLang | `quantization="modelopt"` | `quantization="modelopt_fp4"` |
 | TRT-LLM | auto-detected from checkpoint | auto-detected from checkpoint |
 
-## Models not in this list
+## Models not in the matrix
 
-This matrix covers officially validated combinations. For unlisted models:
+The matrix covers the combinations modelopt tracks, not the full set of what will run. For unlisted models:
 
 1. **Check the framework's own docs** — vLLM and SGLang support many HuggingFace models natively. Use WebSearch to check `vllm supported models` or `sglang supported models`.
 2. **Try it** — if the model uses standard `nn.Linear` layers and has `hf_quant_config.json`, vLLM/SGLang will likely work with `--quantization modelopt`.
-3. **Ask the user** — if unsure, ask: "This model isn't in the validated support matrix. Would you like to try deploying it anyway?"
+3. **Ask the user** — if unsure, ask: "This model isn't in the support matrix. Would you like to try deploying it anyway?"
 
 ## Notes
 
 - **NVFP4 inference requires Blackwell GPUs** (B100, B200, B300, GB200, GB300). Hopper can run FP4 calibration but not inference.
   - **B300/GB300 are `sm_103`** and need a **CUDA-13** serving image — from v0.20.0 the unsuffixed tag is CUDA-13 (`-cu129` opts back to CUDA 12); `cu12` images lack the `sm_103` FP4 kernel and serve NVFP4 as gibberish or error out. See the CUDA-13 note in the deployment `SKILL.md`.
   - **Verify the GPU with `nvidia-smi`** before choosing the image — cluster GPU labels can be stale.
 - INT4_AWQ and W4A8_AWQ are only supported by TRT-LLM (not vLLM or SGLang).
-- Source: `examples/hf_ptq/README.md` and `docs/source/deployment/3_unified_hf.rst`
+- For VLMs, only the language model is quantized; the vision encoder stays in high precision, so multimodal serving depends on the framework's own support for that architecture.
+- Source: `docs/source/deployment/3_unified_hf.rst` and `tests/examples/hf_ptq/test_deploy.py`

CHANGELOG.rst

@@ -88,6 +88,9 @@ Changelog
 - Fix HF checkpoint export failing with ``AttributeError: 'list' object has no attribute 'keys'`` for models whose modeling code still declares tied weights in the ``transformers<5`` list format (NVBug 6518665, observed on ``stepfun-ai/Step-3.7-Flash``). transformers 5.0 changed ``_tied_weights_keys`` to a ``{target: source}`` dict and ``save_pretrained`` calls ``.keys()`` on every submodule's declaration without a type check, so such models — common among ``trust_remote_code`` checkpoints — load fine but die at the end of PTQ, after calibration. ModelOpt's ``save_pretrained`` patch now normalizes a list-style declaration to the equivalent dict for the duration of the save (each entry mapped to itself, which is what the legacy list meant) and restores the original attribute afterwards.
 - Fix unified HF export of multimodal models whose vision tower carries its own ``PrefixChange`` conversion (``LlavaForConditionalGeneration`` on ``transformers>=5.12`` — NVBug 6525511). transformers collects conversion mappings recursively and scopes each sub-model's transforms to that sub-module via ``scope_prefix``, matching only keys under that prefix. ModelOpt's quant-aware reverse conversion read the raw patterns and ignored ``scope_prefix``, so the vision tower's "add a ``vision_model.`` prefix" rule was applied to *every* key in the state dict: an exported llava-1.5-13b checkpoint had all 758 tensors moved under a bogus top-level ``vision_model.`` namespace (``vision_model.language_model.*``, ``vision_model.lm_head.*``), and vLLM rejected it with ``ValueError: There is no module or parameter named 'vision_model' in LlavaForConditionalGeneration``. Reverse rename rules now carry their scope and are applied only to keys under it, matching transformers' own ``WeightTransform._scoped_match`` semantics. ``Gemma3ForConditionalGeneration`` was affected identically and is fixed by the same change.
 - Fix QLoRA export in ``examples/llm_qat/export.py`` failing with ``AssertionError: Model already has modelopt state!`` (NVBug 6542481). The QLoRA training output is an adapter-only checkpoint, so ``from_pretrained`` resolves the quantized base model from ``adapter_config.json`` and ``enable_huggingface_checkpointing`` already restores its ModelOpt state; the export then restored a second time. It now restores only when the loaded model is not already converted. Two further breakages on the same path are also fixed: ``_restore_qtensor_wrappers`` matched no modules because PEFT re-parents the quantized linear as ``<name>.base_layer`` while ``q_tensor_state`` is keyed by the name it was saved with (the packed NVFP4 weight then reached ``F.linear`` and raised a shape error), and ``postprocess_state_dict`` silently dropped every ``base_layer.*`` key missing from a hand-maintained rename map — losing the NVFP4 ``weight_scale_2`` global scale and any linear ``bias`` (Qwen2-style q/k/v biases), and leaving ``base_layer`` in the exported AWQ ``pre_quant_scale`` key. The rename is now a generic ``.base_layer.`` strip.
+- Fix ``--use_fsdp2`` PTQ (``examples/hf_ptq``) failing on models that hold a few parameters in a dtype other than the model's own, with ``AssertionError: FSDP expects uniform original parameter dtype`` on the first calibration forward. Nemotron-3-Nano is one such model: its MoE router gates are declared ``float32`` while the rest of the checkpoint is bfloat16, so each decoder layer's FSDP2 shard group mixed dtypes. ``fsdp2_wrap`` now passes those off-dtype parameters to ``fully_shard(ignored_params=...)``, leaving them replicated in their original dtype instead of casting them, and warns with their names and their share of the model.
+- Fix ``--use_fsdp2`` HF export making no progress for hours on large MoE checkpoints. ``create_fsdp_param_mapping`` resolved each ``FSDPParam``'s module by scanning every ``model.named_parameters()``, and export calls it once per quantized module, so the cost was quadratic in (parameters x modules): harmless for dense models, intractable for a MoE with many experts. Exporting Nemotron-3-Nano-30B-A3B (6,243 parameter tensors, 6,004 quantized modules) spent an estimated 1.9 hours there with every GPU idle. The parameter index is now built once per mapping instead of once per ``FSDPParam`` (1151 ms -> 5.1 ms per call), preserving the previous ``named_parameters()``-order resolution for tied weights.
+- Update HuggingFace checkpoint export to use name-based tied-weight deduplication instead of the previous address-based approach. The address-based deduplication could incorrectly drop an untied weight that happened to share memory with a tied one, producing an incomplete checkpoint (observed as a false positive on MiniMax-M2.7).
 
 0.45 (2026-07-02)
 ^^^^^^^^^^^^^^^^^