Skip to content

[PyTorch] Add torch.compile custom ops for GroupedLinear - #32

Draft
pggPL wants to merge 1 commit into
grouped_linear_refactorfrom
grouped_linear_compile
Draft

[PyTorch] Add torch.compile custom ops for GroupedLinear#32
pggPL wants to merge 1 commit into
grouped_linear_refactorfrom
grouped_linear_compile

Conversation

@pggPL

@pggPL pggPL commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Description

Add torch.compile(fullgraph=True) support to te.pytorch.GroupedLinear on top of the eager refactor in #50. Both custom ops reuse the shared forward/backward computation from that PR.

This is the second of two stacked PRs; its base is grouped_linear_refactor.

Type of change

  • New feature (non-breaking change which adds functionality)

Changes

  • Add custom-op registrations and fake implementations for both paths, sharing weight/cache metadata, gradient allocation descriptions and saved-alias restoration.
  • Extend _FieldPlan / _ArgPlan with tensor-list packing, per-element gradients, subclass flattening and SymInt[] inputs. Flatten mixed tensor/storage lists in one pass, and cover empty SymInt lists in autograd with a regression test.
  • Deduplicate saved-tensor aliases and apply compile-specific allocation restrictions. Preallocate the grouped cuBLAS workspace for CUDA-graph capture.
  • Use shared compile eligibility checks and dispatch for both paths, with path-specific restrictions and dimension checks reusing Linear's guards. Compile tests share numerical comparisons, including bias gradients across weight-cache updates.

Validation

RTX Ada workstation, current source:

  • test_torch_compile.py + test_grouped_linear.py: 849 passed, 608 skipped, 2 XPASS.
  • Additional fused BF16 fullgraph=True probe: forward/backward numerics and changing device splits without recompilation pass with grouped GEMM replaced by matmuls. This does not validate native fused kernels or CUDA-graph capture.
  • A temporary mixed plain-tensor/FP8-wrapper/storage/None list probe preserves exact values and buffer ordering.
  • Changed production files pass pylint; Black and diff checks pass. Full Python lint reports the existing import-order diagnostic in unchanged attention/fused_mla_q_uproj.py.

The installed PyTorch 2.14 development build exposes is_opaque_value_type instead of the is_opaque_constant_type name expected by TE main. Tests used a temporary compatibility alias outside the repository.

Native fused grouped GEMM tests require Hopper/Blackwell and were skipped on RTX Ada.

Limitations

Unsupported configurations fall back to eager: debug instrumentation, distributed weights, CPU offloading, delayed wgrad, fused wgrad accumulation, caller-provided output buffers, single grouped parameters and non-value-opaque quantizers. Legacy compiled execution requires host-side splits; fused execution requires CUDA splits. Fused save_original_input=True falls back to eager.

Checklist

  • I have read and followed the contributing guidelines
  • New and existing tests pass on the available GPU
  • Changes introduce no lint diagnostics
  • Native fused compilation revalidated on Hopper/Blackwell

@pggPL
pggPL force-pushed the grouped_linear_compile branch from 926952e to 45a2c8b Compare August 13, 2026 10:27
@pggPL
pggPL changed the base branch from linear_compile_on_main to linear_torch_compile_final_attempt August 13, 2026 10:27
@pggPL pggPL changed the title [PyTorch] [torch.compile] torch.compile support for GroupedLinear (plan) [PyTorch] [torch.compile] torch.compile support for GroupedLinear Aug 13, 2026
@pggPL
pggPL force-pushed the grouped_linear_compile branch from e431045 to 2859669 Compare September 10, 2026 10:33
@pggPL
pggPL changed the base branch from linear_torch_compile_final_attempt to main September 10, 2026 10:33
@pggPL
pggPL force-pushed the grouped_linear_compile branch from 2859669 to 71732da Compare September 10, 2026 10:44
@pggPL
pggPL changed the base branch from main to grouped_linear_refactor September 10, 2026 10:45
@pggPL
pggPL force-pushed the grouped_linear_compile branch from 71732da to 2db50bd Compare September 10, 2026 11:24
@pggPL
pggPL force-pushed the grouped_linear_refactor branch from 50ca413 to d007d1c Compare September 10, 2026 11:24
@pggPL pggPL changed the title [PyTorch] [torch.compile] torch.compile support for GroupedLinear [PyTorch] Add torch.compile custom ops for GroupedLinear Sep 10, 2026
@pggPL
pggPL force-pushed the grouped_linear_compile branch from 2db50bd to c89b414 Compare September 10, 2026 11:31
Reuse eager computation for both grouped paths and add fake implementations, registrations, saved-state serialization and compile dispatch. Extend argument packing and autograd routing for tensor lists and SymInt lists, including empty lists, with compile regression coverage.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
@pggPL
pggPL force-pushed the grouped_linear_refactor branch from d007d1c to a3c005d Compare September 11, 2026 11:34
@pggPL
pggPL force-pushed the grouped_linear_compile branch from c89b414 to 151f2b2 Compare September 11, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant