Skip to content

feat(infiniop): add lightning_attention op (CPU + NVIDIA) for MiniMax - #1564

Open
y258dd wants to merge 2 commits into
InfiniTensor:InfiniLM-v0.2.9cfrom
y258dd:feat/minimax-lightning-attention
Open

y258dd wants to merge 2 commits into
InfiniTensor:InfiniLM-v0.2.9cfrom
y258dd:feat/minimax-lightning-attention

Conversation

@y258dd

@y258dd y258dd commented Sep 19, 2026

Copy link
Copy Markdown

NVIDIA RTX 4090 / sm_89 Validation (2026-09-19)

Validated with:

  • InfiniCore feat/minimax-lightning-attention at 7bcaa623
  • InfiniLM feat/minimax-model-support at e23eba79
  • RTX 4090, compute capability sm_89, driver 570.124.06, CUDA 12.8 (V12.8.61)
  • PyTorch 2.6.0a0+ecf3bae40a.nv25.01 with CUDA 12.8, torch.cuda.is_available() == True
  • Python 3.12.3, g++ 13.3.0, xmake 3.1.1+20260919

Results

Test Device Result Error
MiniMax lightning attention operator NVIDIA Passed decode: out 4.768e-07, state 2.384e-07; prefill: out 1.907e-06, state 4.768e-07
MiniMax smoke prefill/decode NVIDIA Passed finite logits; state continuity 3.127381e-06
MiniMax vs HuggingFace, 1 expert, FP32 NVIDIA Passed prefill 5.257875e-04; decode 2.117679e-04
MiniMax vs HuggingFace, 4 experts, top-2, FP16 NVIDIA Passed prefill 7.324219e-04; decode 3.662109e-04

All tests are below the 1e-2 model-level tolerance. The 4-expert top-2 result exercises the CUDA Fused MoE path rather than the dense fallback.

Fixes Included

  • CUDA Lightning Attention now supports FP16 and BF16 in addition to FP32.
  • MiniMax linear-attention state uses the model dtype instead of always allocating FP32.
  • Sampling offsets are copied to host before host-side indexing on CUDA.
  • HuggingFace MiniMax MoE weights are remapped to InfiniLM expert keys.
  • Lightning-attention GPU tests use disjoint initial/final state rows to avoid cross-request races.

Server Logs

  • /data/minimax-final-gpu-op.log
  • /data/minimax-final-gpu-smoke.log
  • /data/minimax-final-gpu-vs-hf.log
  • /data/minimax-gpu-test-moe4-top2-fp16-final.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant