Skip to content

From NVIDIA Megatron-LM for visibility - #18

Open
RaymondLi0 wants to merge 7904 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main
Open

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 7904 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main

Conversation

@RaymondLi0

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

Copy link
Copy Markdown
Collaborator

No description provided.

@RaymondLi0
RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0
RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
Jingyue Wu and others added 27 commits August 18, 2026 17:19
Jingyue Wu
test(mfsdp): separate memory tests (#6568)
54c7918 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Teodor-Dumitru Ene
Support PackedSeqParams CP CG as in #2282 (#6571)
d5a019a 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Robert Kirby
Use named module logger in dynamic inference engine (#6001)
aaa4dfd 
Signed-off-by: Robert Kirby <rkirby@nvidia.com>
Tom Long Claude Opus 5 (1M context)
Correct the split-PR skill's stacking rules for copy-pr-bot CI (#6593)
aeae5d7 
Signed-off-by: ilml <tolong@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Shiqing Fan
fix(gtp+grad-reduce-in-fp32): emit the wgrad in main_grad's dtype so …
e5542e8 
…the reduce-scatter honours FP32 (#6624)

Signed-off-by: Shiqing Fan <shiqingf@nvidia.com>
Yashaswi Karnati
Honor high-priority streams in MIMO topology (#6604)
bf51cf4 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Teodor-Dumitru Ene Vitaly Kurin Jorge Albericio
Share prefill in sequence packing (#5887)
3ad8c2a 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Co-authored-by: Vitaly Kurin <yobibyte@users.noreply.github.com>
Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>
GitHub Actions
Update copy-pr-bot.yaml [skip ci]
5f9b2bb
Cory Ye
[MFSDPv1] Fix gradient sharding / allocation schedule with ZeRO-2, an…
342faa4 
…d fix wgrad accumulate fusion for ZeRO-1. (#6357)

Signed-off-by: Cory Ye <cye@nvidia.com>
Jingyue Wu
Add opt-in MFSDP unified communication stream (#6563)
540f420 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Maanu Grover
tests: allow trusted TE pickle extra state in affected tests (#6602)
1d69364 
Signed-off-by: Maanu Grover <maanug@nvidia.com>
Teodor-Dumitru Ene
Reflect #4440 onto GPT models, not just hybrid (#6476)
afa707e 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Charlie Truong
ci: Remove scheduled CI workflow (#6640)
0961f12 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
oliver könig
fix(ci): request pull request permission for queue comments (#6641)
e7d58db 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Philip Petrakian
Remove core teams from FSDP owners (#6623)
29361d6 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
Tom Long Claude Fable 5
Add THD-format per-rank batch fetch with padding mask for sequence pa…
c452d5a 
…cking (#6625)

Signed-off-by: ilml <tolong@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Yashaswi Karnati
Optimize MIMO pipeline communication (#6606)
f481e63 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
github-actions[bot]
chore: rotate oncall schedule
cab7116
Tong Liu Jingyue Wu Jingyue Wu
Support no-shard, ZeRO-1, and ZeRO-2 in MFSDP v2 (#6137)
01dc20b 
Signed-off-by: tongliu <tongliu@nvidia.com>
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Signed-off-by: Tong Liu <liutongt1998@gmail.com>
Co-authored-by: Jingyue Wu <jingyuew@nvidia.com>
Co-authored-by: Jingyue Wu <wujingyue@gmail.com>
wdykas
Add hybrid SSM state handoff (#6584)
731b791 
Signed-off-by: William Dykas <wdykas@nvidia.com>
Carlos Gomes
Checkpoint shared storage when inputs are returned (#4497)
19deef6 
Signed-off-by: CarlosGomes98 <cgomes@nvidia.com>
Guihong Li
Add script-transfer procedure to GPT-to-Hybrid migration skill (#6251)
7e3780f 
Signed-off-by: guihong-nv <guihongl@nvidia.com>
Ajay
Remove leading NaN iteration times from golden values (#6539)
93ebf8d 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
Tom Long Claude Fable 5
Add varlen dataset schema-detection and message-converter helpers (#5908
9691f47 
)

Signed-off-by: ilml <tolong@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
wdykas
MXfp8 Inference Fixes and clean up (#6588)
a1912b3 
Signed-off-by: wdykas <wdykas@nvidia.com>
Ajay
ci: centralize container builds and add cache population workflow (#6646
e7b92ef 
)

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
Dmytro Pykhtar
tokenizers simplification (#6622)
e433171 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com>
Dmytro Pykhtar and others added 30 commits September 8, 2026 18:32
Dmytro Pykhtar
add raise_on_error argument (#7110)
c768946 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com>
Deyu Fu Philip Petrakian
Add native DeepSeek-V4 hybrid attention orchestration (#6402)
907609b 
Signed-off-by: Deyu Fu <deyuf@nvidia.com>
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
svcnemo-autobot
fix: AUT-2228 route partial reruns to the current attempt (#7142)
62fa723 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
going-song Yoonsik Kim
Yechan Kim svcnvidia-nemo-ci
Use metadata dp_cp_group for the tied-embeddings replica_id (#6888)
154f665 
Signed-off-by: Kayeon Song <kayeon.song@navercorp.com>
Signed-off-by: Yoonsik Kim <yoonsik.kim90@navercorp.com>
Signed-off-by: Yechan Kim <yechan.78@navercorp.com>
Signed-off-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Co-authored-by: Yoonsik Kim <yoonsik.kim90@navercorp.com>
Co-authored-by: Yechan Kim <yechan.78@navercorp.com>
Co-authored-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Tom Long
Keep RerunDataIterator wrapping on the hybrid CP data iterator (#6822)
7fa83b5 
Signed-off-by: Tom Long <tolong@nvidia.com>
svcnemo-autobot
ci: AUT-2245 set CUDA major for lockfile updates (#7151)
be85fc5 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
Jiangfei Duan
fix(MoE): Preserve exact expert token counts for expert-bias updates (#…
8dc7ed3 
…7081)

Signed-off-by: Jiangfei Duan <jiangfeid@nvidia.com>
github-actions[bot]
chore: rotate oncall schedule
c6be919
svcnemo-autobot
ci: AUT-2250 retry transient coverage artifact uploads (#7159)
8dcf188 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
Rui Zhu Claude Fable 5 svcnvidia-nemo-ci
Enter both autocast and fp8 contexts in combined 1F1B execution (#5761)
0d2dc9c 
Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu>
Signed-off-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Ahmad Kiswani
Derive MFSDP v2 checkpoint chunk metadata from the parameter layout (#…
b9c22eb 
…6214)

Signed-off-by: Ahmad Kiswani <akiswani@nvidia.com>
Philip Petrakian
Add Slack reminders for re-requested reviews (#7071)
b92c3ed 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
Yixuan Xu Philip Petrakian
Propagate rope_scaling_factor from args to GPT model construction (#6306
9ca681c 
)

Signed-off-by: Alvorecer721 <yixuan.xu@ai.ethz.ch>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
oliver könig
ci(pr-review): extract review prompts into a non-activating skill (#7170
56ef6ed 
)

Signed-off-by: oliver könig <okoenig@nvidia.com>
Shanmugam Ramasamy
First pass at comprehensive user guide (#5325)
5590c4b 
Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com>
Dingqing Yang
Support FP4 in 1F1B A2A overlap (#6135)
74dc1b5 
Signed-off-by: Dingqing Yang <dingqingy@nvidia.com>
Teodor-Dumitru Ene
RL: prevent unnecessary entry into inference mode (#4125)
b2cea22 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Jingyue Wu
Document signing rewritten commits (#7037)
ba638f0 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Jenny Chen
Test SFTTokenizer token loss masking (#7023)
aa9b579 
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Jingyue Wu
Refactor MFSDP parameter group initialization (#7175)
d455089 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Tom Long Jared Casper
Assert the runtime-CP group contract both ways (#6825)
9be4a57 
Signed-off-by: Tom Long <tolong@nvidia.com>
Signed-off-by: ilml <tolong@nvidia.com>
Co-authored-by: Jared Casper <155158+jaredcasper@users.noreply.github.com>
Jingyue Wu
Support full-iteration CUDA graphs with MFSDP v2 (#7075)
8190837 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Shiqing Fan xuwenc
fix(gtp): correct expert gradient scaling for GTP/EGTP mismatch (non-…
b005bf1 
…per-token-loss) (#7165)

Signed-off-by: Shiqing Fan <shiqingf@nvidia.com>
Signed-off-by: xuwenc <xuwenc@nvidia.com>
Co-authored-by: xuwenc <xuwenc@nvidia.com>
Jingyue Wu
Fix MFSDP v2 precision-aware optimizer gradient clipping (#7074)
c9e7ea7 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
svcnemo-autobot
fix(ci): AUT-2260 mirror DCO status before merge queue (#7210)
a7ba373 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
Yan Xu
Re-derive quantized params from FP32 main params on checkpoint load (#…
abccf5b 
…6666)

Signed-off-by: Yan Xu <yxu1@nvidia.com>
Ting SUN svcnvidia-nemo-ci
Fix order local refit copies after source updates (#6823)
79be654 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Co-authored-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Shiqing Fan
fix(moe): keep precision-overridden grouped experts off the op-fuser … (
8962453 
#7212)

Signed-off-by: Shiqing Fan <shiqingf@nvidia.com>
Guihong Li
Select kernel backends through BackendSpecProvider, not HAVE_* flags (#…
c238266 
…6766)

Signed-off-by: guihong-nv <guihongl@nvidia.com>
Fei Wu
Enable NCCL EP tests (#7195)
7613fd7 
Signed-off-by: YangFei1990 <feiw@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.