perf(ds4): batch heterogeneous MoE prefill by expert

[cheese-cakee]

Summary

Accelerates DeepSeek-V4 heterogeneous prefill by batching owner-local routed experts into grouped ggml_mul_mat_id graphs instead of issuing a serial graph for every active expert.

• enables the expert-major heterogeneous path from 32 tokens instead of 512;
• builds grouped gate/up/down MMID graphs per owner GPU;
• adds GPU-native route histogram, prefix, and scatter work for up to 256 experts;
• preserves masked owner routes as exact zero contributions;
• retains DFLASH_MOE_GROUPED_MMID_PREFILL=0 and DFLASH_MOE_EXPERT_MAJOR_MIN_TOKENS=512 as rollback controls.

Classification: primarily a performance PR. It also contains the masked-route correctness fix discovered during qualification, but that fix supports the optimization rather than changing the PR's main purpose.

Fresh main-snapshot requalification

Freshly remeasured on Lucebox6 against recorded main snapshot f686c447f067a04ea100a996e4c826e8cc4decc1, using the same ROCmFP2 model, placement, chunk size, cache-cold requests, two warmups, and five measured samples per cell.

Workload	Recorded main	#640	Delta
401-token prefill, chunk=512	94.59 tok/s (MAD 0.15)	178.98 tok/s (MAD 0.17)	+89.2% (1.89x)
2048-token prefill, chunk=2048	329.16 tok/s (MAD 3.23)	298.97 tok/s (MAD 2.05)	-9.17%

The current positive result is the short/medium 401-token workload: 1.89x the recorded-main-snapshot throughput. #640 alone is not a current wide-context win; at 2048 tokens it regresses. The stacked #647 scheduling PR recovers that wide result while adding another 7.0% at 401 tokens.

All retained responses produced SHA-256 cd5cb9fb5ac3c4f4007e8b41d117da21622439cd05c1728f3e82f90e4f869dad, and every retained request was cache-cold.

Evidence:

• 401-token root: /home/cheese/pr640-pr647-fusion-live-main-short401-20260825T121700Z
• 401 manifest SHA-256: 6a9d6b586ac04fc153b9af2b87a5175c01fe3178d9808c9817f7b89f87ac8d21
• 2048-token root: /home/cheese/pr640-pr647-fusion-live-main-requal2-20260825T112300Z
• 2048 manifest SHA-256: b73afed6803e26ed111f521eb0690cf9d8056e30c5f890b0067a5604d181e66a

Original matched qualification

The original same-session qualification against main at ac22a3ed measured:

Workload	Main	#640	Result
401-token prefill, chunk=512	54.16 tok/s	184.11 tok/s	3.40x (+239.9%)
MoE FFN time, 401 tokens	6,791.6 ms	1,661.2 ms	4.09x reduction
2048-token prefill, chunk=2048	292.42 tok/s	304.69 tok/s	+4.2%

These remain valid historical matched-session results for those exact heads and machine state. The fresh table above is the relevant comparison to the newer recorded main snapshot; absolute rates must not be mixed across sessions.

Correctness and validation

• exact full-model output parity across the retained 401- and 2048-token cells;
• test_deepseek4_mmid_grouped_cuda: PASS across 72 cases on physical gfx1201 and gfx1151, including wide masked ROCmFP2/ROCmFP3 routes;
• test_deepseek4_unit: PASS on physical gfx1201 and gfx1151;
• compact MMID destinations are zero-initialized before dispatch so unwritten masked lanes cannot contribute stale or NaN values;
• git diff --check: clean;
• all currently reported GitHub checks: successful; GitHub reports the PR mergeable and clean.

[cubic-dev-ai]

All reported issues were addressed across 5 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>