[AMD][WIP][AGENTX]: Kimi-K3 DSpark on MI355X by haic0 · Pull Request #2496 · SemiAnalysisAI/InferenceX

haic0 · 2026-08-05T03:26:15Z

Summary

Add the Kimi-K3 MXFP4 TP8 AgentX benchmark on MI355X with the validated DSpark ROCm image.
Start LMCache 0.5.3rc2 MP host-DRAM offload with a 906 GB L1 pool, configurable MP port (default 6000), FP8 KV cache, and Mamba alignment.
Cap vLLM scheduling at --max-num-batched-tokens 3000, use three DSpark speculative tokens, and temporarily enforce eager execution.
Apply the checksum-pinned production-file diff from vLLM #50619 to fix native causal Kimi-K3 DSpark FP8 MLA verification; document the temporary patch in docs/waiver/2496.md.
Route the AgentX MTP matrix to the dedicated DSpark launcher and pre-staged NFS model cache.

Test plan

Validate shell syntax, YAML syntax, and patch formatting.
Generate and validate six TP8 AgentX rows at concurrency 1, 4, 8, 16, 32, and 64.
Pass matrix-logic and changelog-gate tests before synchronizing with current main; the merge did not modify those test paths.
Validate that the pinned #50619 patch applies to the serving image and EngineCore starts on MI355X.
Complete the new MI355X AgentX sweep triggered by commit 040546012.

Add the TP8 AgentX sweep, validated ROCm image, LMCache MP offload recipe, and launcher routing for Kimi-K3 DSpark. 中文：新增 Kimi-K3 DSpark 在 MI355X 上的 TP8 AgentX 扫描、已验证 ROCm 镜像、LMCache MP 卸载配置及启动器路由。 Co-authored-by: Cursor <cursoragent@cursor.com>

Preserve current deprecations and append the Kimi-K3 DSpark sweep entry after the latest performance changelog records. 中文：同步主分支的弃用变更，并将 Kimi-K3 DSpark 扫描记录追加到最新性能变更日志之后。 Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-05T03:26:23Z

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

claude · 2026-08-05T05:16:50Z

Claude finished @haic0's task in 6m 9s —— View job

Review: Kimi-K3 DSpark on MI355X

Gather context (AGENTS.md, PR diff, sibling recipes)
Review benchmark script kimik3_fp4_mi355x_agentic_dspark.sh
Review configs/amd-master.yaml + perf-changelog.yaml gate
Review runners/launch_mi355x-amds.sh routing
Post review

Five 🟡 warnings, no blockers. The config plumbing, launcher routing, and changelog are all clean; every finding is in the benchmark script, and four of the five are places where this recipe silently diverges from its two sibling recipes (dsv4_fp4_mi355x_vllm.sh and kimik3_fp4_b300_vllm_mtp.sh) in ways that will change the numbers this sweep publishes.

The two I'd fix before the sweep: MAX_NUM_SEQS=$CONC clips AgentX subagent fan-out at the scheduler (both siblings use 2 * CONC with an explicit comment about exactly this), and --max-num-batched-tokens 3000 is ~10x below anything used for a long-context agentic replay — that one carries a real risk of blowing the 1800s warmup grace period rather than just running slow. The other three are the un-staged DSpark draft model, --enforce-eager (which also leaves VLLM_USE_BREAKABLE_CUDAGRAPH=0 dead), and a partial copy of the DSv4 LMCache timeout block.

Things I checked that are fine: the changelog entry is correctly appended to the end of the file and pull/XXX is an accepted placeholder (utils/validate_perf_changelog.py:24-26), so the gate will rewrite it on merge. The 906 GB L1 pool fits comfortably under the 2400 GB budget the matrix generates for cluster:mi355x-amds at dram-utilization: 0.80. Expert parallelism isn't hardcoded — the script hard-errors on EP_SIZE > 1 instead, which is consistent with the B300 recipe. The launcher's new agentic-DSpark branch is correctly gated behind -f "$SCRIPT_AGENTIC_DSPARK" so it can't hijack other models, and the HF_HUB_CACHE_MOUNT override is meaningful (the default is /var/lib/hf-hub-cache/). --use-chat-template does not apply here: agentic replay drives /v1/chat/completions through AIPerf, so prompts are already chat-shaped — same rationale documented at kimik3_fp4_b300_vllm_mtp.sh:35-38.

One note on what I could not verify: python utils/matrix_logic/generate_sweep_configs.py full-sweep was blocked by this environment's sandbox, so I validated the config against validation.py's pydantic models and the generator's agentic code path by reading rather than by execution. Worth confirming the generator runs clean locally before labeling full-sweep-fail-fast.
• haic0/kimik3-mtp-agentic

claude · 2026-08-05T05:22:28Z

+    --enable-prefix-caching
+    --mamba-cache-mode align
+    --kv-cache-dtype fp8
+    --enforce-eager


🟡 WARNING: --enforce-eager disables HIP graph capture on a benchmark whose purpose is measuring decode performance, and it makes line 64 dead.

Why it matters: export VLLM_USE_BREAKABLE_CUDAGRAPH=0 at line 64 has no effect once graphs are never captured, which suggests the two flags were assembled from different sources. Concretely, eager decode is where this recipe's low-concurrency arms live — conc 1/4/8 are pure per-step-overhead regimes, so the published TPOT for those rows measures Python dispatch overhead rather than the MI355X kernel path. The B300 Kimi-K3 sibling goes the other way and builds spec-decode-aware capture sizes explicitly (kimik3_fp4_b300_vllm_mtp.sh:200-214), noting that "cudagraph capture sizes are expressed in TOKENS, not sequences" because DSpark rounds sizes up to multiples of 1 + num_speculative_tokens.

If eager is a deliberate workaround for a DSpark + AITER crash on this image, that's fine — but it needs a comment saying so and the dead VLLM_USE_BREAKABLE_CUDAGRAPH line should go, otherwise the next person bumping this image has no way to know the constraint still applies.

claude · 2026-08-05T05:30:56Z

+export VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS="${VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS:-1200}"
+export VLLM_ENGINE_READY_TIMEOUT_S="${VLLM_ENGINE_READY_TIMEOUT_S:-7200}"
+export HSA_NO_SCRATCH_RECLAIM=1
+export SAFETENSORS_FAST_GPU=1
+
+SERVER_LOG="$RESULT_DIR/server.log"
+LMCACHE_LOG="$RESULT_DIR/lmcache_server.log"
+mkdir -p "$RESULT_DIR"
+
+SERVER_PID=""
+LMCACHE_PID=""
+
+cleanup_services() {
+    local exit_code=$?
+    trap - EXIT INT TERM
+    set +e
+    stop_background_process_tree "$SERVER_PID" "vLLM server" 60
+    stop_background_process_tree "$LMCACHE_PID" "LMCache server" 30
+    exit "$exit_code"
+}
+trap cleanup_services EXIT
+trap 'exit 130' INT
+trap 'exit 143' TERM
+
+wait_for_lmcache_ready() {
+    { set +x; } 2>/dev/null
+    local attempts="${LMCACHE_READY_ATTEMPTS:-1800}"
+    local paths=(/healthcheck /health /v1/health /status /)


🔴 This new agentic recipe drives the same Kimi-K3 trace-replay path as kimik3_fp4_b300_vllm.sh/kimik3_fp4_b300_vllm_mtp.sh but never exports AIPERF_HTTP_TCP_USER_TIMEOUT (set by every other vLLM agentic recipe, incl. the same-SKU dsv4_fp4_mi355x_vllm.sh) or VLLM_HTTP_TIMEOUT_KEEP_ALIVE (set by the identical-model B300 siblings, which document a real production incident where omitting it aborted a whole benchmark job). Without these, a long warmup request or an inter-turn idle gap can trip AIPerf into treating a healthy server as dead, aborting the entire agentic sweep.

Extended reasoning...

The bug: kimik3_fp4_mi355x_agentic_dspark.sh sets up an extensive environment block (lines 65-92) for the validated Kimi-K3 + DSpark + LMCache image, but it never exports AIPERF_HTTP_TCP_USER_TIMEOUT or VLLM_HTTP_TIMEOUT_KEEP_ALIVE. Every other vLLM agentic recipe in the repo exports AIPERF_HTTP_TCP_USER_TIMEOUT=900000 (e.g. dsv4_fp4_mi355x_vllm.sh:61, the same MI355X/vLLM SKU+framework), and the two same-model B300 siblings — kimik3_fp4_b300_vllm.sh and kimik3_fp4_b300_vllm_mtp.sh — additionally set VLLM_HTTP_TIMEOUT_KEEP_ALIVE=900 with an explicit comment explaining why it's necessary.

Why this matters — the documented failure mode: kimik3_fp4_b300_vllm.sh:77-86 describes a real production incident: AIPerf pins one pooled keep-alive HTTP connection per agentic session and reuses it across conversation turns. If the inter-turn idle gap exceeds the server's keep-alive timeout (5s by default), the client can grab the socket for reuse in the exact instant the server closes it, producing an aiohttp.ServerDisconnectedError. AIPerf treats this as a terminal warmup failure — not a retryable blip — and aborts the entire benchmark job. The comment explicitly cites run 30324907690, where this "killed the dram c4 arm ~15 min into run" against a server that "kept serving after the client gave up." Separately, AIPERF_HTTP_TCP_USER_TIMEOUT=900000 exists because agentic warmup dispatches large prompts at once, and the client needs up to 15 minutes of TCP progress before AIPerf gives up on the connection as dead — this is the universal convention across every vLLM/sglang agentic recipe in the repo.

The code path that triggers it: This script drives the identical agentic mechanism as the B300 siblings — resolve_trace_source, build_replay_cmd, and run_agentic_replay_and_write_outputs (lines 92, 189-192) — for the same Kimi-K3 model with the same DSpark draft, on a 1.5+ TB checkpoint whose warmup requests are long and whose per-turn idle gaps are exactly the pattern that triggers the keep-alive race. Nothing in benchmark_lib.sh sets a default for either variable — each recipe is individually responsible for exporting them — and grepping the new file confirms neither is present.

Why nothing else prevents it: The script does set generous timeouts elsewhere (VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS=1200, VLLM_ENGINE_READY_TIMEOUT_S=7200), showing the author is aware that this workload needs extended timeouts, but those don't cover the AIPerf-side TCP dead-connection detection or the server-side HTTP keep-alive window that caused the documented B300 incident. VLLM_USE_RUST_FRONTEND isn't set here, so the timeout technically applies to uvicorn's timeout_keep_alive on the standard FastAPI frontend (default 5s) rather than the Rust frontend — but the default is the same 5s, so the same race is structurally possible; the incident report just happens to have been diagnosed on the Rust frontend.

Step-by-step proof of the failure:

The script launches vllm serve with a large model and starts an agentic trace-replay session via run_agentic_replay_and_write_outputs.

AIPerf opens one pooled connection per agentic session and issues turn 1, waits for the response, then some idle time passes before turn 2 is dispatched (normal for coding-agent traces).

If that idle gap exceeds the server's keep-alive timeout (default ~5s, since VLLM_HTTP_TIMEOUT_KEEP_ALIVE is unset), the server closes the idle socket.

AIPerf's connection pool doesn't know the socket was just closed and attempts to reuse it for turn 2, racing the server's close — this yields aiohttp.ServerDisconnectedError.

AIPerf treats this error during warmup as terminal, not retryable, and aborts the entire job — exactly what the B300 sibling's comment documents happening in production (run 30324907690).

Separately/additionally, if a single warmup prompt is large enough that TCP progress alone exceeds AIPerf's default TCP-user-timeout (shorter than 900s), AIPerf can declare the connection dead mid-request even without an idle gap, also aborting the job.

Fix: Add the same two exports used by the B300 siblings to this script's environment block:

export VLLM_HTTP_TIMEOUT_KEEP_ALIVE=900 export AIPERF_HTTP_TCP_USER_TIMEOUT=900000

Impact if unfixed: Since the PR's own test plan leaves the "MI355X End-to-End AgentX smoke validation" and "canonical full sweep" checkboxes unfixed, this failure mode has not yet been exercised against this new recipe, so the risk is currently unvalidated. The consequence when it does trigger is not degraded numbers but a spurious, terminal abort of the whole agentic sweep (wasted GPU time on the 8xMI355X node, a false-negative CI run), matching the exact incident the identical-model B300 recipes were patched to avoid.

claude · 2026-08-05T05:30:56Z

      - { tp: 4, conc-start: 4, conc-end: 128 }

+kimik3-fp4-mi355x-vllm-agentic:
+  image: haihub007/hc-gpu:kimik3-dspark-lmcache-working-20260804


🔴 The new kimik3-fp4-mi355x-vllm-agentic recipe pins haihub007/hc-gpu:kimik3-dspark-lmcache-working-20260804 — a personal Docker Hub image and the only non-upstream image anywhere in amd-master.yaml — with no docs/waiver/2496.md on file to justify it. Per docs/PR_REVIEW_CHECKLIST.md, a patched/forked-engine image (the -lmcache-working tag plus the baked-in DSpark spec-decoding support) requires a waiver documenting what's patched, why the stock upstream image can't run this benchmark, and a removal plan; without it the CODEOWNER sign-off gate can't be satisfied, so this should be resolved (waiver, or swap to an upstream vllm/vllm-openai-rocm image) before merge.

Extended reasoning...

The issue. configs/amd-master.yaml:468 pins the new kimik3-fp4-mi355x-vllm-agentic recipe to haihub007/hc-gpu:kimik3-dspark-lmcache-working-20260804. Tallying every image: line in amd-master.yaml shows this is the only entry outside the vllm/, lmsysorg/, and rocm/ namespaces — every other recipe in the file, including all the other Kimi-K3/K2.5 and DSv4 entries, runs a vendor- or community-maintained image. The tag itself (...-lmcache-working-...) reads as a developer working build, and the recipe's own benchmark script (benchmarks/single_node/agentic/kimik3_fp4_mi355x_agentic_dspark.sh) wires in features that don't ship in stock vLLM: --speculative-config {"method":"dspark", ...} with a custom draft model (Inferact/Kimi-K3-DSpark), and LMCacheMPConnector via lmcache.integration.vllm.lmcache_mp_connector. That's a patched/forked engine build baked into the image, not a stock upstream container.

Why the checklist requires action here. docs/PR_REVIEW_CHECKLIST.md has two relevant, independent gates. The provenance gate (line 26) requires vLLM submissions to run the upstream hub.docker.com/u/vllm image, with an exception for new hardware or model architectures vLLM doesn't yet support — Kimi-K3+DSpark can plausibly claim that exception. But the separate no-patch gate (line 30) is not exempted by the model-architecture carve-out: it requires the pinned image to "run as shipped," explicitly calling out "installing forked/rebuilt engine wheels on top of the pinned image" as prohibited, unless a filled-out waiver exists at docs/waiver/<PR_NUMBER>.md stating what's patched, why the unmodified upstream image can't run this benchmark, the upstream PR/issue link, and a removal plan. I checked docs/waiver/ directly — it contains only an empty .gitkeep, so no 2496.md (or any other) waiver has been filed for this PR.

Step-by-step proof:

grep -n '^\s*image:' configs/amd-master.yaml | grep -vE 'vllm/|lmsysorg/|rocm/' → exactly one hit: line 468, haihub007/hc-gpu:....

ls -la docs/waiver/ → only .gitkeep (0 bytes), no PR-numbered waiver file exists.

The recipe's launch script (kimik3_fp4_mi355x_agentic_dspark.sh) sets SPEC_CONFIG with "method":"dspark" and a custom draft model, and KV_TRANSFER_CONFIG with LMCacheMPConnector — both non-stock vLLM engine behavior baked into the pinned image, which is exactly what the no-patch/waiver rule targets.

Cross-checking configs/nvidia-master.yaml, the sibling kimik3-fp4-b300-vllm-agentic-dspark recipe runs the upstream vllm/vllm-openai:kimi-k3 pre-release tag for the same DSpark+Kimi-K3 combination, showing an upstream-image path for this exact feature set does exist and is used elsewhere — undercutting the case that a personal-namespace image is unavoidable here.

Conclusion: the no-patch gate applies, no waiver was filed, so the CODEOWNER sign-off template (mirrored by the automated sign-off bot per AGENTS.md) cannot be truthfully completed as-is.

On the refutation raised during review: one verifier argued the waiver rule only covers runtime patching (sed/heredoc rewrites, monkey-patching after container start) and that pre-built custom images are already common in this file (citing lmsysorg/sglang-rocm SHA-pinned builds and the lmcache git-SHA pin in dsv4-fp4-mi355x-vllm-agentic). That distinction doesn't hold up: the checklist text explicitly lists "installing forked/rebuilt engine wheels on top of the pinned image" as a covered case, which describes exactly what a -working tagged personal image with baked-in DSpark support is — a rebuilt engine wheel/image, just built once ahead of time rather than patched in the entrypoint. The cited comparisons are also not equivalent: lmsysorg/sglang-rocm and rocm/atom are vendor-maintained namespaces (SGLang's own ROCm builds, AMD's own atom builds), not a third party's personal Docker Hub account, and the lmcache SHA pin in the DSv4 recipe is a config parameter pointing at an open-source library commit, not the serving-engine container image itself. The WIP/draft status is a timing point, not a refutation of the substance — flagging the missing waiver now, before the PR is marked ready for review, is the point of catching it in review.

Impact and fix. Without resolution, this PR cannot pass the CODEOWNER sign-off gate as documented, and merging it as-is means the sweep runs untracked patched-engine code under --container-writable --container-remap-root (root execution of a third-party image on shared CI runners) with no record of what's different from upstream or how/when it gets retired. The fix is straightforward: either point the recipe at an upstream vllm/vllm-openai-rocm (or vLLM's kimi-k3 pre-release, mirroring the NVIDIA sibling recipe) image, or file docs/waiver/2496.md describing the DSpark/LMCache-MP patches, why upstream can't run this benchmark yet, the upstream PR/issue tracking it, and a removal plan.

github-actions · 2026-08-05T05:59:30Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30977751660
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30977751660

…k.sh Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

github-actions · 2026-08-05T06:55:41Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30977751660
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30977751660

…k.sh Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

github-actions · 2026-08-05T06:59:32Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30983027936
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30983027936

Invoke the GPU-drain gate added by PR #2490 before downloading or loading Kimi-K3, preventing overlap with memory still held by a prior Slurm job. 中文：在下载或加载 Kimi-K3 前调用 PR #2490 新增的 GPU 清理门禁，避免与上一 Slurm 任务尚未释放的显存重叠。 Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-05T07:04:20Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30983225748
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30983225748

github-actions · 2026-08-05T07:29:57Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30983559838
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30983559838

Cap batched tokens for stable agentic scheduling and align the validated LMCache and speculative-decoding settings. Co-authored-by: Cursor <cursoragent@cursor.com>

Keep the performance changelog synchronized with the validated scheduler and speculative-decoding configuration. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-05T07:35:24Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30985458858
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30985458858

github-actions · 2026-08-05T08:01:07Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30985536172
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30985536172

github-actions · 2026-08-05T08:33:45Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30987892934
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30987892934

Apply the pinned upstream FP8 verification fix, reduce speculative depth, and use eager execution so the MI355X sweep can start reliably. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-05T09:15:58Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30990683132
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30990683132

Switch to the image with vLLM PR #50619 baked in so runners no longer depend on a fragile runtime patch. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-05T11:07:48Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30996489968
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30996489968

github-actions · 2026-08-05T14:41:52Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31013602529
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31013602529

Pin AgentX to the requested v1.0.0 release and keep agentx-fast sweeps throughput-only so short validation runs do not dispatch a separate GSM8K evaluation. Co-authored-by: Cursor <cursoragent@cursor.com>

Preserve the latest main changelog entries while keeping the Kimi-K3 AgentX sweep entry appended for PR validation. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-06T02:43:35Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31065495212
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31065495212

Move the MI355X sweep to the image containing the skinny-GEMM, FP8 MLA prefill, DSpark verification, and large-KV fixes, while disabling the unsafe skinny-GEMM path as a defensive fallback. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-06T04:17:45Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31069976905
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31069976905

Use a single concurrency-2 AgentX validation with the rebuilt combined ROCm image and keep the changelog aligned with its integrated fixes. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-06T06:03:07Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31075403223
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31075403223

Move the c2 validation to a fresh image tag so MI355X squashfs caching cannot reuse the pre-shadow-fix container. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-06T06:58:58Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31077120323
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31077120323

Route the c2 validation through the image that keeps 12-head single-token decode on Gluon, avoiding the unavailable padded AITER ASM kernel. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-06T11:39:48Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31081089907
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31081089907

Exercise graph execution with prefix caching across concurrency 2 through 32 now that the MLA fallback image completes the c2 run. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-06T12:29:37Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31099220015
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31099220015

Use TRITON_MLA with piecewise graph capture and fused RMS norm gating at concurrency 4 to narrow the prior full-graph sampling failure. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-06T15:30:21Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31112994010
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31112994010

haichzha@amd.com and others added 2 commits August 5, 2026 03:23

github-project-automation Bot added this to InferenceMAX Board Aug 5, 2026

haic0 changed the title ~~feat(agentx): Kimi-K3 DSpark on MI355X / 新增 MI355X DSpark~~ [AMD][WIP][AGENTX]: Kimi-K3 DSpark on MI355X Aug 5, 2026

haic0 marked this pull request as ready for review August 5, 2026 05:16

haic0 requested a review from a team August 5, 2026 05:16

haic0 requested review from 1am9trash, billishyahao, chunfangamd, seungrokj and yctseng0211 as code owners August 5, 2026 05:16

haic0 added the full-sweep-fail-fast label Aug 5, 2026