[Klaud Cold] kimik2.5-fp4-b300-vllm: v0.26.0 image, extend conc to 512, add DEP4 arm / 升级 B300 镜像至 v0.26.0,扩展并发,新增 DEP4 - #2442
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30761976337 |
c3cace5 to
bfc24ba
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30766890597 |
749e714 to
cbe1cf2
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30767935435 |
d5a6cf1 to
50753a8
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30768059159 |
58cb5cd to
22f4d7f
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30768418255 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30772091801 |
fbbdf4a to
80d9ed8
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30785413941 |
e1b0ff9 to
b13329c
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30793840724 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30793898084 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31147659232 |
|
Reopening under new branch name. |
Summary
v0.22.0→vllm/vllm-openai:v0.26.0conc-end: 512tp: 4, ep: 4, dp-attn: true, conc-start: 128, conc-end: 512kimik2.5_fp4_b200.sh):DP_ATTENTIONhandling added — switches to--data-parallel-size, adds--enable-expert-paralleland--prefill-schedule-interval 4in DEP4 mode中文说明
v0.22.0→vllm/vllm-openai:v0.26.0conc-end: 512tp: 4, ep: 4, dp-attn: true, conc-start: 128, conc-end: 512kimik2.5_fp4_b200.sh):新增DP_ATTENTION处理逻辑,DEP4 模式下切换为--data-parallel-size,并启用--enable-expert-parallel和--prefill-schedule-interval 4