Eval bug: failed to load Qwen 3.8 Flash Next: GGML_ASSERT(sinfo.group_stage_slots.size() == allocation_group_stage_slots.size()) failed

[ethernidee]

Name and Version

e:\beellama\build\bin\Release>llama-server --version
version: 0.4.6-dev (build 11791, commit b27c886)
built with MSVC 19.43.34810.0 for x64
ggml op count: 105

Operating systems

Windows

GGML backends

CUDA

Hardware

Ryzen 7 5700X + 128 GB DDR4 RAM + RTX 5060 Ti 16 GB

Models

Qwen3.8-Flash-Next-UD-Q4_K_XL + mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf

Problem description & steps to reproduce

Crash on loading attempt:

0.00.058.748 I srv load_model: loading model 'd:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf'
0.54.097.508 I cmn init: llama threadpool init, n_threads = 8
E:\beellama\src\llama-kv-cache.cpp:2968: GGML_ASSERT(sinfo.group_stage_slots.size() == allocation_group_stage_slots.size()) failed

No crash on another fork.

Config:

e:\beellama\build\bin\Release\llama-server.exe ^
  -m "d:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf" ^
  -md "d:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf" ^
  --host 127.0.0.1 --port 8001 ^
  --n-gpu-layers all ^
  --fit off ^
  --ctx-size 150000 ^
  --n-cpu-moe 47 ^
  --min-p 0.0 ^
  --temp 1 --top-p 0.95 --top-k 20 ^
  -ctk kvarn8 -ctv kvarn6 --kv-tail-tokens 1024 ^
  -ctkd q8_0 -ctvd q8_0 ^
  --flash-attn on ^
  --batch-size 1280 --ubatch-size 1280 ^
  --threads 8 --threads-batch 8 ^
  --no-mmap --mlock ^
  --parallel 1 --prio 2 ^
  --log-colors off ^
  --kv-unified ^
  --metrics ^
  --spec-type draft-mtp --spec-draft-n-max 2 ^
  --reasoning on ^
  --jinja

First Bad Commit

No response

Relevant log output

Logs

e:\beellama\build\bin\Release\llama-server.exe   -m "d:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf"   -md "d:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf"   --host 127.0.0.1 --port 8001   --n-gpu-layers all   --fit off   --ctx-size 150000   --n-cpu-moe 47   --min-p 0.0   --temp 1 --top-p 0.95 --top-k 20   -ctk kvarn8 -ctv kvarn6 --kv-tail-tokens 1024   -ctkd q8_0 -ctvd q8_0   --flash-attn on   --batch-size 1280 --ubatch-size 1280   --threads 8 --threads-batch 8   --no-mmap --mlock   --parallel 1 --prio 2   --log-colors off   --kv-unified   --metrics   --spec-type draft-mtp --spec-draft-n-max 2   --reasoning on   --jinja
�[34m0.00.042.010�[0m �[35mW DEPRECATED: --mmap and --no-mmap are deprecated. use --load-mode mmap instead
�[0m�[34m0.00.042.015�[0m �[35mW DEPRECATED: --mlock is deprecated. use --load-mode mlock instead
�[0m0.00.042.673 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.042.963 W srv  llama_server: -----------------
0.00.042.966 W srv  llama_server: CORS is set to allow all origins ('*') and no API key is set
0.00.042.967 W srv  llama_server: this can be a security risk (cross-origin attacks)
0.00.042.969 W srv  llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
0.00.042.970 W srv  llama_server: -----------------
0.00.058.748 I srv    load_model: loading model 'd:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf'
0.54.097.508 I cmn          init: llama threadpool init, n_threads = 8
E:\beellama\src\llama-kv-cache.cpp:2968: GGML_ASSERT(sinfo.group_stage_slots.size() == allocation_group_stage_slots.size()) failed

[Anbeeld]

Already fixed on v0.4.7 branch.

[ethernidee]

Still no success on v0.4.7. Seems like MTP model with shared layers with the base model are not supported. It's taked from Unsloth Dynamics repository and works on 2 other llamacpp forks.

1.24.008.588 I cmn          init: llama threadpool init, n_threads = 8
1.24.306.023 I common_speculative_init_result: loading draft model 'd:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf'
1.24.724.801 E llama_model_load: error loading model: check_tensor_dims: tensor 'token_embd.weight' not found
1.24.724.807 E llama_model_load_from_file_impl: failed to load model
1.24.724.810 E common_speculative_init_result: failed to load draft model, 'd:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf'
1.24.724.823 E srv    load_model: failed to load draft model, 'd:\AI\Qwen3.8-Flash-Next-GGUF\UD-Q4_K_XL\mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf'
1.24.725.158 I srv   operator (): operator (): cleaning up before exit...
1.24.729.532 E srv  llama_server: exiting due to model loading error