Skip to content

fix: preserve BF16 embedding weights for get_rows - #1959

Merged
leejet merged 1 commit into
leejet:masterfrom
xledx:bf16-get-rows
Sep 11, 2026
Merged

fix: preserve BF16 embedding weights for get_rows#1959
leejet merged 1 commit into
leejet:masterfrom
xledx:bf16-get-rows

Conversation

@xledx

@xledx xledx commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Preserve BF16 token embedding weights when they are consumed by GET_ROWS.

The generic support_get_rows() check did not include GGML_TYPE_BF16, so BF16 embedding weights could be converted to F32 even though the CPU backend already implements BF16 GET_ROWS.

This change:

  • adds GGML_TYPE_BF16 to the generic GET_ROWS capability check
  • marks token_embedding.weight as a GET_ROWS consumer in the CLIP model loader

Related Issue / Discussion

None.

Additional Information

Validated on an Intel Mac in CPU-only inference with MiniMax H3 / Qwen embedding weights.

The BF16 embedding tensor is preserved instead of being expanded to F32, and inference completes successfully.

Checklist

@leejet
leejet merged commit e95ab96 into leejet:master Sep 11, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants