Skip to content

feat: add support for LoRA adapters - #25

Open
oglego wants to merge 2 commits into
mozilla-ai:mainfrom
oglego:feature/support-lora-adapters
Open

oglego wants to merge 2 commits into
mozilla-ai:mainfrom
oglego:feature/support-lora-adapters

Conversation

@oglego

@oglego oglego commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds support for LoRA adapters and closes #18. If there is anything that I should modify or change on this just let me know!

Changes

The main items that I have added on this PR are:

  • LoraAdapterConfig (path, scale) and ModelConfig::loras (std::vector<LoraAdapterConfig>) for configuring one or more adapters.
  • Model::initialize_context() loads and applies LoRA adapters to the model context with individual scaling via llama_set_adapters_lora().
  • Model::get_loras() getter for inspecting loaded adapter pointers.
  • examples/lora/ — Example application with documentation for converting adapters and using task prompt templates (including a worked example using IBM Granite RAG query rewriting).
  • tests/test_lora.cpp — unit tests covering config defaults, default scale, and multiple adapter stacking.
  • Root and examples README.md updates documenting LoRA usage and multi-agent context sharing.

For testing this locally and interacting with it I used the recommended model (granite-4.0-micro-Q8_0.gguf) along with a converted Granite query-rewriter LoRA adapter (query_rewrite_lora.gguf).

Testing

  • ctest — all 5 suites pass (ToolTests, CallbacksTests, GrammarTests, LoraTests, ChatParserTests)
  • clang-format passes
  • Manual verification against a real adapter: granite-4.0-micro-Q8_0.gguf + IBM Granite RAG's query-rewrite LoRA, confirmed the adapter correctly rewrites contextual follow-ups (e.g., "What about Germany?" → "What is the capital of Germany?") that the base model alone does not

Thanks in advance for the review!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Support for GBNF grammar and LoRA adapters in multi‑agent setups

1 participant