diff --git a/.gitignore b/.gitignore index d51a2c31..602abe78 100644 --- a/.gitignore +++ b/.gitignore @@ -22,6 +22,7 @@ site/ debug_code/ .pytest_cache/ *.pt +**/.env # Large test data (download separately if needed for regression tests) tests/onecomp/quantizer/jointq/data/model_layers_0_self_attn_k_proj.pth .uv-sync.lock diff --git a/CHANGELOG.md b/CHANGELOG.md index 556995d1..5da678ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Change log +### Documentation + +- Add troubleshooting information for running OneComp with Llama.cpp on macOS. + ## [v1.3.1] 2026-08-06 ### Bug Fix diff --git a/docs/user-guide/cpu-inference.md b/docs/user-guide/cpu-inference.md index 860a32d9..185de0c8 100644 --- a/docs/user-guide/cpu-inference.md +++ b/docs/user-guide/cpu-inference.md @@ -19,6 +19,48 @@ for inference. The direct export path additionally uses llama.cpp's pure-Python `convert_hf_to_gguf.py` to build the model metadata/tokenizer; it is fetched automatically (a shallow `git clone`) or taken from `$LLAMA_CPP_DIR` if set. + +### macOS + +If you have installed gcc or clang on macOS using a tool like Homebrew, the OpenMP associated with them will confilic with the OpenMP with OneComp's PyTorch backend. + +- OneComp's PyTorch backend is often cofigured to use OpenMP library located within .venv directory. +- Depending on your environment, Llama.cpp is configured to use OpenMP library associated with gcc or clang. + +If the following warning message is shown, this conflict may be occurring. + +```bash +**/multiprocessing/resource_tracker.py:279: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown + warnings.warn('resource_tracker: There appear to be %d ' +``` + +### solution (recommended) + +#### Search for OpenMP associated with PyTorch. + +Search for OpenMP library associated with PyTorch backend (using find command). + +```bash +$ find .venv -type f \( -name 'libomp.dylib' -o -name 'libgomp*.dylib' \) -print | grep torch +.venv/lib/python3./site-packages/torch/lib/libomp.dylib +``` + +#### Create an .env + +Create a .env file and add the OpenMP path as an environment variable, as shown below. + +```bash +DYLD_LIBRARY_PATH=".venv/lib/python3./site-packages/torch/lib" +``` + +#### Run uv + +Pass the following `--env-file` options when running uv run. + +```bash +uv run --env-file /path/to/.env ... python your_script.py +``` + ## One entry point: `export_to_gguf` You do not need to know which path a checkpoint requires. `export_to_gguf`