Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 18 additions & 7 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install Cython==3.0.12
pip install "Cython>=3.3.0,<3.4"

# The cythonized files allow installation from the sdist without cython
- name: Generate cython
Expand All @@ -37,9 +37,15 @@ jobs:
git apply ./tools/sdist.patch
pip3 install build
python3 -m build --sdist
# Isolated pip install uses the patched requires (no Cython).
# test whether tarball contains all files required for compiling
pip3 install dist/cydifflib-*.tar.gz

- name: Test sdist install
run: |
pip3 install pytest
pytest tests

- uses: actions/upload-artifact@v4
with:
name: artifact-sdist
Expand Down Expand Up @@ -72,8 +78,13 @@ jobs:

- uses: actions/setup-python@v5

- name: Skip vendoring host CRT on ARM64
if: matrix.arch == 'ARM64'
shell: bash
run: echo 'CIBW_REPAIR_WHEEL_COMMAND=delvewheel repair --exclude msvcp140.dll --exclude vcruntime140.dll --exclude vcruntime140_1.dll --exclude vcruntime140_threads.dll -w {dest_dir} -v {wheel}' >> "$GITHUB_ENV"

- name: Build wheels
uses: pypa/cibuildwheel@v2.23.2
uses: pypa/cibuildwheel@v4.2.1
with:
package-dir: cydifflib.tar.gz
output-dir: wheelhouse
Expand All @@ -91,7 +102,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [macos-13, macos-14]
os: [macos-15-intel, macos-14]
env:
CIBW_ARCHS: native
CIBW_TEST_SKIP: "pp*-macosx_*"
Expand All @@ -112,7 +123,7 @@ jobs:
run: cp dist/*.tar.gz cydifflib.tar.gz

- name: Build wheels
uses: pypa/cibuildwheel@v2.23.2
uses: pypa/cibuildwheel@v4.2.1
with:
package-dir: cydifflib.tar.gz
output-dir: wheelhouse
Expand All @@ -131,7 +142,7 @@ jobs:
fail-fast: false
matrix:
arch: [auto, aarch64, ppc64le, s390x]
python_tag: ["cp39-*", "cp310-*", "cp311-*", "cp312-*", "cp313-*", "pp39-*", "pp310-*", "pp311-*"]
python_tag: ["cp39-*", "cp310-*", "cp311-*", "cp312-*", "cp313-*", "cp314-*", "cp314t-*", "pp39-*", "pp310-*", "pp311-*"]
exclude:
# PyPy builds not available for these platforms
- arch: ppc64le
Expand Down Expand Up @@ -169,7 +180,7 @@ jobs:
name: Set up QEMU

- name: Build wheel
uses: pypa/cibuildwheel@v2.23.2
uses: pypa/cibuildwheel@v4.2.1
with:
package-dir: cydifflib.tar.gz
output-dir: wheelhouse
Expand Down Expand Up @@ -204,7 +215,7 @@ jobs:
run: cp dist/*.tar.gz cydifflib.tar.gz

- name: Build wheel
uses: pypa/cibuildwheel@v2.23.2
uses: pypa/cibuildwheel@v4.2.1
with:
package-dir: cydifflib.tar.gz
output-dir: wheelhouse
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
__pycache__/
.idea/
.venv/
.venv-*/
build/
_skbuild/
*.egg-info/
Expand All @@ -15,6 +16,7 @@ src/*.html
.coverage
coverage.xml
sde/
.pyproject.toml.sdist.bak

# Sphinx documentation
site/
Expand Down
77 changes: 77 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# CyDifflib Agent Instructions

This file follows the [AGENTS.md](https://agents.md) convention.
Claude Code reads `CLAUDE.md`, which imports this file through its `@AGENTS.md` line.

## Build and Test

Cython/C++ extension via **scikit-build-core** and **CMake**.

Published installs and GitHub Actions use pip and cibuildwheel.
Do not switch CI to uv.

Local work: use uv with the checked-in `uv.toml` if present.

```bash
UV_CONFIG_FILE=uv.toml uv sync --python 3.13 --all-groups --reinstall-package cydifflib
UV_CONFIG_FILE=uv.toml uv run --python 3.13 pytest
```

Default local interpreter is 3.13.
Also test 3.11, 3.12, 3.14, and 3.14t (`3.14` is GIL, `3.14t` is free-threaded).

End-to-end from an installed interpreter (import + the same pytest suite CI runs
after each wheel install):

```bash
for py in 3.11 3.12 3.13 3.14 3.14t; do
UV_CONFIG_FILE=uv.toml uv run --python "$py" python -c "import cydifflib; print(cydifflib.SequenceMatcher(None, 'abcd', 'bcde').ratio())"
UV_CONFIG_FILE=uv.toml uv run --python "$py" pytest
done
```

3.9, 3.10, and PyPy are CI-only.

Wheel jobs in `build.yml` set `CIBW_TEST_REQUIRES=pytest` and
`CIBW_TEST_COMMAND=pytest {package}/tests`. That installs the built wheel, then
runs `tests/` (including `test_gil_stays_disabled` on free-threaded tags).
Skips:

- Linux: `*_{aarch64,ppc64le,s390x}` and `*musllinux_*` (build only)
- Windows: `*-win32` (build only); `win_arm64` is cross-compiled on `windows-latest`
- macOS: `pp*-macosx_*` (build only)

The sdist job generates `.cxx`, strips Cython from `build-system.requires`,
installs the tarball, and runs pytest. Linux wheel tags are `cp39`–`cp314`,
`cp314t`, and `pp39`–`pp311`. macOS/Windows also build cibuildwheel extras
`cp315` / `cp315t` because those jobs do not set `CIBW_BUILD`.

Isolated wheel and sdist:

```bash
UV_CONFIG_FILE=uv.toml uv build --python 3.13
```

Sdist with generated C++ and Cython stripped from `build-system.requires`:

1. Generate `.cxx`:
`UV_CONFIG_FILE=uv.toml uv run --python 3.13 --with "Cython>=3.3.0,<3.4" ./src/cydifflib/generate.sh`
2. `cp pyproject.toml .pyproject.toml.sdist.bak`
3. `git apply ./tools/sdist.patch`
4. `UV_CONFIG_FILE=uv.toml uv build --python 3.13 --sdist`
5. `mv .pyproject.toml.sdist.bak pyproject.toml`

## Conventions

- **Layout:** `src/` (`src/cydifflib/`, tests in `tests/`)
- **Python:** published wheels still support 3.9+; local default is 3.13
- **Build:** Cython `>=3.3.0,<3.4` is build-only; do not list cmake or ninja in
`build-system.requires`
- **Version:** `src/cydifflib/__init__.py` (read at build time)
- **Free-threading:** `_initialize.pyx` sets `freethreading_compatible=True`;
`HtmlDiff._default_prefix` is locked; use one `SequenceMatcher` per thread
- **Lint:** do not run a repository-wide format of the `.pyx` sources

<!-- This document follows common-doc-guidelines.md.
See github.com/jlevy/practical-prose and review guidelines before editing.
-->
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## [Unreleased]
### Changed
- require Cython 3.3+ so free-threaded Python 3.14 can compile
- mark the extension free-threading compatible (one SequenceMatcher per thread;
HtmlDiff's shared prefix counter is locked)
- add support for Python 3.14 and 3.14t
- allow CMake 3.15 through 3.30

## [1.2.0] - 2025-04-11
### Changed
- drop support for Python 3.8
Expand Down
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@AGENTS.md
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.15...3.26)
cmake_minimum_required(VERSION 3.15...3.30)

cmake_policy(SET CMP0054 NEW)
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE)
Expand Down
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,21 @@ For a source build (for example from a SDist packaged) you only require a C++11
pip install git+https://github.com/rapidfuzz/CyDifflib.git@main
```

## Development

`pip install` remains the supported user path.
Local work can use uv with the checked-in `uv.toml`:

```bash
UV_CONFIG_FILE=uv.toml uv sync --python 3.13 --all-groups --reinstall-package cydifflib
UV_CONFIG_FILE=uv.toml uv run --python 3.13 pytest
```

Default is 3.13; also 3.11, 3.12, 3.14, and 3.14t (`3.14` GIL, `3.14t` free-threaded).
Isolated: `UV_CONFIG_FILE=uv.toml uv build --python 3.13`.

Free-threaded 3.14 (`3.14t`) needs Cython 3.3+ at build time. Use a separate `SequenceMatcher` per thread.

## 📖 Usage

The library can be used in the same way as difflib. Just use the `cydifflib` module instead of `difflib`:
Expand Down
18 changes: 15 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[build-system]
requires = [
"scikit-build-core>=0.11",
"Cython>=3.0.12,<3.1.0"
"scikit-build-core>=1.0",
"Cython>=3.3.0,<3.4"
]
build-backend = "scikit_build_core.build"

Expand All @@ -25,6 +25,7 @@ classifiers=[
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
]

[project.urls]
Expand All @@ -33,7 +34,13 @@ Repository = "https://github.com/rapidfuzz/CyDifflib.git"
Issues = "https://github.com/rapidfuzz/CyDifflib/issues"
Changelog = "https://github.com/rapidfuzz/CyDifflib/blob/main/CHANGELOG.md"

[dependency-groups]
dev = [
"pytest>=8",
]

[tool.scikit-build]
minimum-version = "build-system.requires"
sdist.include = [
"src/cydifflib/*.cxx",
]
Expand All @@ -49,10 +56,15 @@ wheel.exclude = [
"generate.sh"
]

[tool.scikit-build.metadata.version]
[[tool.dynamic-metadata]]
provider = "scikit_build_core.metadata.regex"
field = "version"
input = "src/cydifflib/__init__.py"

[tool.cibuildwheel]
# 3.14t needs no enable in cibuildwheel 4.2.1; pypy-eol keeps pp39/pp310.
enable = ["pypy", "pypy-eol"]

[tool.black]
line-length = 120

Expand Down
21 changes: 16 additions & 5 deletions src/cydifflib/_initialize.pyx
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# distutils: language=c++
# cython: language_level=3, binding=True, linetrace=True
# cython: language_level=3, binding=True, linetrace=True, freethreading_compatible=True

__all__ = ['get_close_matches', 'ndiff', 'restore', 'SequenceMatcher',
'Differ','IS_CHARACTER_JUNK', 'IS_LINE_JUNK', 'context_diff',
'unified_diff', 'diff_bytes', 'HtmlDiff', 'Match']

from heapq import nlargest as _nlargest
from collections import namedtuple as _namedtuple
import threading
# todo add this once it is supported in all Python versions
#from types import GenericAlias

Expand Down Expand Up @@ -116,6 +117,9 @@ cdef class SequenceMatcher:
case. SequenceMatcher is quadratic time for the worst case and has
expected-case behavior dependent in a complicated way on how many
elements the sequences have in common; best case time is linear.

A SequenceMatcher stores mutable scratch state on the instance. Do not
share one instance across threads; create one matcher per thread.
"""

cdef public object a
Expand All @@ -129,7 +133,7 @@ cdef class SequenceMatcher:
cdef public set bpopular
cdef public object autojunk

# todo this is not threadsafe, which could be an problem in the long run
# Per-instance scratch for find_longest_match. Not process-global.
cdef vector[Py_ssize_t] j2len_
cdef vector[Py_ssize_t] newj2len_
cdef Py_hash_t* a_
Expand Down Expand Up @@ -1722,13 +1726,19 @@ class HtmlDiff(object):
make_file -- generates complete HTML file with a single side by side table

See tools/scripts/diff.py for an example usage of this class.

make_table writes per-instance state; do not overlap calls on one HtmlDiff.
The shared HTML anchor counter is locked so separate instances can run
concurrently.
"""

_file_template = _file_template
_styles = _styles
_table_template = _table_template
_legend = _legend
# Unique fromN_/toN_ anchors across tables on one page.
_default_prefix = 0
_prefix_lock = threading.Lock()

def __init__(self,tabsize=8,wrapcolumn=None,linejunk=None,
charjunk=IS_CHARACTER_JUNK):
Expand Down Expand Up @@ -1929,9 +1939,10 @@ class HtmlDiff(object):

# Generate a unique anchor prefix so multiple tables
# can exist on the same HTML page without conflicts.
fromprefix = "from%d_" % HtmlDiff._default_prefix
toprefix = "to%d_" % HtmlDiff._default_prefix
HtmlDiff._default_prefix += 1
with HtmlDiff._prefix_lock:
fromprefix = "from%d_" % HtmlDiff._default_prefix
toprefix = "to%d_" % HtmlDiff._default_prefix
HtmlDiff._default_prefix += 1
# store prefixes so line format method has access
self._prefix = [fromprefix,toprefix]

Expand Down
Loading