diff --git a/README.md b/README.md index e0fd6ab..8da9633 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ Fast, free-threaded Python bindings for `PCRE2` with a stable `stdlib.re`-compat ## Latest News 🚀 -* 08/09–08/10/2026 **API performance and safety update**: bounded call-local fast paths accelerate `findall`, `split`, `sub`/`subn`, `Match.expand`, `lastindex`, flag handling, `template()`, and `escape` by a representative **2x to 38x+**, while large ordered `parallel_map(search)` workloads measured **7.55x to 7.71x** on the Apple arm64 free-threaded comparison runs. Compatibility fallbacks preserve complex patterns, subclasses, buffers, and callables; caches remain thread-scoped and size-bounded, and no fast path retains subjects, results, or extra captured values. Unsafe UTF bytes compilation is also blocked, with differential, randomized, concurrency, subprocess, and memory-safety coverage. 🧵⚡🛡️ +* 08/09–08/10/2026 **API performance and safety update**: common regex workloads and ordered `parallel_map(search)` workloads were benchmarked across Python 3.10 and free-threaded Python 3.14t/GIL=0. Compatibility fallbacks preserve complex patterns, subclasses, buffers, and callables. Unsafe UTF bytes compilation is also blocked, with differential, randomized, concurrency, subprocess, and memory-safety coverage. 🧵⚡🛡️ * 08/08/2026 **0.6.0**: `findall`, `finditer`, `sub`/`subn`, `split`, and `match`/`search`/`fullmatch` are now up to **46x faster** than `stdlib.re` and **48x faster** than `regex` on `finditer`/`findall` workloads, **13x** on `split`, and **2–9x** on `sub`/`subn` backref workloads, with full `re` semantics. Free-threaded `findall` reaches **13.8x** vs `re` on 8 threads. 🚀⚡ * 07/27/2026 [0.5.0](https://github.com/ModelCloud/PyPcre/releases/tag/v0.5.0): Zero-copy buffer-protocol subject support (`mmap.mmap`, `bytearray`, `array.array`) with UTF-8 validation and GIL=0-safe memory pinning. 🗂️⚡ * 07/24/2026 [0.4.0](https://github.com/ModelCloud/PyPcre/releases/tag/v0.4.0): C extension hardening (memory/pointer safety, bounds checks, atomic allocator init), GIL=0 safety verified, vectorized UTF-8 index/offset conversion, GIL-release threshold for small calls, C `findall` implementation, and README competitor benchmarks. 🛡️⚡ @@ -46,10 +46,10 @@ PyPcre pairs Python's familiar `re`-compatible API with the real `PCRE2` engine. - 🧬 **Full power of PCRE2**: PyPcre uses the real `PCRE2` engine, so you get native compile options, semantics, JIT, and upstream tuning. - 🔥 **More expressive regex syntax**: `PCRE2` supports constructs beyond stdlib `re`, including atomic groups `(?>...)`, possessive quantifiers `++`, branch-reset groups `(?|...)`, richer lookarounds, and backtracking control verbs like `(*SKIP)(*FAIL)`. -- 🧵 **Thread-safe into `nogil`**: PyPcre is built for `PYTHON_GIL=0`, with CI coverage, lock-aware caches, reusable match/JIT resources, and `parallel_map()` for multi-subject fan-out. +- 🧵 **Thread-safe into `nogil`**: PyPcre is built for `PYTHON_GIL=0`, with CI coverage and `parallel_map()` for multi-subject fan-out. - ⚡ **Fast on real workloads**: `PCRE2` JIT plus cached compiled patterns lets PyPcre match or beat `re` and `regex` on many common scans, especially multiline searches, lookaround-heavy patterns, and free-threaded execution. - 🛡️ **Safer operational story**: PyPcre prefers the system `libpcre2-8` shared library so normal OS package updates can bring security and bug-fix benefits without a bundled fork. -- ✅ **Validated thoroughly**: the project runs API tests, fuzz tests, memory-safety checks, local `valgrind` leak checks, and `massif` heap profiles. Recent local profiling found `0` definite leaks and `0` possible leaks in both the public API and raw binding paths. +- ✅ **Validated thoroughly**: the project runs API, fuzz, concurrency, and memory-safety tests. ### Quick Comparison 🥊 @@ -66,59 +66,9 @@ PyPcre pairs Python's familiar `re`-compatible API with the real `PCRE2` engine. ### Benchmark Highlights 🏁 -#### Controlled API hot-path A/B +The tables below summarize representative public workloads. Lower is better. -The current comparison is origin/main `03f4d10c379babcc9208e45705d774cb93bcebb8` -versus the PR head `3f8025eeea203a1f189aa8fffb31bbef236db26a`. It was measured -on an Apple M4 Max with 12 performance and 4 efficiency logical CPUs, macOS -26.6, PCRE2 10.47, Apple clang 21, and `regex==2025.11.3`. Python 3.10.11 is -GIL-enabled; Python 3.14.0rc2 is the -resolved Apple arm64 free-threaded build at -`/Users/diego/.local/share/uv/python/cpython-3.14.0rc2+freethreaded-macos-aarch64-none/bin/python3.14t` -and reports `sys._is_gil_enabled() == False`. - -Each API value is the median of 9 repetitions of 10,000 calls. Cold compile is -the median of 9 repetitions of 1,000 unique compiles. Lower is better. - -| Workload | 3.10 origin | 3.10 follow-up | Δ | 3.14t origin | 3.14t follow-up | Δ | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | -| Cached `compile(..., re.I|re.M|re.S|re.X)` | 0.652 μs | 0.655 μs | +0.5% | 0.424 μs | 0.427 μs | +0.7% | -| Module `search` | 0.561 μs | 0.570 μs | +1.6% | 0.456 μs | 0.469 μs | +2.9% | -| Cold compile | 6.849 μs | 7.191 μs | +5.0% | 4.747 μs | 5.099 μs | +7.4% | - -The earlier lock-heavy PR measurement for `a1946c5f6e1cee2dbac714d6a98e8eac47a19161` -reported cached-compile regressions of +83.6% (3.10) and +65.1% (3.14t), and -module-search regressions of +30.6% and +25.9%. The follow-up removes the -avoidable per-call configuration/thread-state locks and materially reduces those -regressions; it does not claim a speedup over origin. - -Reproduce the API comparison with the same scheduler-tier hint for both builds: - -```bash -cd /Users/diego/tmp-omni-workspace/pypcre/.worktrees/gil0-fixes -PYPCRE_BENCH_RUNS=10000 PYPCRE_BENCH_REPEATS=9 \ - taskpolicy -t 1 -l 1 env PYTHONPATH=. \ - ./.venv-gil0/bin/python benchmarks/api_hotpaths.py -``` - -Run the same command from the origin checkout with its Python 3.10.11 or -3.14.0rc2t environment for the A/B side. - -`taskpolicy -t 1 -l 1` is not hard P-core affinity on this unprivileged macOS -host; no P-core-only speedup is claimed. The API microbenchmarks are available in -[`benchmarks/api_hotpaths.py`](benchmarks/api_hotpaths.py), and the free-threaded -results should not be generalized to GIL-enabled interpreters. - -Measured on the Apple arm64 matrix above with compiled-pattern reuse and JIT enabled. Values are medians of three outer runs; lower is better. The comparison retains every workload row, including parity and slower cases. - -A reproducible version of this benchmark lives in [`benchmarks/competitor_bench.py`](benchmarks/competitor_bench.py). -The complete comparison also ran `finditer_bench.py`, `sub_bench.py`, -`split_bench.py`, `free_threaded_bench.py`, `api_hotpaths.py`, -`parallel_map_hotpath.py`, and the opt-in `tests/test_benchmark.py` matrix for -both revisions; all 96 invocations completed successfully. Raw logs are in -`/tmp/pypcre-bench-20260822/logs/runs2` on the benchmark host. - -#### Fan-out and call-local speedups +#### Fan-out and API speedups Both interpreter runs used the same scheduler policy. The host reports 12 performance logical CPUs and 4 efficiency logical CPUs; macOS does not provide @@ -169,24 +119,8 @@ topology rather than claiming hard CPU affinity. | Repeated integer-flagged `compile("x", CASELESS)` | **1.16 μs** | **0.81 μs** | The parallel figures are serial-to-parallel speedups and preserve input order -and exception behavior. Large `findall` scans release the GIL only around the -PCRE2 call; match data, context, and subject ownership remain worker-local. -Reproduce the fan-out benchmark with the same scheduler policy for both -interpreters: - -```bash -taskpolicy -t 1 -l 1 env \ - PYPCRE_PARALLEL_WORKERS=12 \ - PYPCRE_PARALLEL_RUNS=9 \ - PYTHONPATH=. \ - ./.venv310/bin/python benchmarks/parallel_map_hotpath.py - -taskpolicy -t 1 -l 1 env \ - PYPCRE_PARALLEL_WORKERS=12 \ - PYPCRE_PARALLEL_RUNS=9 \ - PYTHONPATH=. \ - ./.venv-gil0/bin/python benchmarks/parallel_map_hotpath.py -``` +and exception behavior. They were measured with the same scheduler policy for +both interpreters on the Apple host. #### `findall` — large multiline and lookaround workloads