Skip to content

docs: remove internal benchmark details #105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Qubitium merged 1 commit into from
Aug 22, 2026
Merged

docs: remove internal benchmark details #105

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 7 additions & 73 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Fast, free-threaded Python bindings for `PCRE2` with a stable `stdlib.re`-compat


## Latest News 🚀
* 08/09–08/10/2026 **API performance and safety update**: bounded call-local fast paths accelerate `findall`, `split`, `sub`/`subn`, `Match.expand`, `lastindex`, flag handling, `template()`, and `escape` by a representative **2x to 38x+**, while large ordered `parallel_map(search)` workloads measured **7.55x to 7.71x** on the Apple arm64 free-threaded comparison runs. Compatibility fallbacks preserve complex patterns, subclasses, buffers, and callables; caches remain thread-scoped and size-bounded, and no fast path retains subjects, results, or extra captured values. Unsafe UTF bytes compilation is also blocked, with differential, randomized, concurrency, subprocess, and memory-safety coverage. 🧵⚡🛡️
* 08/09–08/10/2026 **API performance and safety update**: common regex workloads and ordered `parallel_map(search)` workloads were benchmarked across Python 3.10 and free-threaded Python 3.14t/GIL=0. Compatibility fallbacks preserve complex patterns, subclasses, buffers, and callables. Unsafe UTF bytes compilation is also blocked, with differential, randomized, concurrency, subprocess, and memory-safety coverage. 🧵⚡🛡️
* 08/08/2026 **0.6.0**: `findall`, `finditer`, `sub`/`subn`, `split`, and `match`/`search`/`fullmatch` are now up to **46x faster** than `stdlib.re` and **48x faster** than `regex` on `finditer`/`findall` workloads, **13x** on `split`, and **2–9x** on `sub`/`subn` backref workloads, with full `re` semantics. Free-threaded `findall` reaches **13.8x** vs `re` on 8 threads. 🚀⚡
* 07/27/2026 [0.5.0](https://github.com/ModelCloud/PyPcre/releases/tag/v0.5.0): Zero-copy buffer-protocol subject support (`mmap.mmap`, `bytearray`, `array.array`) with UTF-8 validation and GIL=0-safe memory pinning. 🗂️⚡
* 07/24/2026 [0.4.0](https://github.com/ModelCloud/PyPcre/releases/tag/v0.4.0): C extension hardening (memory/pointer safety, bounds checks, atomic allocator init), GIL=0 safety verified, vectorized UTF-8 index/offset conversion, GIL-release threshold for small calls, C `findall` implementation, and README competitor benchmarks. 🛡️⚡
Expand All @@ -46,10 +46,10 @@ PyPcre pairs Python's familiar `re`-compatible API with the real `PCRE2` engine.

- 🧬 **Full power of PCRE2**: PyPcre uses the real `PCRE2` engine, so you get native compile options, semantics, JIT, and upstream tuning.
- 🔥 **More expressive regex syntax**: `PCRE2` supports constructs beyond stdlib `re`, including atomic groups `(?>...)`, possessive quantifiers `++`, branch-reset groups `(?|...)`, richer lookarounds, and backtracking control verbs like `(*SKIP)(*FAIL)`.
- 🧵 **Thread-safe into `nogil`**: PyPcre is built for `PYTHON_GIL=0`, with CI coverage, lock-aware caches, reusable match/JIT resources, and `parallel_map()` for multi-subject fan-out.
- 🧵 **Thread-safe into `nogil`**: PyPcre is built for `PYTHON_GIL=0`, with CI coverage and `parallel_map()` for multi-subject fan-out.
- ⚡ **Fast on real workloads**: `PCRE2` JIT plus cached compiled patterns lets PyPcre match or beat `re` and `regex` on many common scans, especially multiline searches, lookaround-heavy patterns, and free-threaded execution.
- 🛡️ **Safer operational story**: PyPcre prefers the system `libpcre2-8` shared library so normal OS package updates can bring security and bug-fix benefits without a bundled fork.
- ✅ **Validated thoroughly**: the project runs API tests, fuzz tests, memory-safety checks, local `valgrind` leak checks, and `massif` heap profiles. Recent local profiling found `0` definite leaks and `0` possible leaks in both the public API and raw binding paths.
- ✅ **Validated thoroughly**: the project runs API, fuzz, concurrency, and memory-safety tests.

### Quick Comparison 🥊

Expand All @@ -66,59 +66,9 @@ PyPcre pairs Python's familiar `re`-compatible API with the real `PCRE2` engine.

### Benchmark Highlights 🏁

#### Controlled API hot-path A/B
The tables below summarize representative public workloads. Lower is better.

The current comparison is origin/main `03f4d10c379babcc9208e45705d774cb93bcebb8`
versus the PR head `3f8025eeea203a1f189aa8fffb31bbef236db26a`. It was measured
on an Apple M4 Max with 12 performance and 4 efficiency logical CPUs, macOS
26.6, PCRE2 10.47, Apple clang 21, and `regex==2025.11.3`. Python 3.10.11 is
GIL-enabled; Python 3.14.0rc2 is the
resolved Apple arm64 free-threaded build at
`/Users/diego/.local/share/uv/python/cpython-3.14.0rc2+freethreaded-macos-aarch64-none/bin/python3.14t`
and reports `sys._is_gil_enabled() == False`.

Each API value is the median of 9 repetitions of 10,000 calls. Cold compile is
the median of 9 repetitions of 1,000 unique compiles. Lower is better.

| Workload | 3.10 origin | 3.10 follow-up | Δ | 3.14t origin | 3.14t follow-up | Δ |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Cached `compile(..., re.I|re.M|re.S|re.X)` | 0.652 μs | 0.655 μs | +0.5% | 0.424 μs | 0.427 μs | +0.7% |
| Module `search` | 0.561 μs | 0.570 μs | +1.6% | 0.456 μs | 0.469 μs | +2.9% |
| Cold compile | 6.849 μs | 7.191 μs | +5.0% | 4.747 μs | 5.099 μs | +7.4% |

The earlier lock-heavy PR measurement for `a1946c5f6e1cee2dbac714d6a98e8eac47a19161`
reported cached-compile regressions of +83.6% (3.10) and +65.1% (3.14t), and
module-search regressions of +30.6% and +25.9%. The follow-up removes the
avoidable per-call configuration/thread-state locks and materially reduces those
regressions; it does not claim a speedup over origin.

Reproduce the API comparison with the same scheduler-tier hint for both builds:

```bash
cd /Users/diego/tmp-omni-workspace/pypcre/.worktrees/gil0-fixes
PYPCRE_BENCH_RUNS=10000 PYPCRE_BENCH_REPEATS=9 \
taskpolicy -t 1 -l 1 env PYTHONPATH=. \
./.venv-gil0/bin/python benchmarks/api_hotpaths.py
```

Run the same command from the origin checkout with its Python 3.10.11 or
3.14.0rc2t environment for the A/B side.

`taskpolicy -t 1 -l 1` is not hard P-core affinity on this unprivileged macOS
host; no P-core-only speedup is claimed. The API microbenchmarks are available in
[`benchmarks/api_hotpaths.py`](benchmarks/api_hotpaths.py), and the free-threaded
results should not be generalized to GIL-enabled interpreters.

Measured on the Apple arm64 matrix above with compiled-pattern reuse and JIT enabled. Values are medians of three outer runs; lower is better. The comparison retains every workload row, including parity and slower cases.

A reproducible version of this benchmark lives in [`benchmarks/competitor_bench.py`](benchmarks/competitor_bench.py).
The complete comparison also ran `finditer_bench.py`, `sub_bench.py`,
`split_bench.py`, `free_threaded_bench.py`, `api_hotpaths.py`,
`parallel_map_hotpath.py`, and the opt-in `tests/test_benchmark.py` matrix for
both revisions; all 96 invocations completed successfully. Raw logs are in
`/tmp/pypcre-bench-20260822/logs/runs2` on the benchmark host.

#### Fan-out and call-local speedups
#### Fan-out and API speedups

Both interpreter runs used the same scheduler policy. The host reports 12
performance logical CPUs and 4 efficiency logical CPUs; macOS does not provide
Expand Down Expand Up @@ -169,24 +119,8 @@ topology rather than claiming hard CPU affinity.
| Repeated integer-flagged `compile("x", CASELESS)` | **1.16 μs** | **0.81 μs** |

The parallel figures are serial-to-parallel speedups and preserve input order
and exception behavior. Large `findall` scans release the GIL only around the
PCRE2 call; match data, context, and subject ownership remain worker-local.
Reproduce the fan-out benchmark with the same scheduler policy for both
interpreters:

```bash
taskpolicy -t 1 -l 1 env \
PYPCRE_PARALLEL_WORKERS=12 \
PYPCRE_PARALLEL_RUNS=9 \
PYTHONPATH=. \
./.venv310/bin/python benchmarks/parallel_map_hotpath.py

taskpolicy -t 1 -l 1 env \
PYPCRE_PARALLEL_WORKERS=12 \
PYPCRE_PARALLEL_RUNS=9 \
PYTHONPATH=. \
./.venv-gil0/bin/python benchmarks/parallel_map_hotpath.py
```
and exception behavior. They were measured with the same scheduler policy for
both interpreters on the Apple host.

#### `findall` — large multiline and lookaround workloads

Expand Down