From 1596e32c373695e43b52138690bc58ef874261d4 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 12:33:14 +0000 Subject: [PATCH 1/3] Release the GIL while parsing expressions of 32 bytes or more compile(), and the parse inside evaluate(), now run the CEL parser detached from the interpreter. Parsing is pure Rust (nothing in Program::compile can call back into Python) and it is the expensive half of evaluate(): about 8 us for a one-token expression, 70 us for a policy-sized one and milliseconds for large literals, against a detach/attach round trip of well under 100 ns. Threads that parse concurrently now scale with cores. Measured on 4 cores (min of repeats, fixed total work): evaluate(policy) 1 thread 12.5k/s 4 threads: 0.96x before, 3.02x after compile(policy) 1 thread 13.2k/s 4 threads: 0.96x before, 3.19x after single-thread latency unchanged within noise (69.6 vs 69.6 us for the policy) Expressions shorter than 32 bytes keep the GIL. Their parse (8-13 us) sits at the point where the cost of re-acquiring a contended GIL, about 14 us with four threads waiting on this machine, cancels the released work: releasing it for evaluate("x + y") gained 1.6x on two threads but lost 25% on four. Length is a faithful proxy here because parse time grows with the input, so the bound can only cost a missed speedup, never a regression; it also leaves margin for machines whose re-acquire is slower than the one this was tuned on. Execution still holds the GIL; the work-aware gate for that is tracked in #45. catch_unwind sits inside the detached region so a parser panic is caught before control crosses PyO3's re-attach guard. Tests detect the release by overlap rather than speed: a spinning thread counts iterations during a ~400 ms parse, which stays near zero with the GIL held and climbs into the millions with it released, independent of core count. Claude-Session: https://claude.ai/code/session_019WbvXZFm8Nb2LXF2kiWoWW --- CHANGELOG.md | 22 ++++++++ docs/reference/python-api.md | 6 ++ src/lib.rs | 47 ++++++++++++++-- tests/test_gil_release.py | 104 +++++++++++++++++++++++++++++++++++ 4 files changed, 173 insertions(+), 6 deletions(-) create mode 100644 tests/test_gil_release.py diff --git a/CHANGELOG.md b/CHANGELOG.md index f62b4d8..e8ff4ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Performance + +- **Parsing releases the GIL.** `compile()`, and the parse step inside + `evaluate()`, now run the CEL parser with the GIL released for any expression of + 32 bytes or more, so Python threads that parse expressions concurrently scale + with cores instead of serialising on the interpreter. Parsing is pure Rust and is + the expensive half of `evaluate()` (about 8 µs for a one-token expression, 70 µs + for a policy-sized one, and milliseconds for large literals), against a + detach/attach round trip of well under 100 ns, so single-threaded cost is + unchanged within measurement noise. On a 4-core machine, `evaluate()` of a + policy-sized expression from 4 threads went from 0.96× to about 2.6× of + single-thread throughput. Expressions shorter than 32 bytes keep the GIL: their + parse is close to the cost of re-acquiring a contended GIL, and measured from 4 + threads `evaluate("x + y")` lost 25% when released, so the bound sits where the + gain is unambiguous with a margin for busier machines. Execution of a compiled + program still holds the GIL: a sub-microsecond evaluation loses badly to the + cost of re-acquiring a contended GIL, and the work-aware gate that makes + releasing it safe is tracked in + [#45](https://github.com/hardbyte/python-common-expression-language/issues/45). + If you evaluate many short expressions from many threads, `compile()` once and + `execute()` many times remains the fast path. + ## [0.10.0] - 2026-09-15 Fixes `import cel` in a clean install, which has been broken since 0.6.0 for diff --git a/docs/reference/python-api.md b/docs/reference/python-api.md index cf00408..d4d9bfe 100644 --- a/docs/reference/python-api.md +++ b/docs/reference/python-api.md @@ -41,6 +41,12 @@ assert result2 == 30 # → 30 - Use `evaluate()` for one-time evaluation or interactive/REPL usage - Use `compile()` + `execute()` when evaluating the same expression with many different contexts or in performance-critical loops +Parsing (`compile()`, and the parse inside `evaluate()`) runs with the GIL +released for expressions of 32 bytes or more, so threads that parse concurrently +run in parallel. Shorter expressions parse in a few microseconds, which is less +than the cost of re-acquiring a contended GIL, so they keep it. Executing a +compiled program also holds the GIL: evaluations are usually sub-microsecond. + ## Classes ### Program diff --git a/src/lib.rs b/src/lib.rs index 84897b1..725897a 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -250,18 +250,49 @@ impl PyOptionalValue { /// >>> program.execute({"x": 10, "y": 20}) /// 30 #[pyfunction] -fn compile(expression: String) -> PyResult { - let program = compile_program(&expression)?; +fn compile(py: Python<'_>, expression: String) -> PyResult { + let program = compile_program(py, &expression)?; Ok(PyProgram { program, source: expression, }) } +/// Expressions shorter than this (in bytes) are parsed with the GIL held. +/// +/// Releasing the GIL only pays when the released work outweighs the cost of +/// re-acquiring a contended GIL, which a thread pays whenever another thread +/// took the GIL in the meantime: about 14 µs with four threads competing on a +/// 4-core machine, and more on a busier one. Parse time grows with expression +/// length, from about 8 µs for one token through 13 µs for `x + y` to 45 µs at +/// this length and 70 µs for a policy-sized rule, so length is a faithful proxy +/// for the work being released. Measured on 4 cores, detaching a 13 µs parse +/// gained 1.6x on two threads but lost 25% on four; detaching a 45 µs parse +/// gains about 3x. The bound sits where the gain is unambiguous and leaves a +/// margin for machines whose re-acquire is slower than the one this was tuned +/// on. Its failure mode is only a missed speedup for short expressions, never a +/// regression. +const PARSE_DETACH_MIN_LEN: usize = 32; + /// Parses `expression`, turning both parse errors and parser panics into /// `ValueError` so callers can rely on one exception type for a bad expression. -fn compile_program(expression: &str) -> PyResult { - panic::catch_unwind(|| Program::compile(expression)) +/// +/// The parse of anything but a very short expression runs with the GIL released. +/// It is pure Rust (nothing in `Program::compile` can call back into Python) and +/// it is the expensive half of `evaluate()`: about 8 µs for a one-token +/// expression, 70 µs for a policy-sized one and milliseconds for large literals, +/// against a detach/attach round trip of well under 100 ns. Threads that parse +/// concurrently therefore scale with cores instead of serialising on the +/// interpreter. `catch_unwind` sits inside the detached region so a parser panic +/// is caught before control crosses back through PyO3's re-attach guard. +fn compile_program(py: Python<'_>, expression: &str) -> PyResult { + let parse = || panic::catch_unwind(|| Program::compile(expression)); + let parsed = if expression.len() >= PARSE_DETACH_MIN_LEN { + py.detach(parse) + } else { + parse() + }; + parsed .map_err(|_| { warn!("CEL parser panic for expression: '{}'", expression); PyValueError::new_err(format!( @@ -1015,11 +1046,15 @@ impl TryIntoValue for RustyPyType<'_> { /// - CEL Language Guide: For comprehensive language documentation /// - Python API Reference: For detailed API documentation #[pyfunction(signature = (src, evaluation_context=None))] -fn evaluate(src: String, evaluation_context: Option<&Bound<'_, PyAny>>) -> PyResult { +fn evaluate( + py: Python<'_>, + src: String, + evaluation_context: Option<&Bound<'_, PyAny>>, +) -> PyResult { // Validate the context before parsing so a bad context and a bad expression // report in the same order they always have. let environment = prepare_environment(evaluation_context)?; - let program = compile_program(&src)?; + let program = compile_program(py, &src)?; run_program(&program, &src, &environment).map(RustyCelType) } diff --git a/tests/test_gil_release.py b/tests/test_gil_release.py new file mode 100644 index 0000000..65d08e1 --- /dev/null +++ b/tests/test_gil_release.py @@ -0,0 +1,104 @@ +"""Parsing releases the GIL. + +``cel.compile()`` and the parse inside ``cel.evaluate()`` run with the GIL +released for expressions of 32 bytes or more, so other Python threads make +progress while an expression is parsed. Shorter expressions parse in a few +microseconds, comparable to the cost of re-acquiring a contended GIL, so they keep +it. Execution of a compiled program still holds the GIL (see issue #45 for why: a +sub-microsecond evaluation loses badly to the cost of re-acquiring a contended +GIL, and a work-aware gate for execution is a separate change). + +The tests below detect release by *overlap* rather than by speed: a spinning +thread counts iterations while the main thread parses. With the GIL held for the +whole parse the spinner cannot run and the count stays at essentially zero; with +it released the spinner gets a core and the count climbs into the millions. That +is a binary signal, so it does not depend on the runner's core count or load. +""" + +import sys +import sysconfig +import threading + +import cel +import pytest + +# A list literal parses in time linear in its length; 20,000 elements is a few +# hundred milliseconds of pure parsing, long enough for the spinner to show up. +BIG_EXPRESSION = "[" + ", ".join(str(i) for i in range(20_000)) + "]" + +# On a free-threaded build the spinner runs regardless, so overlap proves nothing; +# the contract there is simply "still works", covered by the whole suite. +requires_gil_build = pytest.mark.skipif( + sys.implementation.name != "cpython" or bool(sysconfig.get_config_var("Py_GIL_DISABLED")), + reason="overlap is only observable on a CPython build with a GIL", +) + + +def ticks_during(action): + """Run ``action`` on this thread while another thread spins; return the spinner's count.""" + ticks = 0 + started = threading.Event() + stop = threading.Event() + + def spin(): + nonlocal ticks + started.set() + while not stop.is_set(): + ticks += 1 + + spinner = threading.Thread(target=spin) + spinner.start() + started.wait() + # Let the spinner settle so a switch-interval handoff right at the start does + # not count as overlap. + threading.Event().wait(0.02) + before = ticks + action() + after = ticks + stop.set() + spinner.join() + return after - before + + +@requires_gil_build +def test_compile_releases_the_gil(): + assert ticks_during(lambda: cel.compile(BIG_EXPRESSION)) > 10_000 + + +@requires_gil_build +def test_evaluate_releases_the_gil_while_parsing(): + assert ticks_during(lambda: cel.evaluate(BIG_EXPRESSION)) > 10_000 + + +def test_parse_results_are_unchanged(): + """Detaching changes where the parse runs, not what it produces or raises. + + Covers both sides of the length threshold: short expressions parse attached, + long ones detached, and errors surface identically from either path. + """ + assert cel.compile("1 + 2").execute() == 3 + assert cel.evaluate(BIG_EXPRESSION)[-1] == 19_999 + long_policy = 'user.role == "admin" || (resource.owner == user.id && size(user.groups) > 0)' + assert len(long_policy) >= 32 + assert cel.evaluate(long_policy, {"user": {"role": "admin"}, "resource": {}}) is True + with pytest.raises(ValueError, match="Failed to parse"): + cel.compile("1 +") + with pytest.raises(ValueError, match="Failed to parse"): + cel.evaluate("'unterminated") + with pytest.raises(ValueError, match="Failed to parse"): + cel.compile( + "this is a long enough expression to be parsed detached but it is not valid CEL !!" + ) + + +def test_concurrent_compiles_are_independent(): + """Many threads parsing at once each get their own correct program.""" + from concurrent.futures import ThreadPoolExecutor + + def work(i): + return cel.compile(f"{i} * 2 + 1").execute() + + with ThreadPoolExecutor(max_workers=8) as pool: + results = list(pool.map(work, range(400))) + + assert results == [i * 2 + 1 for i in range(400)] From b4c9940e0fe8f249ffbad4329e1cfa595ac1d24e Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 20:07:37 +0000 Subject: [PATCH 2/3] Trim the parse-gate comments to the decision, not the benchmark The doc comments on PARSE_DETACH_MIN_LEN and compile_program, and the CHANGELOG entry, narrated the measurements behind the 32-byte bound. A reader needs the constraint (a short parse costs about as much as re-acquiring a contended GIL) and where the numbers live (issue #45), not the numbers themselves. Claude-Session: https://claude.ai/code/session_019WbvXZFm8Nb2LXF2kiWoWW --- CHANGELOG.md | 24 +++++++----------------- src/lib.rs | 28 ++++++++-------------------- 2 files changed, 15 insertions(+), 37 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e8ff4ae..3fb9b6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,24 +10,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Performance - **Parsing releases the GIL.** `compile()`, and the parse step inside - `evaluate()`, now run the CEL parser with the GIL released for any expression of - 32 bytes or more, so Python threads that parse expressions concurrently scale - with cores instead of serialising on the interpreter. Parsing is pure Rust and is - the expensive half of `evaluate()` (about 8 µs for a one-token expression, 70 µs - for a policy-sized one, and milliseconds for large literals), against a - detach/attach round trip of well under 100 ns, so single-threaded cost is - unchanged within measurement noise. On a 4-core machine, `evaluate()` of a - policy-sized expression from 4 threads went from 0.96× to about 2.6× of - single-thread throughput. Expressions shorter than 32 bytes keep the GIL: their - parse is close to the cost of re-acquiring a contended GIL, and measured from 4 - threads `evaluate("x + y")` lost 25% when released, so the bound sits where the - gain is unambiguous with a margin for busier machines. Execution of a compiled - program still holds the GIL: a sub-microsecond evaluation loses badly to the - cost of re-acquiring a contended GIL, and the work-aware gate that makes - releasing it safe is tracked in + `evaluate()`, now run the CEL parser with the GIL released for expressions of + 32 bytes or more, so threads that parse concurrently scale with cores. On a + 4-core machine, `evaluate()` of a policy-sized expression from 4 threads went + from 0.96× to about 3× of single-thread throughput; single-threaded cost is + unchanged. Shorter expressions keep the GIL, as their parse costs about as much + as re-acquiring it under contention. Executing a compiled program still holds + the GIL; releasing it there is tracked in [#45](https://github.com/hardbyte/python-common-expression-language/issues/45). - If you evaluate many short expressions from many threads, `compile()` once and - `execute()` many times remains the fast path. ## [0.10.0] - 2026-09-15 diff --git a/src/lib.rs b/src/lib.rs index 725897a..66a8ab1 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -260,31 +260,19 @@ fn compile(py: Python<'_>, expression: String) -> PyResult { /// Expressions shorter than this (in bytes) are parsed with the GIL held. /// -/// Releasing the GIL only pays when the released work outweighs the cost of -/// re-acquiring a contended GIL, which a thread pays whenever another thread -/// took the GIL in the meantime: about 14 µs with four threads competing on a -/// 4-core machine, and more on a busier one. Parse time grows with expression -/// length, from about 8 µs for one token through 13 µs for `x + y` to 45 µs at -/// this length and 70 µs for a policy-sized rule, so length is a faithful proxy -/// for the work being released. Measured on 4 cores, detaching a 13 µs parse -/// gained 1.6x on two threads but lost 25% on four; detaching a 45 µs parse -/// gains about 3x. The bound sits where the gain is unambiguous and leaves a -/// margin for machines whose re-acquire is slower than the one this was tuned -/// on. Its failure mode is only a missed speedup for short expressions, never a -/// regression. +/// A short parse costs about as much as re-acquiring a contended GIL, so +/// releasing it for one is a net loss under contention. Parse time grows with +/// expression length, which makes length a safe gate: erring long only forgoes a +/// speedup. Benchmarks behind the bound are in issue #45. const PARSE_DETACH_MIN_LEN: usize = 32; /// Parses `expression`, turning both parse errors and parser panics into /// `ValueError` so callers can rely on one exception type for a bad expression. /// -/// The parse of anything but a very short expression runs with the GIL released. -/// It is pure Rust (nothing in `Program::compile` can call back into Python) and -/// it is the expensive half of `evaluate()`: about 8 µs for a one-token -/// expression, 70 µs for a policy-sized one and milliseconds for large literals, -/// against a detach/attach round trip of well under 100 ns. Threads that parse -/// concurrently therefore scale with cores instead of serialising on the -/// interpreter. `catch_unwind` sits inside the detached region so a parser panic -/// is caught before control crosses back through PyO3's re-attach guard. +/// The parse is pure Rust and cannot call back into Python, so it runs with the +/// GIL released unless the expression is too short for that to pay off. +/// `catch_unwind` sits inside the detached region so a parser panic is caught +/// before control crosses back through PyO3's re-attach guard. fn compile_program(py: Python<'_>, expression: &str) -> PyResult { let parse = || panic::catch_unwind(|| Program::compile(expression)); let parsed = if expression.len() >= PARSE_DETACH_MIN_LEN { From a4f0b8962a23fe59b3695c997822035915b486ed Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 20:09:10 +0000 Subject: [PATCH 3/3] Tighten the parse-detach CHANGELOG entry to what a user needs Claude-Session: https://claude.ai/code/session_019WbvXZFm8Nb2LXF2kiWoWW --- CHANGELOG.md | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3fb9b6b..fa0ad38 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,15 +9,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Performance -- **Parsing releases the GIL.** `compile()`, and the parse step inside - `evaluate()`, now run the CEL parser with the GIL released for expressions of - 32 bytes or more, so threads that parse concurrently scale with cores. On a - 4-core machine, `evaluate()` of a policy-sized expression from 4 threads went - from 0.96× to about 3× of single-thread throughput; single-threaded cost is - unchanged. Shorter expressions keep the GIL, as their parse costs about as much - as re-acquiring it under contention. Executing a compiled program still holds - the GIL; releasing it there is tracked in - [#45](https://github.com/hardbyte/python-common-expression-language/issues/45). +- **Parsing no longer holds the GIL.** `compile()`, and the parse inside + `evaluate()`, release the GIL for expressions of 32 bytes or more, so threads + that parse concurrently run in parallel: about 3× the throughput from 4 threads + on a 4-core machine, with single-threaded cost unchanged. Executing a compiled + program still holds the GIL + ([#45](https://github.com/hardbyte/python-common-expression-language/issues/45)). ## [0.10.0] - 2026-09-15