Skip to content

Optimize CEL PrattParser and Lexer performance - #1216

Merged
copybara-service[bot] merged 1 commit into
mainfrom
test_979511160
Sep 12, 2026
Merged

Optimize CEL PrattParser and Lexer performance#1216
copybara-service[bot] merged 1 commit into
mainfrom
test_979511160

Conversation

@copybara-service

@copybara-service copybara-service Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Optimize CEL PrattParser and Lexer performance

Improve parsing throughput and reduce memory allocations across CEL expressions.

Measured with CelParserBenchmark (parseOnly, built -c opt), comparing three
parsers back to back in one session: ANTLR, the Pratt parser before this change,
and the Pratt parser after it.

Objects allocated per parse:

Case ANTLR Pratt before Pratt after Pratt vs ANTLR Delta this CL
SMOKE_TEST 357 131 123 2.9x smaller -6.1%
CHAINED_ORS 968 374 350 2.8x smaller -6.4%
LIST_COMPREHENSION 512 218 166 3.1x smaller -23.9%
MESSAGE_CREATION 1,253 502 427 2.9x smaller -14.9%
LONG_LIST 81,794 19,310 19,271 4.2x smaller -0.2%

Bytes allocated per parse:

Case ANTLR Pratt before Pratt after Pratt vs ANTLR Delta this CL
SMOKE_TEST 12,256 3,776 3,608 3.4x smaller -4.4%
CHAINED_ORS 32,160 10,288 9,864 3.3x smaller -4.1%
LIST_COMPREHENSION 17,320 6,120 4,736 3.7x smaller -22.6%
MESSAGE_CREATION 43,128 13,888 12,240 3.5x smaller -11.9%
LONG_LIST 2,907,488 553,160 568,080 5.1x smaller +2.7%

Wall clock, mean of 3 caliper trial medians:

Case ANTLR Pratt before Pratt after Pratt vs ANTLR Delta this CL
SMOKE_TEST 4,940 ns 733 ns 692 ns 7.1x faster -5.7%
CHAINED_ORS 14,641 ns 2,090 ns 2,041 ns 7.2x faster -2.3%
LIST_COMPREHENSION 7,514 ns 1,640 ns 1,218 ns 6.2x faster -25.7%
MESSAGE_CREATION 20,979 ns 3,962 ns 3,487 ns 6.0x faster -12.0%
LONG_LIST 1,616,631 ns 144,980 ns 142,583 ns 11.3x faster -1.7%

LONG_LIST is the one case that allocates slightly more than before. It is an
extreme outlier (1,000 list elements, ~20k objects per parse) and the +2.7% comes
from letting the positions map grow from its default capacity instead of
presizing it; presizing cost more on every other case, so the tradeoff is worth
it. The map is removed entirely later in this series.

@copybara-service
copybara-service Bot force-pushed the test_979511160 branch 3 times, most recently from 1134ca9 to 0354ca5 Compare September 12, 2026 01:08
Improve parsing throughput and reduce memory allocations across CEL expressions.

Measured with CelParserBenchmark (parseOnly, built -c opt), comparing three
parsers back to back in one session: ANTLR, the Pratt parser before this change,
and the Pratt parser after it.

Objects allocated per parse:

| Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL |
| :--- | ---: | ---: | ---: | ---: | ---: |
| SMOKE_TEST | 357 | 131 | 123 | 2.9x smaller | -6.1% |
| CHAINED_ORS | 968 | 374 | 350 | 2.8x smaller | -6.4% |
| LIST_COMPREHENSION | 512 | 218 | 166 | 3.1x smaller | -23.9% |
| MESSAGE_CREATION | 1,253 | 502 | 427 | 2.9x smaller | -14.9% |
| LONG_LIST | 81,794 | 19,310 | 19,271 | 4.2x smaller | -0.2% |

Bytes allocated per parse:

| Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL |
| :--- | ---: | ---: | ---: | ---: | ---: |
| SMOKE_TEST | 12,256 | 3,776 | 3,608 | 3.4x smaller | -4.4% |
| CHAINED_ORS | 32,160 | 10,288 | 9,864 | 3.3x smaller | -4.1% |
| LIST_COMPREHENSION | 17,320 | 6,120 | 4,736 | 3.7x smaller | -22.6% |
| MESSAGE_CREATION | 43,128 | 13,888 | 12,240 | 3.5x smaller | -11.9% |
| LONG_LIST | 2,907,488 | 553,160 | 568,080 | 5.1x smaller | +2.7% |

Wall clock, mean of 3 caliper trial medians:

| Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL |
| :--- | ---: | ---: | ---: | ---: | ---: |
| SMOKE_TEST | 4,940 ns | 733 ns | 692 ns | 7.1x faster | -5.7% |
| CHAINED_ORS | 14,641 ns | 2,090 ns | 2,041 ns | 7.2x faster | -2.3% |
| LIST_COMPREHENSION | 7,514 ns | 1,640 ns | 1,218 ns | 6.2x faster | -25.7% |
| MESSAGE_CREATION | 20,979 ns | 3,962 ns | 3,487 ns | 6.0x faster | -12.0% |
| LONG_LIST | 1,616,631 ns | 144,980 ns | 142,583 ns | 11.3x faster | -1.7% |

LONG_LIST is the one case that allocates slightly more than before. It is an
extreme outlier (1,000 list elements, ~20k objects per parse) and the +2.7% comes
from letting the positions map grow from its default capacity instead of
presizing it; presizing cost more on every other case, so the tradeoff is worth
it. The map is removed entirely later in this series.

PiperOrigin-RevId: 980131070
@copybara-service
copybara-service Bot merged commit 33da350 into main Sep 12, 2026
2 checks passed
@copybara-service
copybara-service Bot deleted the test_979511160 branch September 12, 2026 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant