Add StateManager benchmarks for get/set/modify operations - #7161
Merged
Merged
Conversation
There were no benchmarks over StateManager.get_state, set_state or modify_state, so regressions in token formatting, state tree traversal, (de)serialization and lock bookkeeping went unnoticed. Add tests/benchmarks/test_state_manager.py covering those three operations across the memory, disk and redis managers, plus redis with opportunistic locking enabled, and a fourth case for reads that have to rebuild the tree rather than hit a process-local copy. The point is to measure Reflex's own work, not the runner's hardware, so each manager is wired to an in-process store: the redis manager talks to the existing mock_redis fake, and the disk manager writes to tmpfs with debouncing off so set_state still serializes inline. Lock, lease and token expirations are pinned far beyond any run, because the instrumented runner is much slower than wall clock and an expiry mid-run would silently change which code path is measured. Two things this turned up while being written, both handled here: - Seeding via modify_state alone left the opportunistic locking manager holding the state in its lease cache, so get_state read an empty store and measured fresh instance creation instead of deserialization. The fixture now seeds through set_state and asserts the substate round trips before establishing the lease. - A single loop.run_until_complete costs a few microseconds, which was over 80% of an apparent cached get_state. Each measured call now batches ITERATIONS operations per loop entry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJ1phPZa6tv6bEHeU7pjKq
Contributor
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Merging this PR will not alter performance
Performance Changes
Comparing 
Footnotes
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 1 file
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…plock get/set rows The seeding guard read back the process-local cache for the disk manager, so it never proved the state actually reached the store; evict the cache first so the verification read deserializes from disk. Also stop parametrizing get_state/set_state over the oplock variant, which only diverges inside modify_state.
…failure The state_manager fixture seeds and verifies the store before yielding; a failure in that setup (notably the round-trip guard asserts, which are meant to fire on store regressions) skipped manager.close() and left the mkdtemp directory behind on RAM-backed /dev/shm. Wrap setup in try/finally so close and rmtree run regardless. Claude-Session: https://claude.ai/code/session_01K6k7qph4XYh2RXaKntT5J4
FarhanAliRaza
approved these changes
Sep 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type of change
Description
This PR adds comprehensive benchmarks for the
StateManagerget/set/modify code paths across all manager implementations (memory, disk, and redis with/without opportunistic locking).The benchmarks measure the actual work Reflex performs—formatting tokens, walking the state tree, (de)serializing states, and bookkeeping locks—rather than external I/O performance. Each manager is wired to an in-process backing store:
StateManagerMemory: No external storeStateManagerDisk: Writes to tmpfs with debouncing disabled for inline serializationStateManagerRedis: Uses the in-processmock_redisfake for command executionThe benchmarks include:
test_get_state: Steady-state reads across all managerstest_get_state_uncached: Reads that require rebuilding/reloading the state tree (memory and disk only)test_set_state: Dirty-and-write cycles across all managerstest_modify_state: Lock-acquire, dirty, release cycles across all managersEach measured call batches
ITERATIONS=50operations to amortize event loop entry overhead and surface real regressions. Lock/lease expirations are pinned far beyond any run to prevent mid-measurement expiration from changing code paths.Test Plan
The benchmarks are designed to run with
pytest-codspeedand integrate with the existing test infrastructure. They use parametrized fixtures to test all manager implementations and include proper setup/teardown (state seeding, directory cleanup, manager closure).https://claude.ai/code/session_01QJ1phPZa6tv6bEHeU7pjKq