Skip to content

Add holistic table event processing benchmark - #7061

Merged
masenf merged 3 commits into
reflex-dev:mainfrom
FarhanAliRaza:benchmark/holistic-table-events
Sep 16, 2026
Merged

masenf merged 3 commits into
reflex-dev:mainfrom
FarhanAliRaza:benchmark/holistic-table-events

Conversation

@FarhanAliRaza

@FarhanAliRaza FarhanAliRaza commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Add an order-management workload to measure backend event performance beyond counter increments. Six events filter and sort 1,000 dataclass rows, recompute totals, generate deltas, and serialize StateUpdate envelopes through Reflex's JSON encoder.

The benchmark warms state before timing and returns to the same filter and sort direction after each sample. A separate correctness test verifies serialized rows and totals across repeated batches. The README documents how to run it and the measurement boundary: processor lifecycle is included; hydration, network transport, database access, and browser rendering are excluded.

Related to #7056, which includes the original benchmark alongside runtime optimizations. This PR isolates the benchmark and adds repeatable samples, correctness assertions, and run documentation. Encoding is measured through json_dumps; changes confined to Socket.IO's encoder configuration require updating the benchmark's encoder too.

Validation

  • Event tests: 3 passed.
  • CodSpeed mode: 2 benchmarks passed, correctness test deselected.
  • Commit hooks: Ruff formatting/checks, codespell, stub update, Pyright, and ty passed.
  • Full uv run pyright reflex tests: passed.
  • git diff --check: passed.

Repository-wide Ruff checks report unrelated existing issues under .states.bak/ and ignore/. The full unit suite was not run; this change only adds benchmark coverage and documentation.

No package source or public behavior changes; no changelog fragment included.

Review in cubic

@FarhanAliRaza
FarhanAliRaza requested a review from a team as a code owner September 7, 2026 17:56
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@greptile-apps

greptile-apps Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; the restored correctness test addresses the previous coverage concern and no new actionable issues remain.

Summary

Adds repeatable benchmarks for counter and order-table event processing through the in-memory backend pipeline.

  • Introduces a 1,000-row table state with filtering, sorting, computed rows, and totals.
  • Measures serialized StateUpdate envelopes for table events.
  • Restores correctness coverage for serialized rows, sort direction, and totals across repeated batches.
  • Documents benchmark execution and the measurement boundary.

Reviews (3) · Last reviewed commit: "Restore table delta correctness test and..."

Comment thread tests/benchmarks/test_event_processing.py
@codspeed

codspeed Bot commented Sep 7, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 0.55%

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 49 untouched benchmarks
🆕 2 new benchmarks
⏩ 9 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
test_import_reflex 9.7 ms 10.2 ms -4.53%
test_isinstance_container[list_typeddict] 106.9 ms 103.2 ms +3.6%
🆕 test_process_event[counter] N/A 13.9 ms N/A
🆕 test_process_event[table] N/A 692.1 ms N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing FarhanAliRaza:benchmark/holistic-table-events (fca0ca5) with main (1bc8936)

Open in CodSpeed

Footnotes

  1. 9 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread tests/benchmarks/test_event_processing.py Outdated
Comment thread tests/benchmarks/test_event_processing.py

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread tests/benchmarks/test_event_processing.py Outdated
Comment thread tests/benchmarks/test_event_processing.py Outdated
Re-add test_table_event_deltas, dropped in the harness simplification, so
the serialized rows, sort direction, and totals the table benchmark
encodes are verified across repeated batches, sharing one _encode_delta
helper with the timed path.

Replace the counter batch's three increments with two increments and two
decrements: the old batch grew the counter every sample, so the computed
vars derived from it did strictly more work on each invocation. Both
workloads now return state to its starting point between samples.
@masenf
masenf merged commit 24f0dea into reflex-dev:main Sep 16, 2026
121 of 122 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants