Skip to content

Make CLI startup lazy and benchmark command paths - #7050

Merged
masenf merged 4 commits into
mainfrom
codex/cli-startup-performance
Sep 8, 2026
Merged

Make CLI startup lazy and benchmark command paths#7050
masenf merged 4 commits into
mainfrom
codex/cli-startup-performance

Conversation

@Alek99

@Alek99 Alek99 commented Sep 5, 2026

Copy link
Copy Markdown
Member

What changed

  • Register component, cloud, and deploy through lazy Click command proxies so root CLI discovery does not import their implementations.
  • Lazily expose Reflex Base plugin/compiler and SQLAlchemy-aware type helpers while preserving the existing public import surfaces, EAGER_IMPORT behavior, Click metadata, and missing-package help.
  • Narrow the frontend dependency cache key to inputs that actually affect package installation.
  • Add per-package version-check TTLs, with a shorter retry cooldown after failed checks.
  • Add fresh-process CLI benchmarks and update CodSpeed simulation to follow fork/exec subprocesses on the existing Run benchmarks job.

Measured impact

Controlled fresh-process measurements on the same machine:

Metric main This branch Change
reflex --version p50 460.8 ms 66.2 ms 85.6% faster (7.0x)
Root CLI max RSS 147–154 MB 35.1 MB about 76% lower

Current p50s are 66.2 ms for root help, 66.1 ms for run --help, 67.6 ms for component --help, 78.0 ms for cloud --help, and 68.1 ms for deploy --help.

Validation

  • uv run pytest -q tests/units — 8,329 passed, 18 skipped
  • uv run pytest -q tests/benchmarks/test_cli.py — 8 passed
  • uv run ruff check .
  • uv run ruff format --check .
  • uv run pyright reflex tests
  • Compatibility coverage for lazy command metadata/invocation, missing hosting CLI guidance, plugin exports and eager mode, hybrid properties, version-check caching, and frontend cache invalidation.

Related work

#7049 targets backend/dev-process import overhead. This PR is scoped to CLI command discovery and cold-start latency; the two changes overlap in type/prerequisite tests and may need rebase ordering if both land.

Checklist

  • Followed CONTRIBUTING.md.
  • Checked existing open pull requests and documented the related overlap above.
  • Added tests and performance benchmarks for core changes.
  • Ran the unit suite, lint, formatting, and type checks locally.
  • Added news fragments for user-visible performance changes.

@Alek99
Alek99 requested a review from a team as a code owner September 5, 2026 23:51
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 17 files

Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR reduces cold CLI startup work by lazily resolving selected commands, plugin exports, and optional SQLAlchemy helpers. It also narrows frontend dependency cache inputs, adds per-package version-check cooldowns, and introduces fresh-process CLI benchmarks under CodSpeed simulation.

  • Registers component, cloud, and deploy through lazy Click proxies while preserving command metadata and missing-package guidance.
  • Defers plugin/compiler and optional SQLAlchemy imports while retaining compatibility exports.
  • Keys frontend installation caching on resolved package inputs and lockfile behavior rather than unrelated backend configuration.
  • Caches successful version checks for one day and failed attempts for one hour.
  • Runs fresh-process CLI benchmarks with subprocess tracking and a workflow-configured timeout.

Confidence Score: 4/5

The code changes appear functionally safe, but the outstanding repository requirement for human-readable comments on the version-check durations must be satisfied before merging.

No new actionable defect was found in the changes since the previous review. The persisted-key finding was manually resolved and is fixed by named constants, while the existing duration-comment finding remains unresolved because the one-day and one-hour timedelta constants still lack the comments required by the repository rule.

Files Needing Attention: reflex/utils/prerequisites.py

Important Files Changed

Filename Overview
reflex/reflex.py Adds lazy Click command proxies and defers component and hosting CLI imports until command use.
packages/reflex-base/src/reflex_base/plugins/init.py Converts plugin package exports to lazy module and attribute resolution while preserving the public surface.
packages/reflex-base/src/reflex_base/utils/types.py Lazily resolves optional SQLAlchemy property classes and retains wildcard-import compatibility.
reflex/utils/types.py Re-exports reflex-base type utilities without eagerly resolving the optional property-class tuple.
reflex/utils/js_runtimes.py Narrows the frontend package installation cache key to dependencies and installation-affecting configuration.
reflex/utils/prerequisites.py Adds per-package success and failure TTLs; the previously reported duration-comment rule violation remains outstanding.
tests/benchmarks/test_cli.py Adds fresh-process CLI benchmarks with a configurable subprocess timeout.
.github/workflows/performance.yml Switches CodSpeed to subprocess-aware simulation and supplies the benchmark timeout through the job environment.

Reviews (4): Last reviewed commit: "ci: configure CLI benchmark timeout expl..." | Re-trigger Greptile

Comment thread reflex/utils/prerequisites.py Outdated
Comment thread reflex/utils/prerequisites.py
@codspeed-hq

codspeed-hq Bot commented Sep 6, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 32 untouched benchmarks
⏩ 8 skipped benchmarks1


Comparing codex/cli-startup-performance (44661b1) with main (c49a85d)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@Alek99 Alek99 changed the title Make CLI startup lazy and add macro benchmarks Make CLI startup lazy and benchmark command paths Sep 6, 2026

@FarhanAliRaza FarhanAliRaza left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will need to set reflex-base floor on release.

@masenf

masenf commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Will need to set reflex-base floor on release.

At least for now, all reflex packages are hard pinned to the matching reflex-base, so that shouldn't be an issue.

That said, it seems like the reflex and reflex-base changes are not actually interdependent...unless i'm missing something

@masenf
masenf merged commit 77cfe61 into main Sep 8, 2026
111 checks passed
@masenf
masenf deleted the codex/cli-startup-performance branch September 8, 2026 20:39
masenf pushed a commit that referenced this pull request Sep 11, 2026
…an refresh

Verifies two process-level changelog claims against the previous stable. The
shared-asset fix (#7039) is confirmed on its deterministic half: a stale
symlink pointing at the wrong file is repointed on 0.9.11a1 and silently left
alone on 0.9.10.post2, so the old release served the wrong asset. The
concurrent-first-create half could not be provoked on this filesystem on either
version, which is recorded rather than reported. CLI startup (#7050) is ~2.3x
faster across every subcommand.

The release plan now carries FINDING-035/036, the verifier confirmation of
FINDING-033, and the completed enterprise sweep.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0167TFRX1CXqwGK8uh7DBE7c
masenf pushed a commit that referenced this pull request Sep 11, 2026
…nded late

Second passes over config_assets_cli, ent_mantine_highcharts_tickets and
memo_hash finished after the campaign was closed out. The important result is a
correction: FINDING-036 is re-rated from LOW to HIGH. The two console errors the
first pass called harmless set backend_state_mismatch, a one-way latch that
discards every user event, so the enterprise tickets demo's UI never dispatches
anything in dev or prod on either reflex version — while its REST API works.
Rendering one var from every substate the backend knows about is the workaround.

Adds FINDING-040 to FINDING-049: non-reproducible memo module names across
identical compiles, same-named ComponentState subclasses in different modules
crashing the compile, client_state(global_ref=False) breaking when memoization
splits reader from writer, and smaller CLI, changelog and enterprise items. Two
"verified clean" claims are qualified: #7050's frontend-reinstall half is not
observable and #7039's multi-process scenario still aborts one step past the
fixed asset link.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0167TFRX1CXqwGK8uh7DBE7c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants