Skip to content

test(studio): ratchet React Compiler bail-outs per file - #3773

Draft
miguel-heygen wants to merge 2 commits into
feat/studio-u16-vite8-vitest4from
refactor/studio-c1-compiler-bailout-ratchet
Draft

test(studio): ratchet React Compiler bail-outs per file#3773
miguel-heygen wants to merge 2 commits into
feat/studio-u16-vite8-vitest4from
refactor/studio-c1-compiler-bailout-ratchet

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

Lands unit C1 (bail-out ratchet) of the Studio React Compiler coverage plan. Stacked on the Vite 8 PR (#3620). Baseline: 126 files bail out today (54 components, 72 hooks); the gate fails on any rise and the units that follow lower it to zero.

What

A per-file ratchet that fails the Studio suite when any non-test file under packages/studio/src gains a React Compiler bail-out.

Four files, no component changed:

  • packages/studio/scripts/compiler-bailouts.mjs, the scan. Runs oxc-transform-react over every non-test .ts/.tsx under src, groups what the compiler declines by file and cause.
  • packages/studio/src/styles/compilerBailouts.test.ts, the gate. Same shape as the hex ratchet: per-file counts, a total, a rise fails naming the file, a fall passes and prints the command that banks it.
  • packages/studio/src/styles/compiler-bailouts.json, the committed baseline. 126 files, 126 bail-outs.
  • bun run compiler:bailouts in packages/studio, which prints the grouped report for a human. --json prints the same data as { total, files: { path: [causes] } }.

The baseline is only ever written by COMPILER_BAILOUTS_WRITE=1 bunx vitest run src/styles/compilerBailouts.test.ts. A normal run never regenerates it, and a missing baseline fails loudly naming that flag, so the number in git is always one a person chose to accept.

Why

react({ compiler: true }) is silent about what it declines. A component the compiler skips is emitted exactly as written: it still renders, still passes its tests, still builds, and simply is not memoized. There is no warning in the build log and nothing in a diff to notice. Turning the compiler on without a gate means the covered set can shrink for months and the only symptom is that Studio is slower than it should be.

This is the alarm, and it is a vitest test rather than a lint rule because Test is a required check on main and Lint is not.

How

Three facts about oxc-transform-react 0.149.0, each measured against this version

  1. There is no non-fatal diagnostic channel. outputMode: "lint" at the default panicThreshold returns zero errors even for a component written specifically to violate the rules of React. Escalating panicThreshold to "all_errors" is the only way a skip surfaces. This confirms the earlier finding that logDiagnostics reports nothing; that option does not exist on the type at all in this version.
  2. The escalation reports every diagnostic inside the first function it declines, then aborts the module. Two components each with one bad ref read produce one error, not two. One component with two bad ref reads produces two. So a second declined function in a file is invisible until the first is fixed.
  3. A "use no memo" / "use no forget" directive is never reported. It is honoured before any diagnostic is produced, so with the escalation on, a file whose only problem is the directive looks clean. The scan counts directives from the source text instead. A react-hooks suppression is the opposite case: the compiler already reports it as "React rule suppression prevents optimization", for eslint-disable and oxlint-disable, next-line and file-level, and for both exhaustive-deps and rules-of-hooks. Counting those separately would double count, so the scan does not. An eslint-disable for an unrelated rule is correctly ignored by the compiler.

The count is one per file, not one per diagnostic

Fact 2 means a diagnostic count is not monotone, and a ratchet needs monotone. App.tsx reports 21 ref reads, all from one function; a second declined function further down reports nothing. Fixing that first function would uncover the second and the number could go up while the code got better. A gate that fires on an improvement gets disabled.

So count is one per opt-out the scan can see: one per directive, plus one if the compiler declines the file at all. That is monotone. It catches every file going 0 to 1, which is the transition that matters while the baseline is driven down, and once a file reaches 0 the gate on that file is exact. The hole it accepts, stated plainly: a second bail-out added to a file that already bails does not fire. A true per-function count needs the compiler to expose non-fatal diagnostics; it is not something a cleverer caller can recover.

Scope

Scanned: every .ts and .tsx under packages/studio/src. Excluded: *.test.ts(x), *.stories.tsx, *.d.ts, src/test-setup.ts, and anything outside src. 737 files scanned, 126 bail out.

A file the baseline has never seen has a baseline of zero, so a rename cannot smuggle a bail-out past the gate.

Files that fail to parse are not counted as bail-outs: when the escalated pass errors, the scan re-runs that one file with the compiler off, and throws naming the file if it still errors. Only the files that already errored pay for the second pass.

Reconciling with the count measured when the compiler was turned on

That measurement reported 53 non-test .tsx files. This scan finds 126, and the difference is entirely scope, not disagreement:

  • 54 .tsx files. The same 53, plus components/renders/renderQueueTestHarness.tsx. That file is imported only by four *.test.tsx files but its own name does not match *.test.tsx, so this scan's exclusion rule keeps it. Being a superset is the safe direction for a ratchet.
  • 72 .ts files. Hooks and helpers, which the earlier table did not enumerate. useInspectorGestureTransaction.ts, useDomEditNudge.ts, useCaptionSync.ts and others bail for the same reasons the components do. They are listed in full below, because the follow-up units were scoped from a .tsx-only list and these are not in any of them yet.

Baseline by cause

Cannot access refs during render (70)

  • App.tsx
  • captions/components/CaptionOverlay.tsx
  • captions/hooks/useCaptionSync.ts
  • components/TimelineToolbar.tsx
  • components/editor/BlockParamsPanel.tsx
  • components/editor/DomEditOverlay.tsx
  • components/editor/EaseCurveSection.tsx
  • components/editor/MotionPathOverlay.tsx
  • components/editor/SnapGuideOverlay.tsx
  • components/editor/SourceEditor.tsx
  • components/editor/TopologyLens.tsx
  • components/editor/Transform3DCube.tsx
  • components/editor/propertyPanelColor.tsx
  • components/editor/propertyPanelColorGradingSlider.tsx
  • components/editor/propertyPanelCommitField.tsx
  • components/editor/propertyPanelFlatPrimitives.tsx
  • components/editor/propertyPanelPrimitives.tsx
  • components/editor/propertyPanelSections.tsx
  • components/editor/useColorGradingScopes.ts
  • components/editor/useDomEditNudge.ts
  • components/editor/useFxAudition.ts
  • components/editor/useInspectorGestureTransaction.ts
  • components/editor/useLayerRevealOverride.ts
  • components/editor/useMotionPathData.ts
  • components/feedback/StudioFeedbackCard.tsx
  • components/nle/NLEContext.tsx
  • components/nle/NLEPreview.tsx
  • components/nle/PreviewPane.tsx
  • components/nle/TimelineResizeDivider.tsx
  • components/nle/useCanvasZOrderTimelineMirror.ts
  • components/sidebar/LeftSidebar.tsx
  • components/sidebar/PromptPreviewModal.tsx
  • components/ui/useDialogBehavior.ts
  • contexts/DomEditContext.tsx
  • hooks/useAppHotkeys.ts
  • hooks/useClipboard.ts
  • hooks/useDomEditPreviewSync.ts
  • hooks/useDomSelection.ts
  • hooks/useElementPicker.ts
  • hooks/useExternalFileChangeCoordinator.ts
  • hooks/useFileManager.ts
  • hooks/useGestureRecording.ts
  • hooks/useGsapPropertyDebounce.ts
  • hooks/useGsapScriptCommits.ts
  • hooks/useGsapSelectionHandlers.ts
  • hooks/useLivePlayheadTime.ts
  • hooks/useMusicBeatAnalysis.ts
  • hooks/usePanelLayout.ts
  • hooks/usePersistentEditHistory.ts
  • hooks/usePreviewPersistence.ts
  • hooks/useProjectSignaturePoll.ts
  • hooks/useRazorSplit.ts
  • hooks/useSdkSession.ts
  • hooks/useThumbnailLease.ts
  • hooks/useTimelineEditing.ts
  • player/components/PlayerControls.tsx
  • player/components/Timeline.tsx
  • player/components/TimelineCanvas.tsx
  • player/components/TimelineClipDiamonds.tsx
  • player/components/TimelineGestureOverlay.tsx
  • player/components/TimelineLanes.tsx
  • player/components/useTimelineClipDrag.ts
  • player/components/useTimelineFocusCoordinator.ts
  • player/components/useTimelinePlayhead.ts
  • player/components/useTimelineRangeSelection.ts
  • player/components/useTimelineSelectionLifecycle.ts
  • player/components/useTimelineTrackLayout.ts
  • player/hooks/usePlaybackKeyboard.ts
  • player/hooks/useTimelinePlayer.ts
  • webmcp/useStudioAgentTools.ts

React rule suppression prevents optimization (20)

  • components/editor/GestureTrailOverlay.tsx
  • components/editor/PropertyPanel.tsx
  • components/editor/PropertyPanelFlat.tsx
  • components/editor/propertyPanelAudioFxGroup.tsx
  • components/editor/propertyPanelFxSection.tsx
  • components/editor/useColorGradingController.ts
  • components/editor/useFxCarve.ts
  • components/nle/useCompositionStack.ts
  • components/nle/useTimelineEditCallbacks.ts
  • components/panels/VariablesPanel.tsx
  • contexts/TimelineEditContext.tsx
  • contexts/VariablePromoteContext.tsx
  • hooks/useBlockHandlers.ts
  • hooks/useGsapTweenCache.ts
  • hooks/useMountEffect.ts
  • hooks/useProjectCompositionVariables.ts
  • hooks/useSlideshowTabState.ts
  • hooks/useTimelineSelectionPreviewSync.ts
  • player/components/useTimelineGeometry.ts
  • player/components/useTimelineScrollViewport.ts

(BuildHIR::lowerStatement) Handle TryStatement with a finalizer ('finally') clause (14)

  • components/editor/propertyPanelColorSecondary.tsx
  • components/editor/propertyPanelFill.tsx
  • components/editor/propertyPanelFlatMediaSection.tsx
  • components/editor/propertyPanelFont.tsx
  • components/editor/propertyPanelMediaSection.tsx
  • components/editor/useColorGradingPreviews.ts
  • components/editor/useFxLevelling.ts
  • components/panels/SlideshowPanel.tsx
  • components/storyboard/StoryboardFrameFocus.tsx
  • components/storyboard/useFrameComments.ts
  • hooks/useBlockCatalog.ts
  • hooks/useFrameCapture.ts
  • hooks/useGestureCommit.ts
  • hooks/useLintModal.ts

Existing memoization could not be preserved (9)

  • App.tsx
  • components/editor/useInspectorGestureTransaction.ts
  • components/storyboard/StoryboardLoaded.tsx
  • hooks/useFileManager.ts
  • player/components/useAutomationLaneGestures.ts
  • player/components/useTimelineClipDrag.ts
  • player/components/useTimelinePlayhead.ts
  • player/components/useTimelineRangeSelection.ts
  • player/components/useTimelineStackingSync.ts

(BuildHIR::lowerStatement) Support ThrowStatement inside of try/catch (8)

  • captions/hooks/useCaptionSync.ts
  • components/editor/propertyPanelColorSecondary.tsx
  • components/editor/useColorGradingScopes.ts
  • hooks/timelineAudioGroupCreate.ts
  • hooks/useAnimatedPropertyCommit.ts
  • hooks/useElementLifecycleOps.ts
  • hooks/useFrameCapture.ts
  • hooks/useTimelineDeleteOps.ts

Logical assignment operators (||=, &&=, ??=) are not yet supported (5)

  • components/editor/marqueeCommit.ts
  • components/editor/useFxAudition.ts
  • hooks/useDomEditSession.ts
  • hooks/useRazorSplit.ts
  • player/hooks/useTimelinePlayer.ts

try/finally without catch is not supported by React Compiler (5)

  • components/sidebar/AssetsTab.tsx
  • components/storyboard/StoryboardSourceEditor.tsx
  • hooks/useElementLifecycleOps.ts
  • hooks/useRemoveBackground.ts
  • hooks/useTimelineGroupEditing.ts

This value cannot be modified (3)

  • components/editor/DomEditOverlay.tsx
  • hooks/useConsoleErrorCapture.ts
  • player/components/useTimelineRangeSelection.ts

Cannot modify local variables after render completes (3)

  • components/editor/DomEditOverlay.tsx
  • hooks/useConsoleErrorCapture.ts
  • player/components/useTimelineRangeSelection.ts

(BuildHIR::node.lowerReorderableExpression) Expression type MemberExpression cannot be safely reordered (3)

  • components/editor/useColorGradingPreviews.ts
  • hooks/usePreviewPersistence.ts
  • player/components/TimelineClipDiamonds.tsx

(BuildHIR::lowerExpression) Support UpdateExpression where argument is a global (3)

  • hooks/useGestureCommit.ts
  • hooks/useGsapAwareEditing.ts
  • hooks/useToast.ts

Cannot access variable while it is being initialized (3)

  • player/components/useTimelineClipDrag.ts
  • player/components/useTimelinePlayhead.ts
  • player/components/useTimelineRangeSelection.ts

(BuildHIR::lowerExpression) Handle Import expressions (2)

  • components/editor/OffCanvasIndicators.tsx
  • player/components/Player.tsx

Hooks may not be referenced as normal values; they must be called (2)

  • player/components/PlayerControls.tsx
  • player/hooks/useTimelinePlayer.ts

Hooks must be called at the top level of a function component or custom Hook (1)

  • components/ExternalFileConflictBanner.tsx

Expected a node for all identifiers, none found for 0 (1)

  • components/editor/FileTreeNodes.tsx

Cannot reassign variables declared outside of the component/hook (1)

  • components/renders/renderQueueTestHarness.tsx

Support non-trivial for..of inits (1)

  • hooks/useGsapAwareEditing.ts

Use of incompatible library (1)

  • player/components/useTimelineVirtualRows.ts

Test plan

Ten tests in src/styles/compilerBailouts.test.ts, seven of them the scenarios this gate has to get right.

Scenario Result
A component reading ref.current during render is reported with the file and the cause pass
A component the compiler can compile reports zero pass
A "use no memo" directive is counted, which the compiler itself never reports pass
A count above baseline fails naming the file and both numbers pass
A count below baseline passes and prints the lowering command pass
A missing baseline fails naming the write flag pass
The write flag rewrites the baseline and a second run passes pass

Proved end to end, not only against synthetic maps. A temporary component reading a ref during render was appended to a currently clean file, and the gate failed with src/components/AskAgentModal.tsx: 1 bail-outs, baseline 0 plus the lowering command. The file was restored.

Proved non-vacuous. The scan was broken on purpose by dropping panicThreshold: "all_errors" back to the default. The ref-during-render test failed, along with the whole-tree gate. Restored, both pass.

Proved the missing-baseline path for real. The gate was run before the baseline existed and failed with src/styles/compiler-bailouts.json is missing. Write it with: COMPILER_BAILOUTS_WRITE=1 bunx vitest run src/styles/compilerBailouts.test.ts.

Commands:

  • bunx vitest run src/styles/compilerBailouts.test.ts --maxWorkers=4: 10 passed, 0.96 s duration, 2.6 s wall including startup. The whole-tree scan inside it is 0.73 s over 737 files.
  • Full Studio suite, bunx vitest run --maxWorkers=4 from packages/studio, no Studio dev server running: 428 passed | 1 skipped (429) files, 4753 passed | 18 todo (4771) tests, 64.8 s. The branch before this one was 427 passed | 1 skipped (428) files and 4743 passed | 18 todo (4761) tests, so the delta is exactly this one new file and its ten tests. Nothing else moved.
  • bun run typecheck: clean.
  • bun run build: succeeds.
  • bunx oxlint on the two new source files: Found 0 warnings and 0 errors. bunx oxfmt: clean.
  • Fallow audit against the base branch: No issues in 4 changed files. No .fallowrc.jsonc entry was needed for the new script.

Not run, and why: the render-parity capture and the two Studio CI gates. This change adds a test file, a script and a JSON baseline. No file that ships in the bundle is touched, so there is nothing whose rendered output could differ.

Not covered

  • No component is fixed. The baseline is the number as it stands. Driving it down is the follow-up work.
  • The per-file ceiling. A second bail-out added to a file that already bails does not fire the gate. Stated above with the reason it is not a bug that can be fixed here.
  • The 72 .ts files are not in any follow-up unit's file list yet. The follow-up units were scoped from a .tsx-only list. They are enumerated above so they can be assigned.
  • components/renders/renderQueueTestHarness.tsx is a test-only harness that this scan counts as source, because its name does not match *.test.tsx. It is in the baseline at 1. Either fix it or rename it; the gate does not care which.
  • Cause strings are the compiler's own messages and are not part of the baseline, only of the report. An oxc-transform-react upgrade that rewords a message changes the report and not the gate, which is the intended split.
  • Test files are not scanned. The *.test.tsx files that drive re-renders by mutating captured let bindings are a separate unit.

Prints, for every React Compiler diagnostic, the file, the cause, and
oxc-transform-react's own codeframe (line, column, caret) so an
engineer lands on the exact statement instead of grepping the file
for the cause. Default report and --json output are unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant