Skip to content

fix(engine): pin capture surface scale to 1 device pixel per CSS pixel - #3782

Open
veogeek-no1 wants to merge 1 commit into
heygen-com:mainfrom
veogeek-no1:fix/pin-capture-surface-scale
Open

fix(engine): pin capture surface scale to 1 device pixel per CSS pixel#3782
veogeek-no1 wants to merge 1 commit into
heygen-com:mainfrom
veogeek-no1:fix/pin-capture-surface-scale

Conversation

@veogeek-no1

Copy link
Copy Markdown

What

Add --force-device-scale-factor=1 to buildChromeArgs(), pinning the browser's compositor surface to one device pixel per CSS pixel.

Why

On a HiDPI host every render came out at twice the composition's resolution, and no flag could bring it back down.

A composition declaring data-width="1920" data-height="1080" encoded to a 3840×2160 file. hyperframes info reported Resolution 1920x1080, resolveDeviceScaleFactor() returned 1, and compileStage computed outputWidth/outputHeight as 1920×1080 — every internal value was already correct. Passing --resolution landscape or --resolution 1080p changed nothing, because those resolve to a scale of exactly 1 for a 1920-wide composition. There was no setting that produced 1× output.

The cause is that capture reads the browser surfacepageScreenshotCapture uses Page.captureScreenshot with fromSurface: true, and the beginframe path reads the same surface — and that surface is rasterized at the host display's backing scale. page.setViewport({ deviceScaleFactor }) goes through Emulation.setDeviceMetricsOverride, which only moves window.devicePixelRatio on the page side; it does not resize the surface. So output pixels were compositionWidth × clip.scale × hostSurfaceScale, with the last factor invisible to the pipeline.

Deliberate supersampling stacked on top of it: --resolution landscape-4k on a Retina host produced 7680×4320, not 3840×2160.

Confirmed with a standalone Puppeteer probe against the same Chrome binary and the same launch args:

emulated DSF clip.scale launch flag PNG size
1 1 (none) 3840×2160
1 1 --force-device-scale-factor=1 1920×1080
1 1 --force-device-scale-factor=2 3840×2160
2 2 (none) 7680×4320
2 2 --force-device-scale-factor=1 3840×2160

The emulated DSF does not affect output size at all; the host surface scale is the whole defect.

How

One argument in buildChromeArgs(), which is the single arg builder for every launch path (render, snapshot, studio thumbnails, validate, layout, motionShot, distributed chunks), so one line covers them all.

After the change, the capture clip's scale — derived from resolveDeviceScaleFactor() — is the only thing that multiplies output pixels, which is what the resolution presets were always meant to control.

The flag is a no-op on Linux, where the surface is already 1×, so CI and cloud output are unchanged.

Test plan

Both binaries built from this tree (bun run build, then node packages/cli/dist/cli.js). "Before" is the identical build with only the new argument stripped from the bundle, so the comparison isolates exactly this change. Same project and flags in both runs; composition is 1920×1080, 10.1 s, 304 frames.

before after
dimensions 3840×2160 1920×1080
wall clock 51.7 s 39.8 s
user CPU 89.5 s 44.1 s
capture stage 32.6 s 27.4 s
encode stage 5.66 s 1.58 s
file size 23.7 MB 7.1 MB

Frame content verified visually at t=4 s — full frame, correct framing, overlays intact, no crop. Supersampling still works: --resolution landscape-4k on the same project now yields exactly 3840×2160.

  • Unit tests added/updated — buildChromeArgs asserts the flag on darwin, win32 and linux; swept over every platform because the surface scale, not the platform, is the defect
  • Manual testing performed — the A/B render above, plus the Puppeteer probe table
  • Documentation updated — not applicable; no documented behaviour changes, the presets now simply do what they say

Targeted suites run (not the full tree):

  • packages/enginevitest run src/services/browserManager.test.ts src/services/screenshotService.test.ts → 88 passed; tsc --noEmit clean
  • packages/producerbun test .../compileStage.test.ts .../distributed/plan.test.ts → 59 passed; vitest run .../captureStage.test.ts .../probeStage.test.ts → passed
  • packages/parsersvitest run src/outputResolutionCompatibility.test.ts → 12 passed
  • packages/clivitest run src/capture/ → 234 passed

Notes for reviewers

Two things this fix makes visible rather than causes, both left out to keep the patch minimal:

  1. snapshot --zoom loses density on macOS. captureRegionCrop (packages/cli/src/capture/captureCompositionFrame.ts) raises the viewport DSF and calls page.screenshot({ clip }) without a clip.scale. Per the probe table, viewport DSF does nothing to output size — so those crops were only high-density by accident on Retina, and are already 1× on Linux today. After this change macOS matches Linux, i.e. --zoom-scale is a no-op everywhere. The honest repair is to pass scale inside the clip and drop the viewport dance; studioServer.ts's thumbnailDeviceScaleFactor has the same shape.
  2. Text rasterization on macOS changes. Renders on a Retina host previously rasterized at 2× and the file really was 4K. At 1× glyph edges rasterize differently. This is a correctness gain — macOS output now matches the Linux/cloud reference — but any macOS-local golden images or compare / grade-compare baselines captured on a Retina host will shift.

Also unverified: HeadlessExperimental.beginFrame takes no clip or scale, so supersampling on the Linux BeginFrame path may be separately broken. Out of scope here and not testable on the host I have (Apple Silicon Retina); Windows HiDPI is likewise reasoned about rather than measured.

Capture reads the browser's compositor surface (`Page.captureScreenshot`
with `fromSurface: true`, and `HeadlessExperimental.beginFrame` likewise),
and that surface is rasterized at the host display's backing scale.
`page.setViewport({ deviceScaleFactor })` goes through
`Emulation.setDeviceMetricsOverride`, which only moves
`window.devicePixelRatio` on the page side — it does not resize the
surface.

On a HiDPI host every render therefore came out at twice the composition:
a 1920x1080 composition encoded to a 3840x2160 file, and no flag could
bring it down, because the pipeline's own deviceScaleFactor was already 1.
Supersampling stacked on top of it, so `--resolution landscape-4k`
produced 7680x4320 rather than 3840x2160.

Pinning the surface with `--force-device-scale-factor=1` leaves the
deliberate supersampling path (the capture clip's `scale`, from
`resolveDeviceScaleFactor`) as the only thing that multiplies output
pixels. The flag is a no-op on Linux, where the surface is already 1x, so
CI and cloud output are unchanged.
@veogeek-no1

Copy link
Copy Markdown
Author

@miguel-heygen — flagging you as the main author of browserManager.ts; I don't have permission to request a review from a fork, so mentioning instead.

Short version: capture reads the compositor surface, which is rasterized at the host display's backing scale, while page.setViewport({ deviceScaleFactor }) only moves window.devicePixelRatio on the page side. So on any HiDPI host every render came out at 2x the composition (a 1920x1080 composition encoded to a 3840x2160 file), and --resolution could not bring it down because it already resolved to a scale of 1. Supersampling stacked on top, so --resolution landscape-4k produced 7680x4320.

The Puppeteer probe table in the description isolates it: the emulated DSF does not change output size at all, only the launch flag does. Fix is one argument in buildChromeArgs(), no-op on Linux.

Happy to split out the two follow-ups I noted (snapshot --zoom losing density once the surface is pinned, and the thumbnailDeviceScaleFactor path) into separate PRs if you'd rather keep them apart.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant