Skip to content

The gem does not wait for webfonts, so users mask the font-swap race with tolerances and retries #273

Description

@pftg

Third finding from a real project's workaround layer (jetthoughts.github.io). Unscheduled — recorded now so a kickoff has the evidence.

What users have to build themselves

# Fonts settled before capture: a webfont swapping in mid-capture reflows
# text bimodally (the "font-swap race" this suite otherwise masks with
# skip_areas, 0.03 tolerances, and stability retries). fonts.ready
# resolves instantly once fonts are cached, so warm sessions pay ~0.
page.evaluate_async_script("var done = arguments[0]; document.fonts.ready.then(function() { done(true) })")

One line, standard browser API, resolves instantly when fonts are cached. Without it, a webfont swapping in mid-capture reflows text bimodally — the screenshot is one of two valid layouts depending on timing.

The important part is the parenthetical: the alternative they were living with was skip_areas, 0.03 tolerances, and stability retries. Every one of those is expensive, and every one degrades the test's ability to catch real regressions — a 0.03 tolerance on a 1920x1080 frame lets ~62,000 pixels change before failing.

So a race we could eliminate is currently paid for in three separate forms of blindness.

Related, same class: lazy-loaded images

def preload_all_images
  scroll_to :bottom
  assert_text "JetThoughts. All Rights Reserved", exact: false
  scroll_to :top
end

A hand-rolled scroll-to-bottom-and-back to force lazy images to load before capture. Also standard, also reinvented per project.

Measured cross-platform noise, for context

From the same file, after measurement:

Measured run-to-run noise is ~1e-6 (three runs: 0.013838252 twice, then 0.013837770 — about 2 px of 2,073,600) … That 1e-6 is macOS local; two CI recordings of the same commit left 135 of 147 baselines byte-identical but moved 11 font/SVG-heavy pages by 0.0003–0.002, every one of which pins 0.03.

Local noise is ~2 px. On CI, font- and SVG-heavy pages move 300–2000× more. The pages that move are the font-heavy ones — which is what a font-swap race predicts.

Questions a kickoff should answer, not assume

  • Should document.fonts.ready be awaited before every capture by default? It is instant when fonts are cached, but it is still a synchronous browser round-trip and not every driver supports evaluate_async_script equally.
  • Is this better as a default, an opt-in capture option, or a documented recipe? The evidence that users reinvent it argues against "documented recipe" — the whole point is that they had to discover it.
  • Do the other stabilisation sources deserve the same treatment (lazy images, CSS animations, scroll-behavior, caret blink)? Two independent users were previously observed hand-rolling stabilising CSS injection.
  • Would this make stability_time_limit tunable down, which is the dominant per-screenshot cost (Instability is undiagnosable, so people add sleeps instead — and the wait is the dominant cost #271)?

Not proposing an implementation

Per the standing rule, this is a maintainer-adjacent idea until a kickoff decides its shape. What is not in doubt is the evidence: a real project reinvented it, measured the noise, and is paying for the race with tolerances that blind the tests.

Related: #271 (instability is undiagnosable, so people add sleeps), #272 (skip_area selectors block 5s each).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementneeds-evidenceMaintainer idea; unproven. Needs demand or measurement before scheduling.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions