Skip to content

Lift the 128 KB Vale file cap: 3.21.0 makes single-block lint time linear #350

Description

@thecodedrift

What

Lift the 128 KB oversized-file guard, VALE_MAX_FILE_BYTES in packages/cli/src/rules/vale/run.ts:120 (introduced by #323), now that the CLI vendors Vale 3.21.0 (#336).

Why it existed

#325 measured, on Vale 3.20.0, that lint time was superlinear in the size of a single Markdown block: a 3 MB one-block file took ~80 s while the same sentences with blank lines between them took ~4 s, and a 128 KB single block cost ~194 ms, 512 KB ~2 s. Combined with Vale emitting nothing until every file is done, one oversized block under a time limit cost the whole run its findings. File size was the proxy we could afford to measure before spawning, so #323 capped it at 128 KB.

What changed

Vale 3.21.0 ships two perf commits (ed711769 index the walker's context; 252d0963 index rune positions and resume column counts per block). The Vale upgrade in #339 re-measured our own single-block fixtures on the vendored binary (see the tables in packages/cli/test/vale-run.test.ts):

single block 3.20.0 3.21.0
152 KB ~1,020 ms ~45 ms
323 KB ~4,400 ms ~67 ms
6.7 MB ~1,000 ms
19 MB ~2,800 ms

Cost is now linear at ~2.8 µs per repetition, block structure no longer matters, and the 128 KB line no longer separates a cheap file from an expensive one: 128 KB of one block costs tens of milliseconds.

What to do

  1. Re-run Track upstream: Vale lint time is superlinear in single-block size, and no output is emitted until the run finishes #325's reproduction (its 3 MB single-block huge.md) against the vendored 3.21.0 to confirm on that exact shape, since the issue says the repro is ours to re-run on each vendored Vale. Record the number here.
  2. Remove VALE_MAX_FILE_BYTES and the per-run override that wraps it (run.ts:546), the skip/notice path in formats.ts (:317, :322, :560) that reports a file as oversized, and their tests. If we would rather keep a guard against a genuinely pathological input, raise it to something a linear cost justifies and say in the docblock what it now protects against; the current docblock's justification is the superlinear measurement, which is gone.
  3. Update the update.md ledger only if a user could observe a file that was previously skipped now being linted — they can, so one ledger line: files over 128 KB are linted again.

Still true

Vale still writes nothing until the run finishes, so a run that hits an external time limit still loses every finding. That half of #325 is not fixed upstream; it is just far harder to reach when cost is linear. Keep the timeout test's headroom reasoning in vale-run.test.ts as-is.

Refs #325
Refs #323

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions