Preserve CSS suffix bytes when escaping URL prefixes - #315
Merged
Conversation
This was referenced Sep 8, 2026
adamziel
added a commit
that referenced
this pull request
Sep 9, 2026
Lets a CSS token span input chunks and continue in a new PHP process without restarting the file. Review order: #315 (merged) → #316 → #317 → #314. The processor keeps unfinished input and parses it again after another read. Whole strings and streamed input use the same `next_token()`, getters, and value setter. `flush_processed_css()` returns edited, completed input and releases its source bytes. The cursor contains the remaining bytes and whether more input is expected; the caller saves it with the source and output offsets after writing the output. This is layer 3 of the CSS stack, based on #316. It adds input, flushing, and the token cursor together. It contains no URL mappings or `rewrite_chunk()`; those remain in #314. ASCII spans are scanned together so reparsing long identifiers and URLs does not walk every byte in PHP. A large token increases memory use, cursor size, and repeated parsing work. Token size is not capped. The README spells out that cost and links the real-file caller. ## Testing The streamed tokens match the whole-string corpus, including one-byte reads, escapes, and invalid UTF-8. Separate PHP processes edit a multi-chunk file, stop before or after a saved checkpoint, and resume with the unfinished token. The written file must match whole-string edits without missing or duplicated bytes. The full DataLiberation run completed with 2,618 tests and 16 skips. PHPCS passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds CSS prefix edits that leave the existing quotes,
url()wrapper, and unmatched URL suffix unchanged.Review order: #315 → #316 → #317 → #314.
measure_value_prefix()counts the source bytes behind a decoded prefix. It scans ordinary URL bytes withstrspn()instead of decoding them one by one; CSS escapes and UTF-8 still use the existing decoder. The scan stops at the matched prefix, not at the end of the URL. The docstring shows three spellings of the same 19-byte decoded host, occupying 19, 22, and 25 source bytes. Both the actual CSS text and the decoded prefix length are inputs. The caller uses the measured source length to replace the old host without touching the filename.escape_value_prefix()escapes the replacement for quoted or unquoted CSS without adding quotes. Escaping uses astrtr()table rather than a PHP character loop. An early return skips replacement lookups when nothing needs escaping. Whole-value replacements still add quotes. CRLF emits one newline, the character after a lone CR survives, and inserted escape spaces and backslashes are not escaped again.This is layer 1 of the CSS stack split from #314. It contains no streamed input, URL-context changes, mappings, or saved cursors. Later layers add those separately.
Testing
Each file test shows its input CSS, replacement URL, and literal expected CSS.
replace_url_prefix_in_file()andreplace_whole_url_in_file()return the written CSS; the process helper checks successful exit internally and reports the script log on failure. Separate fixture scripts keep prefix edits and whole-URL edits distinct.Eight tests use real files and separate PHP processes to check quote and filename preservation, unsafe URL characters, malformed URLs, strings containing a literal newline, string line continuations, CRLF, and lone carriage returns. Direct prefix checks cover ASCII scan boundaries, escapes, Unicode, NUL, invalid UTF-8, control-byte decoding, and every byte that must trigger the escaping table even when it is the only unsafe byte.
The focused suites pass with 55 tests and 101 assertions. They also pass against the preceding commit: this optimization preserves its output. The two original whole-URL newline tests fail against trunk, where CRLF produces an extra escape and the character after a lone CR disappears. The full DataLiberation run completed with 2,423 tests and 16 skips. PHPCS passes.
In a local PHP 8.4 microbenchmark, 20,000 plain-host prefix measurements went from 110 ms to 3.85 ms; a host with one CSS escape went from 114 ms to 14.75 ms. Fully escaped prefixes were about 11% slower. Prefix escaping was about 1.4–1.7 times faster for the short tested URLs. These are helper timings, not whole-migration timings.