Rewrite URL mappings in streamed CSS file chunks - #314
Open
adamziel wants to merge 41 commits into
Open
Conversation
# Conflicts: # components/DataLiberation/README.md
Keep URL context tracking shared here because rewrite_chunk and next_url both use it. Earlier stack layers now keep the tracking directly in next_url.
Keep the nesting explanation beside the shared URL-context check.
adamziel
added a commit
that referenced
this pull request
Sep 9, 2026
Lets a CSS token span input chunks and continue in a new PHP process without restarting the file. Review order: #315 (merged) → #316 → #317 → #314. The processor keeps unfinished input and parses it again after another read. Whole strings and streamed input use the same `next_token()`, getters, and value setter. `flush_processed_css()` returns edited, completed input and releases its source bytes. The cursor contains the remaining bytes and whether more input is expected; the caller saves it with the source and output offsets after writing the output. This is layer 3 of the CSS stack, based on #316. It adds input, flushing, and the token cursor together. It contains no URL mappings or `rewrite_chunk()`; those remain in #314. ASCII spans are scanned together so reparsing long identifiers and URLs does not walk every byte in PHP. A large token increases memory use, cursor size, and repeated parsing work. Token size is not capped. The README spells out that cost and links the real-file caller. ## Testing The streamed tokens match the whole-string corpus, including one-byte reads, escapes, and invalid UTF-8. Separate PHP processes edit a multi-chunk file, stop before or after a saved checkpoint, and resume with the unfinished token. The written file must match whole-string edits without missing or duplicated bytes. The full DataLiberation run completed with 2,618 tests and 16 skips. PHPCS passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rewrites source-site URLs as CSS file chunks arrive. A migration can read a file in parts, write each result, and resume after the PHP process stops.
URL changes
With
https://old.examplemapped tohttps://new.example/local:url("https://old.example/photo.png?v=2#crop")url("https://new.example/local/photo.png?v=2#crop")@import "https://old.example/theme.css";@import "https://new.example/local/theme.css";image-set("//old.example/photo.png" 1x)image-set("//new.example/local/photo.png" 1x)url(https:\/\/old.example\/photo.png)url(https://new.example/local\/photo.png)a{content:"https://old.example/photo.png"}/* url(https://old.example/photo.png) */url(https://old.example.org/photo.png)Only the matched URL base changes. Quotes, parentheses, spaces, and the remaining URL bytes keep their original form. The escaped slash before
photo.pngin the fourth row stays escaped.The longest matching source base wins. Scheme and host matching ignores letter case. Path matching uses letter case and requires
/,?,#, or the URL end after the base. For example, a mapping forhttps://old.example/blogmatches/blog/photo.png, but not/blogger/photo.pngor/Blog/photo.png.Split input and resume
This example splits a URL between two calls. It also recreates the processor from JSON. The saved parser state is called a cursor. It includes the unfinished URL.
Combined output:
For real files, write and flush all output from a call before saving the cursor and both file byte offsets together. On resume, seek the source to its saved offset. Remove output bytes after the saved output offset, then append there. The source offset includes all supplied bytes: the cursor already holds the unfinished part. The file-rewrite test caller shows these steps.
Errors and limits
Resume requires the same URL mappings. For example, restoring the cursor above with
https://old.examplemapped tohttps://other.examplethrows:Saving a cursor while output remains unread also throws. Source and target bases must be HTTP(S) URLs without credentials, query strings, or fragments. Set
is_lasttotrueonly at the actual end of the CSS file.Each output chunk is at most 64 KiB. This does not limit total memory use. An unfinished CSS unit, such as a comment or URL, stays in memory and in the cursor. It is parsed again when more input arrives. A large unfinished unit therefore increases memory use, saved state size, and parsing work. More than 128 open, nested
image-set()functions causes an error.Testing
The stream tests check each possible byte split in their CSS examples, JSON resume, escaped URLs, output expansion, changed mappings, and attempts to save state before reading all output.
Separate PHP processes rewrite real files. One case runs to completion. Two cases stop just before or just after saving state, then resume in a new process. All three must produce the expected output. The tests also check that a nesting-limit error occurs on both the first run and resume. The source file stays unchanged, and the last saved state remains available.
Dependencies
The prerequisite PRs #315, #316, and #317 are merged. This PR adds URL rewriting to the streaming CSS parser without changing
CSSProcessor. WordPress/reprint#764 needs a published package release with this change before it can use the API.