Build/Test Tools: Add Code Reference Playground previews - #12954
Draft
sirreal wants to merge 60 commits into
Draft
Build/Test Tools: Add Code Reference Playground previews#12954sirreal wants to merge 60 commits into
sirreal wants to merge 60 commits into
Conversation
The preview needs one checked-in source of truth for its toolchain, upstreams, runtime, limits, and behavioral smoke targets. Loose refs make the cache lie and make builds drift. Validate official repositories and full commits, resolve beta to a concrete versioned download, bind every material base input into the cache key, and keep arbitrary forks inert. Review follow-ups: move from Node 20 when the harness is exercised on a supported release; make cache-key field naming and ordering explicit when the build caller lands.
The base cache is only valid when every material input names the same build. Bind the runner, PHP version, concrete WordPress beta, dependency manifest, and harness into one exact key. Stage only eligible Core PHP as parser input. Fetch the pinned commit before every checkout; otherwise a reused clone breaks as soon as a dependency pin moves.
The DevHub themes do not carry a complete immutable Node toolchain. Letting their ranges resolve at build time makes the invariant cache depend on the day it was built. Pin Yarn and the WordPress build packages in the central manifest and lock their dependency graph. A changed pin now changes the exact base-cache identity.
Rebuilding themes, plugins, parser dependencies, and the empty DevHub site on every preview wastes the expensive half of the job. Build that state once under the exact cache key and write the completion marker last. The untrimmed base was 98,884,490 bytes. Drop unused CJK preview fonts before packaging; the real cold build falls to 43,224,482 bytes and leaves room for the reference import without weakening the 100 MiB limit.
The preview must parse the exact Core checkout without loading it. Stage only eligible PHP, run the pinned parser, and normalize source roots for the later import. Current trunk yields 1,318 files, 764 classes, 7,316 methods, 4,386 functions, 1,031 hooks, and 2,397 filters. Fixed low floors catch collapse without pretending counts are stable. Empty output is advisory metadata, not a publisher candidate.
The invariant base is not a runnable reference. Restore it, import every parser record, cache source, remove the parser, and only then lock the runtime. An oversized result must never reach publication.
Runtime policy cannot live only in build metadata. Install it after import, show the exact source and run on every page, and expose the final state for behavioral validation. Block WordPress HTTP requests so the preview cannot quietly depend on the network.
Parser success does not prove a runnable preview. Boot the exported snapshot and exercise the index, representative symbols, search, provenance, import completion, and runtime policy through Playground. Return defects as validation state so advisory mode can report them without publishing the candidate.
Workflow YAML must not own build semantics. Resolve the pinned inputs, build or reuse the exact base, parse Core, package once, boot the result, and emit one terminal handoff from a repository-local command. A behavioral failure is never publishable; a fatal build still reports identity and still fails.
The parser importer includes the Core version file. Pointing every record at the pull request source let untrusted PHP run during snapshot import. Give that one record a trusted empty replacement and create it before import. Route and search checks also accepted generic responses. Require representative symbol content and the expected search result path. Non-blocking: imported WordPress version metadata remains unset. Exact source-link behavior is deferred.
Preview builds must use the exact fork head without write authority. Run only for the one-shot label or a labeled synchronize event, and bind the invariant base to one exact cache key. A repeated SHA must not burn another full import when its public snapshot is healthy. Reuse only after the Playground proxy returns the expected headers, bytes, and digest. A rerun bypasses reuse. A missing handoff is a build failure. Publication remains a separate trusted change; this slice is already at the complexity tripwire.
The publisher must not infer trust from a successful build job. Bind every handoff to the enabled repository, exact workflow attempt, fetched pull request, trunk base, head repository, head SHA, schema, filename, size, and digest. Fresh and reused candidates share the same runtime and dependency checks. Failed or advisory results are terminal states, never publishable candidates. The snapshot is hashed but never opened. The network layer must refresh the latest run immediately before every mutation. A cached lookup defeats same-SHA supersession.
Fork workflow runs do not reliably carry pull request associations. Resolve the open pull request from the exact head owner, branch, repository, and SHA instead of trusting an empty field. Latest-wins needs a fresh workflow lookup, including rerun attempts. Keep that query and the release, comment, and label calls behind one narrow GitHub API client so transaction ordering remains visible in the publisher.
A release upload is not a usable preview until the public path returns the expected bytes. Validate the public metadata, launch pointer, proxy headers, size, and digest before accepting it. Obsolete fork runs exit without touching current state. Previous-preview discovery fetches accepted metadata twice. That is redundant network work, not a correctness defect.
Publication is the privileged boundary. Recheck the current head before every mutation, validate public delivery before moving the comment, and upload before deleting the old preview. Failed cleanup touches only asset IDs uploaded by this invocation. Comment, cleanup, and label failures remain visible instead of masquerading as supersession.
Publication failures must not erase the last working preview or leave the request label stuck behind unrelated cleanup. Exercise fresh publication, advisory failure, missing artifacts, latest-wins, retry collisions, public delivery failure, reuse, and ordinary transport errors. Mutation ordering is inferred from final state and injected failures rather than a separate operation log. The product boundary is still covered.
The build job cannot publish with repository authority. Run the publisher from default-branch code, bind the handoff to the exact workflow attempt and pull request, and validate public delivery before moving the sticky comment. GitHub still emits workflow runs when the build job is skipped. Ignore those exact attempts and exclude them from latest-wins selection or an unrelated label consumes the real preview request.
Pull request caches live under refs/pull/<number>/merge. List that exact scope and delete only returned cache IDs; a broad cache deletion can destroy the shared trunk base. The cache assertions share the release transport test. That is test organization, not a boundary gap.
A new unlabeled commit makes the published preview stale; closing the pull request removes only that PR's release assets and merge-ref docs caches. Recheck the exact head or closed state before every mutation. Do not mark the comment expired until every preview asset is gone. The tests assert final deletion sets rather than a separate ordering log; the transaction order remains explicit in the driver.
Fork pull requests need trusted stale and cleanup mutations without ever running their code. Use default-branch lifecycle code, separate stale permissions from destructive cleanup, and cancel obsolete events per pull request.
A missing label is not proof that an old stale run still owns the comment. Bind stale mutation to the absence of a non-skipped current-SHA build, and retain the prior attempted SHA when no healthy preview exists. Close cleanup must run after both the cache-writing build and trusted publisher. Cancel the build, queue cleanup durably behind publication, then enumerate assets and PR-scoped caches. Public proxy retries and reuse-validator deduplication remain follow-ups. Partial asset cleanup deliberately leaves the historical comment unchanged until a rerun completes deletion.
Label absence is not ownership. Compare the exact marked comment before mutation so a newer publisher state wins. Keep the original attempted SHA across stale transitions. Enter build concurrency only from the eligible open-PR job so closed events cannot cancel cleanup. Non-blocking follow-up: the stale lifecycle job retains an unused actions: read permission.
The stable link needs an immutable candidate identity before it can move. Bind trunk snapshots to the exact SHA, run, attempt, runtime, and size boundary, then validate the Blueprint through the public proxy. Non-blocking follow-ups: consolidate the duplicated snapshot-size constant and document the public-Blueprint validator's boundary-4 justification.
The publisher cannot trust its trigger once trunk moves. Read the current trunk ref and newest workflow attempt before each mutation, and use GitHub's asset rename operation for the stable pointer transaction. Non-blocking follow-up: add direct coverage for ordering two valid trunk runs.
Two asset renames can strand the stable URL between names. Validate an immutable Blueprint commit publicly, move one Git ref atomically, and re-read its SHA before deleting the previous snapshot. Unknown ref state retains both asset generations. The pointer cannot outlive the snapshot it names. Non-blocking follow-ups: exercise the stable branch URL in staging and tolerate unreachable pointer commits from failed attempts.
The pointer transaction is only useful if failures cannot erase its target. Cover public candidate rejection, atomic-ref ambiguity, unknown ref state, cleanup failure, and strict mutation ordering. Non-blocking follow-ups: assert metadata-fetch ordering and unchanged ref identity in invalid terminal cases directly.
The shared cache and stable pointer rot if trunk does not exercise the real build. Build the exact pushed SHA with read-only authority, then hand it to queued default-branch publisher code. Non-blocking follow-ups: reduce duplicated PR/trunk orchestration and broaden the publisher concurrency comment.
The preview is useless if contributors cannot operate or recover it. Document the exact local command, pins, cache scopes, label lifecycle, publication names, rollback behavior, and justified validators. Keep the repository README on one stable Blueprint URL. Trunk publication moves the ref, not this link.
A validated commit is not the URL contributors open. The public proxy can still serve the old Blueprint after the ref moves; deleting that snapshot then breaks the stable link. Fetch the stable URL before cleanup. If delivery has not caught up, retain both complete generations. Non-blocking follow-up: retry transient public-pointer propagation before retaining both generations.
Job-level environment expressions cannot read the runner context. Both preview builds would fail workflow validation before the local pipeline started. Use the runner-provided temporary directory inside each shell step. Keep the required publication queue and narrowly silence the repository's older actionlint for that one current GitHub syntax key. Non-blocking follow-up: upgrade actionlint when it accepts GitHub's current concurrency queue key, then remove the narrow suppressions.
GitHub exposes four CPUs, so Playground selected three workers and exported a malformed SQLite database in both staging builds. Invoke the pinned programmatic API with its recommended six workers and validate with the same count. If this is wrong, every reference route dies with a database error.
Playground exports a live filesystem after running the Blueprint. Multiple build workers left SQLite corrupt after the full reference import. Build with one worker, then boot the finished snapshot with six. If this regresses, every reference route dies before publication.
GitHub can expose a stale ref immediately after creating the stable pointer. Treating that read as proof of failure deleted the assets the new pointer named. Any unconfirmed mutation now retains both generations; the next successful publication cleans the orphan.
Fork pull_request runs do not receive the base repository's Actions variables. The old condition made the required fork lifecycle impossible to exercise in staging.\n\nAllow the hard-coded staging repository to run only the untrusted read-only build. Keep trunk, publication, lifecycle, and cleanup behind DOCS_PREVIEW_STAGING.
Fork pull_request runs cannot read the base repository's Actions variables. The old check silently left every fork in advisory mode, so DOCS_PREVIEW_ENFORCE did not enforce anything.\n\nApply the exact base-repository value in the trusted publisher after it records the failure and preserves the live preview.
The publisher collapsed multiple failures into an AggregateError and discarded every cause. That leaves a broken comment or label path with no actionable log.\n\nPrint the nested causes after the transaction has already failed. Publication behavior and exit status stay unchanged.
The trusted publisher could upload snapshots but GitHub rejected every PR comment and label mutation. Declaring pull requests read-only made the required lifecycle impossible.\n\nGrant write access to pull requests and drop the redundant issue scope. The trunk publisher remains unchanged.
Stale lifecycle work discarded the last working link when public revalidation was temporarily unavailable. Its token also used a different PR write scope than the publisher. Reuse the marked comment's last-successful identity as the fallback and grant the same pull-request write permission. A proxy outage now leaves the working preview reachable while the comment moves to Stale.
Test using WordPress PlaygroundThe changes in this pull request can previewed and tested using a WordPress Playground instance. WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser. Some things to be aware of
For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation. |
sirreal
commented
Aug 10, 2026
|
|
||
| Welcome to the WordPress development repository! Please check out the [contributor handbook](https://make.wordpress.org/core/handbook/) for information about how to open bug reports, contribute patches, test changes, write documentation, or get involved in any way you can. | ||
|
|
||
| [Open the latest successful Core Code Reference preview in WordPress Playground](https://playground.wordpress.net/?blueprint-url=https%3A%2F%2Fwordpress-playground-cors-proxy.net%2Fhttps%3A%2F%2Fraw.githubusercontent.com%2FWordPress%2Fwordpress-develop%2Fdocs-preview-code-reference%2Fcode-reference-trunk.json). |
Member
Author
There was a problem hiding this comment.
Member
Author
Member
Author
|
WordPress/wporg-developer#578 adds extension points to the theme, allowing for the code links to work correctly in PR previews. |
Preview builds resolve WordPress from the beta channel and fail outright when that channel offers no beta or release candidate. Every labeled pull request build and every trunk build therefore breaks for the weeks between a stable release and the next beta, and the pinned channel leaves no configuration escape. Fall back to the stable offer from the same version check response, keeping the strict download URL binding, and accept that channel in published metadata. A beta or release candidate still wins whenever the API offers one.
Release asset names are derived from the run and attempt, so a partial publication that also fails to clean up leaves a name that can never be uploaded again. Every re-run of that attempt fails with HTTP 422, reports the build as broken, and consumes the request label until someone deletes the asset by hand. Confirm the duplicate by listing the release, delete the stale asset, and retry the upload once. Report the HTTP status on API errors so callers can branch on it.
The base image downloads composer.phar over the network and executes it without any integrity check, and reuses the cached copy unverified. It is the only unpinned executable in an otherwise digest-pinned pipeline, so a compromised download would run inside the trusted trunk build that produces the published Blueprint. Pin the expected SHA-256 for the manifest's Composer version and verify both the downloaded and cached phar before use. A version without a recorded digest now fails closed.
Only the first authorization in the stale pass caught supersession, so re-adding the label during the slow snapshot download turned the trusted lifecycle job red. Expiry folded per-item supersession into an AggregateError, so reopening a pull request mid-cleanup failed the job with the preview half deleted. Route every supersession through the same graceful result in both paths, and stop expiry at the first superseded item so the comment is never marked expired while assets remain. Genuine failures still fail the job.
Both publishers validate the triggering run before the session exists, outside the wrapper that turns supersession into a graceful result. A build re-run, a still-running attempt, or a fork deleted between build and publication therefore fails the trusted job red, even though the identical checks inside the session report the run as superseded. Mark the routine races distinctly from identity failures and report them as superseded from the establishment path as well. Genuine API outages and invalid run identities still fail the job.
Updating the preview comment has no fallback when the comment is gone. Deleting it while a publication runs turns the update into an HTTP 404 that reaches the transaction handler, which deletes the freshly validated snapshot, reports the successful build as failed, and removes the request label. Create a replacement comment when the update reports that it no longer exists. Every other update failure still fails the publication.
Trunk publication also required the branch head to equal the built commit, so a push that produces no build run supersedes a finished one. A commit landing with [skip ci], or a run lost to an incident, therefore discards the completed snapshot with a green job, and no later run can ever satisfy the check, leaving the stable preview stale until an unrelated push happens to succeed. Supersede a publication only when a newer trunk build run exists, and record a notice when the branch head has moved past the commit being published. Every other authorization check is unchanged.
The pointer transaction reads the stable Blueprint back over the raw CDN, which caches the branch path for minutes and can still serve the previous Blueprint. That fails a publication which actually succeeded and strands its assets. The same cache keeps handing out Blueprints that reference the snapshot the publication deletes as soon as the pointer moves, so the stable preview link breaks for several minutes after every trunk build. Read the moved pointer back through the contents API, which is never cached, and retain the previous snapshot generation for one more publication so cached Blueprints stay resolvable.
The trunk publisher returns an invalid handoff as a successful no-op and never reads the enforcement variable that the pull request publisher honors. A behavioral validation failure, including a transient Playground boot flake, therefore skips publication with every job green, no annotation, and no comment, leaving the stable trunk preview silently pinned to an older commit. Enforce or annotate the invalid result the way the pull request publisher does, and pass the enforcement variable to the trunk publish step.
Every trunk authorization looks up the newest trunk build, and that lookup collects every page of the workflow's run history before selecting one run. The publisher authorizes before each of roughly nine mutations, so the request cost per publication grows with the age of the repository and eats into a token budget shared with the pull request publishers and the lifecycle jobs. Walk the pages lazily and stop at the first page holding a match, which the newest-first ordering makes the newest run. A history with no matching run on early pages still pages through to the end.
Every push to an open trunk pull request completes a build run, and unlabeled pull requests skip all of its jobs. The publisher only recognizes that after checking out the repository and installing Node, so each of those pushes spends a runner job to reach a no-op result. Filter the skipped conclusion out of the publish job condition. Failed and cancelled builds still reach the publisher, which reports them on the pull request.
Three small defects share a shape: a boundary that fails quietly. Asset pagination spreads the null that a deleted release returns, so the reuse check reports an unrelated type error instead of an absent release. The size boundary reaches the guards as NaN when the argument is missing or malformed, and every guard compares with a greater-than that NaN passes, so the cap silently disappears. Captured output concatenates each pipe chunk as its own string, so a character split across a chunk boundary reaches log files and error text as replacement characters. Treat a missing page as the end of the assets, refuse reuse without a usable size boundary, and decode captured streams as whole text.
No request in the preview pipeline carries a timeout. A stalled proxy or API connection therefore runs until the job timeout kills it, which in the ten minute stale job leaves the preview comment advertising a ready preview for an outdated commit, with nothing in the log explaining why. Attach a deadline to every remote call: thirty seconds for API reads and metadata, five minutes for the snapshot transfers that move a hundred megabytes through the proxy. The Playground probes keep their own boot deadline.
A single transient failure aborts a publication partway through its sequence, because no API call is ever attempted twice. Rate limiting and brief server errors are routine on a shared token, and the publisher issues many reads per publication, so an outage lasting seconds can strand uploaded assets and report a healthy build as broken. Repeat a failed read up to three times, waiting one to two seconds and then two to three, the jitter keeping concurrent jobs from returning together. Mutations are never repeated, since the server may have applied the attempt that appeared to fail.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds an opt-in pipeline for building the complete Core Code Reference from an exact
wordpress-developcommit and packaging it as a WordPress Playground snapshot.It publishes:
docs-previewlabel, tracked in one persistent comment; andtrunksnapshot, linked from the README through a stable Blueprint.The build is JavaScript scription. GitHub Actions handles orchestration, identity, caching, and publication. Dependencies and Actions are pinned. Playground
betais resolved to a concrete build before caching, and the invariant site base uses an exact cache key.Security and lifecycle
PR source is parsed as data; it is never executed. Fork builds have no secrets or write authority. A separate trusted
workflow_runpublisher validates the source, run, artifact, size, and digest before changing public state.Publication is transactional. A stale, invalid, or failed build cannot replace the last good preview. Runtime policy disables cron, updates, file modifications, automatic login, and WordPress-side outbound networking.
docs-previewbuilds the exact current head SHA.Possible follow-up: snapshot revalidation on the stale path
Marking a preview stale re-reads the published snapshot in full.
markStale()callsfindPreviousPreview(), which reachesloadPublishedPreview()and thenvalidatePublicSnapshot(), so every push to a pull request that already has a preview pulls up to 100 MiB through the CORS proxy and re-hashes it, only to recover the link its comment already carries. The stale job allows ten minutes.This is left as is, because the tradeoff is worth observing rather than guessing:
HEAD, or a rangedGET) would confirm the asset is still served without moving the payload, at the cost of no longer checking that the bytes are the published ones.6f3f931already falls back to the marked comment's last successful identity when revalidation fails, so the verification may now be redundant on this path, and removing it is a third option.To create the conditions:
wordpress-playground-cors-proxy.netslow or unreachable for that job. Before the deadlines added in this PR, the job ran until its ten-minute timeout killed it, and the comment kept reporting a ready preview for the superseded commit.Testing
git diff --check, PHP syntax checks, and PHPCS pass.Key staging evidence (via
sirreal/wordpress-developfork):Trac ticket:
Use of AI Tools
AI assistance: Yes
Tool(s): Claude, Codex
Model(s): Fable 5, Opus 5; GPT-5.6 Sol
Used for: Planning, implementation, review.
This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.