Skip to content

fix(registry): verify staged asset content before publishing manifests - #3832

Merged
jrusso1020 merged 4 commits into
mainfrom
fix/security-registry-asset-snapshot
Sep 10, 2026
Merged

fix(registry): verify staged asset content before publishing manifests#3832
jrusso1020 merged 4 commits into
mainfrom
fix/security-registry-asset-snapshot

Conversation

@jrusso1020

Copy link
Copy Markdown
Collaborator

Registry asset hosting previously checked a path, reopened it to hash its contents, and later copied it again. A source replacement between those operations could publish different bytes under an immutable CDN key derived from the original file (CodeQL #902).

Read and validate the source through one descriptor, then verify the full SHA-256 of the staged copy against the planned digest. If the bytes changed, remove that staged object and stop before uploading or rewriting manifests. Manifest updates now happen after all staging succeeds, so a failed verification leaves the original manifests available for retry. Existing CDN keys, deduplication, upload arguments, dry-run output, and no-upload source retention are preserved.

Validation: the actual script runs against isolated fixture registries. A deterministic replacement between hashing and copying fails on the original code and is rejected by this change; normal staging, byte/key identity, unchanged dry-run manifests, and source retention pass. The regression is wired into test:scripts: 190 Node tests and 86 catalog tests pass. Full workspace build, lint/format, Fallow, typecheck hooks and signed commits pass.

This addresses the checked-source/staged-content race. The maintainer-owned checkout and staging directory retain their existing local trust assumption; this does not claim atomic protection from another process with the same user's filesystem permissions. No CDN upload was performed during validation.

@miga-heygen miga-heygen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head 61757bee04445fa066410ac2747bac6a799995c3 (3 files, +112/−24). Verified locally in an isolated checkout built the way the Test job builds (parsers/lint/studio-server → core → core runtime), then node --import tsx --test scripts/host-registry-assets.test.ts, typecheck:scripts, oxlint/oxfmt on the two changed files, and a read-only --all --dry-run against the real registry. No upload, no manifest write to the tree.

Strengths

  • scripts/host-registry-assets.ts:132-141 — validation and hashing go through one descriptor (openSyncfstatSync(fd)readFileSync(fd)), O_NONBLOCK keeps a FIFO from hanging the run, and the fd is closed in finally on both the throw and success paths. This is the structural fix js/file-system-race asks for, not a check-reorder.
  • scripts/host-registry-assets.ts:177-183 — the staged copy is re-hashed against the full planned digest (not the 16-hex key prefix), the mismatched object is unlinked, and the error text ("No upload was attempted") is accurate for every reachable path.
  • scripts/host-registry-assets.ts:253-254 — manifests are only rewritten after stage() returns. Confirmed with a two-item probe where the second item's source is swapped mid-copy: both registry-item.json files stay byte-identical to their originals and both sources remain on disk, so a plain re-run is the retry.
  • scripts/host-registry-assets.test.ts:53-70 — the regression is real: with the main version of the script swapped in, this test fails with Missing expected rejection; at head both tests pass. Isolating via a temp-root copy of the script (so repoRoot/STAGING_DIR resolve under tmpdir) is the right shape for a script that otherwise writes into registry/.

Claims checked

  • "Behavior unchanged" for the third commit: main():246 filters empty-plan items before flatMap, which is exactly the set the removed inline loop wrote; the plans.length === 0 early-return is still reached from the same state. Equivalent.
  • Deduplication: two entries with identical bytes stage one object under one key and both manifest entries carry the same URL (probed).
  • Dry-run: no longer write-then-restore; nothing on disk changes and no staging dir is created (probed on the real registry: 4 items / 70 files reported, tree clean).
  • No-upload source retention: covered by the first test; also holds on the abort path.
  • CodeQL: the PR merge ref has 0 open alerts across 103 rules; the js/file-system-race instance on main (hostFile, line 130) has no counterpart at head. The main alert itself will only flip once main is re-scanned post-merge.

Nits

  • scripts/host-registry-assets.ts:132 — a missing declared file now surfaces as the raw ENOENT: no such file or directory, open '/abs/path' instead of the previous <item>: files[] declares "<path>", which is not on disk. (directories still get the friendly message via fstat). Catching ENOENT around openSync and rethrowing the item-scoped message would keep the operator context without reintroducing a path pre-check.
  • scripts/host-registry-assets.ts:175seen dedups on the 16-hex key prefix. Now that plan.digest is in hand, a Map<key, digest> could throw on a prefix collision with differing full digests instead of silently pointing the second manifest at the first file's bytes. Astronomically unlikely for honest content; cheap to make explicit.

Notes (no action)

  • On abort, objects staged for earlier plans remain in .registry-assets-staging (only the mismatched one is unlinked). Harmless: stage() starts with rmSync, and neither --no-upload nor upload() is reached.
  • Trust model: the stated maintainer-owned checkout/staging assumption is appropriate for this path. The script runs on a maintainer workstation with that maintainer's AWS profile; a same-user adversary already holds the credentials, so the residual windows (discardStagedSources:226 check-then-unlink, source modified between stage() and post-upload discard) don't buy an attacker anything they lack. What this PR closes is the realistic failure — an editor save, asset regeneration, or git checkout mid-run — which now aborts loudly before any manifest or CDN write instead of publishing mismatched bytes under an immutable key.
  • Cost: each asset is now read twice (plan + staged verification). Inherent to verifying; fine for this tool.

Companion PR
#3827 touches packages/producer only; there is no code dependency between the two. Sequencing #3827's alert closure first is a process preference, not something this PR's correctness relies on.

Verdict: APPROVE
Reasoning: The race is closed structurally (single-descriptor hash, full-digest verification of the staged bytes, manifests written last), the regression test genuinely fails on the baseline, and every behavior-preservation claim in the description held up under local probing. Remaining items are nits.

— Miga (pr-review)

@miga-heygen miga-heygen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-check of head 19a30a85 after the main-branch conflict refresh (merged=null; native CI green at this head: Test, CLI smoke, Fallow audit, CodeQL, Windows shards all passed; the two skipped jobs — one Preflight variant and regression-shards — were skipped identically at 61757bee).

Verified

  • The three PR commits (49cdfa58, 8a147f0f, 61757bee) are unchanged; 19a30a85 is a merge of current main (merge base is now 8bf5b442 = origin/main). Interdiff of the PR delta against the base before and after the refresh is exactly one line: package.json test:scripts now lists both scripts/host-registry-assets.test.ts (yours) and packages/core/scripts/writeGeneratedFile.test.ts (main's). Nothing else moved.
  • scripts/host-registry-assets.ts and its test are byte-identical to the approved head, so the single-descriptor hash, full-digest verification of the staged copy, and manifests-written-last ordering all stand as reviewed.
  • Local at 19a30a85: host-registry-assets.test.ts 2/2 (staging match + dry-run/no-upload retention; source replacement between plan and stage aborts before publication), and the full test:scripts chain (node test runner + vitest run scripts/catalog/) exits 0.

The two nits from the first review (raw ENOENT path in the missing-source error; key-prefix dedup could assert the full digest) are unchanged and remain non-blocking, as agreed.

Verdict: APPROVE
Reasoning: The refresh is a pure merge with a one-line, correct test-list resolution; the security change is unchanged and re-verified locally with CI green at this head.

— Miga (pr-review)

@jrusso1020
jrusso1020 merged commit 962c954 into main Sep 10, 2026
54 checks passed
@jrusso1020
jrusso1020 deleted the fix/security-registry-asset-snapshot branch September 10, 2026 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants