Skip to content

metro-file-map: Identify TreeFS directory nodes by shape, not realm - #1934

Open
robhogan wants to merge 1 commit into
mainfrom
robhogan/treefs-realm-independent-directory-check
Open

robhogan wants to merge 1 commit into
mainfrom
robhogan/treefs-realm-independent-directory-check

Conversation

@robhogan

@robhogan robhogan commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

TreeFS uses an internal instanceof Map check in its internal isDirectory function, to identify whether a given node represented a directory (Map) or file (Array/tuple).

The only problem with instanceof Map is under Jest, in tests where we're exercising the file map cache. In Jest, tests are executed in v8 contexts where Map gets a new prototype, such that a file map cache v8-serialised in one test is deserialised but can't be traversed in another test, because instanceof Map checks fail on the foreign maps.

I discovered this while writing some unrelated tests that failed unexpectedly. We already have at least three tests in main that pass by accident (isDirectory returns false despite the node being a Map, just a foreign one) and effectively weren't testing anything. (Luckily, they all pass anyway)

An easy fix is to invert the implementation of isDirectory so that it becomes "not null and not a file", using Array.isArray, which is portable across realms.

As it turns out, this is also marginally faster.

Changelog: [Internal]

Test plan

Microbenchmark

isDirectory is hot so just to confirm this doesn't regress:

const m = new Map([['a', 1]]);
const f = [0, 1, 2, 3, 0];
const nodes = [m, f, m, f, m, m, f, m];
const fn =
  process.argv[2] === 'instanceof'
    ? n => n instanceof Map
    : n => n != null && !Array.isArray(n);
let best = Infinity;
for (let r = 0; r < 5; r++) {
  let c = 0;
  const t = process.hrtime.bigint();
  for (let i = 0; i < 2e8; i++) c += fn(nodes[i & 7]) ? 1 : 0;
  best = Math.min(best, Number(process.hrtime.bigint() - t) / 1e6);
}
console.log(process.argv[2], (best | 0) + ' ms');
Check Run 1 Run 2 Per call
node instanceof Map 216 ms 216 ms 1.08 ns
node != null && !Array.isArray(node) 180 ms 179 ms 0.90 ns

Array.isArray is marginally faster with a mix of inputs - implemented with a slot read rather than a prototype walk.

Summary:
`TreeFS` told directory nodes apart from file nodes with `node instanceof Map`, which holds only when the tree was built in the same realm as the code reading it. A tree restored through `TreeFS.fromDeserializedSnapshot` is used as-is, so a snapshot deserialized in another realm traverses wrongly: every directory below the root reads as a symlink and lookups through it fail with `Expected symlink target to be populated`, while `hierarchicalLookup` treats the root as a non-directory and returns null without probing it.

Nothing in a Metro process crosses a realm, but Jest does: each test file runs in its own `vm` context while `node:v8` is a host module, so a file map cache read back within a test holds `Map`s from another realm. The `metro` tests that build two `DependencyGraph`s on one config have been traversing such a tree, and pass only because the walks they make happen to fail in the direction their expectations need.

File nodes are metadata tuples, so `isDirectory` checks `!Array.isArray(node)` instead, which reads an internal slot and is realm-independent. The new test round-trips a snapshot through `v8.serialize`/`deserialize` and looks up a nested file in the result.

Changelog: [Internal]
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 16, 2026
@facebook-github-tools facebook-github-tools Bot added the Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team. label Sep 16, 2026
@meta-codesync

meta-codesync Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

@vzaidman has imported this pull request. If you are a Meta employee, you can view this in D120355345.

robhogan added a commit that referenced this pull request Sep 18, 2026
…andidates

Stacked on #1944.

## Summary
`resolveFile` tries each candidate extension in turn, and for each one `resolveSourceFileForExt` calls `redirectModulePath`, which asks `context.getPackageForModule` for the package scope of the candidate path. Every candidate in a directory belongs to the same package, so that's the same question asked once per extension - and because Metro's `PackageCache` memoises by module path, each distinct candidate string misses the first time it's seen, mostly for files that don't exist.

This asks for the package of the directory once, up front, and shares it with every candidate via the source file context. `redirectPackageSubpath` is the part of `redirectModulePath` that applies `mainFields` redirections when the containing package is already known. `redirectModulePath` delegates to it, and behaves as before for its other callers.

One wrinkle: files directly inside a `node_modules` directory belong to no package, but the package of that directory itself is whatever contains it, so `getPackageForFilesIn` handles that case explicitly.

`package-scope-test.js` pins the sequence of `getPackageForModule` calls the resolver makes for the common specifier shapes, so that the call count is a tested property rather than incidental.

This also removes the `@deprecated` tag from `getPackageForModule` - the resolver depends on it, and its doc now says what it returns for symlinked and missing paths.

### `resetCache` in tests
Four tests in `resolver-test.js` build a second `DependencyGraph` on the same config, and the second reads the first's file map cache. The integration tests do the same across test files and across runs, because their file map cache defaults to `os.tmpdir()`. Under Jest a cached tree comes back from `v8.deserialize` with `Map`s from another realm, which fail `TreeFS`'s `instanceof Map` checks. Asking for the package of a directory reaches `hierarchicalLookup`'s root-ancestor invariant, which turns what was previously a silent wrong answer into a failure, so both configs set `resetCache: true`. #1934 fixes the underlying problem - once that lands, the line in `resolver-test.js` can go. The one in the integration config is worth keeping regardless, since those tests shouldn't depend on what a previous run left in the temp directory.

## Benchmark
Same harness as #1944 - replaying resolutions under a default RN config with a synthetic app, 30% of source outside projectRoot (6,935 resolutions), 7 interleaved rounds. Resolver output is byte-identical.

| metric | #1944 | this diff |
|---|---|---|
| `getPackageForModule` calls per pass | 64,091 | **17,450 (−72.8%)** |
| first pass after startup (ms) | 119.6 | **85.7 (−28.3%)** |
| warm pass (ms) | 55.2 | **53.5 (−3.1%)** |
| after a source change (ms) | 63.0 | **60.9 (−3.4%)** |
| after a `package.json` change (ms) | 68.4 | **64.0 (−6.4%)** |

The first pass is where this lands, because that's where every distinct candidate path is a `PackageCache` miss and a walk up the tree.

## Test plan
```
yarn jest packages/metro-resolver packages/metro-file-map packages/metro/src/node-haste packages/metro/src/DeltaBundler/__tests__/resolver-test.js
yarn flow check
yarn verify-api-snapshots
```

Changelog: Internal
robhogan added a commit that referenced this pull request Sep 18, 2026
…andidates

Stacked on #1944.

## Summary
`resolveFile` tries each candidate extension in turn, and for each one `resolveSourceFileForExt` calls `redirectModulePath`, which asks `context.getPackageForModule` for the package scope of the candidate path. Every candidate in a directory belongs to the same package, so that's the same question asked once per extension - and because Metro's `PackageCache` memoises by module path, each distinct candidate string misses the first time it's seen, mostly for files that don't exist.

This asks for the package of the directory once, up front, and shares it with every candidate via the source file context. `redirectPackageSubpath` is the part of `redirectModulePath` that applies `mainFields` redirections when the containing package is already known. `redirectModulePath` delegates to it, and behaves as before for its other callers.

One wrinkle: files directly inside a `node_modules` directory belong to no package, but the package of that directory itself is whatever contains it, so `getPackageForFilesIn` handles that case explicitly.

`package-scope-test.js` pins the sequence of `getPackageForModule` calls the resolver makes for the common specifier shapes, so that the call count is a tested property rather than incidental.

This also removes the `@deprecated` tag from `getPackageForModule` - the resolver depends on it, and its doc now says what it returns for symlinked and missing paths.

### `resetCache` in tests
Four tests in `resolver-test.js` build a second `DependencyGraph` on the same config, and the second reads the first's file map cache. The integration tests do the same across test files and across runs, because their file map cache defaults to `os.tmpdir()`. Under Jest a cached tree comes back from `v8.deserialize` with `Map`s from another realm, which fail `TreeFS`'s `instanceof Map` checks. Asking for the package of a directory reaches `hierarchicalLookup`'s root-ancestor invariant, which turns what was previously a silent wrong answer into a failure, so both configs set `resetCache: true`. #1934 fixes the underlying problem - once that lands, the line in `resolver-test.js` can go. The one in the integration config is worth keeping regardless, since those tests shouldn't depend on what a previous run left in the temp directory.

## Benchmark
This cuts package scope lookups 3.7x, and takes 28% off first-pass resolution time, on our benchmark app (described in the test plan). 7 interleaved rounds. Resolver output is byte-identical.

| metric | #1944 | this diff |
|---|---|---|
| `getPackageForModule` calls per pass | 64k | **17k (3.7x fewer)** |
| first pass after startup (ms) | 119.6 | **85.7 (−28.3%)** |
| warm pass (ms) | 55.2 | **53.5 (−3.1%)** |
| after a source change (ms) | 63.0 | **60.9 (−3.4%)** |
| after a `package.json` change (ms) | 68.4 | **64.0 (−6.4%)** |

The first pass is where this lands, because that's where every distinct candidate path is a `PackageCache` miss and a walk up the tree.

## Test plan
```
yarn jest packages/metro-resolver packages/metro-file-map packages/metro/src/node-haste packages/metro/src/DeltaBundler/__tests__/resolver-test.js
yarn flow check
yarn verify-api-snapshots
```

**Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the resolutions a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds.

Changelog: Internal
robhogan added a commit that referenced this pull request Sep 18, 2026
…andidates

Stacked on #1944.

## Summary
`resolveFile` tries each candidate extension in turn, and for each one `resolveSourceFileForExt` calls `redirectModulePath`, which asks `context.getPackageForModule` for the package scope of the candidate path. Every candidate in a directory belongs to the same package, so that's the same question asked once per extension - and because Metro's `PackageCache` memoises by module path, each distinct candidate string misses the first time it's seen, mostly for files that don't exist.

This asks for the package of the directory once, up front, and shares it with every candidate via the source file context. `redirectPackageSubpath` is the part of `redirectModulePath` that applies `mainFields` redirections when the containing package is already known. `redirectModulePath` delegates to it, and behaves as before for its other callers.

One wrinkle: files directly inside a `node_modules` directory belong to no package, but the package of that directory itself is whatever contains it, so `getPackageForFilesIn` handles that case explicitly.

`package-scope-test.js` pins the sequence of `getPackageForModule` calls the resolver makes for the common specifier shapes, so that the call count is a tested property rather than incidental.

This also removes the `@deprecated` tag from `getPackageForModule` - the resolver depends on it, and its doc now says what it returns for symlinked and missing paths.

### `resetCache` in tests
Four tests in `resolver-test.js` build a second `DependencyGraph` on the same config, and the second reads the first's file map cache. The integration tests do the same across test files and across runs, because their file map cache defaults to `os.tmpdir()`. Under Jest a cached tree comes back from `v8.deserialize` with `Map`s from another realm, which fail `TreeFS`'s `instanceof Map` checks. Asking for the package of a directory reaches `hierarchicalLookup`'s root-ancestor invariant, which turns what was previously a silent wrong answer into a failure, so both configs set `resetCache: true`. #1934 fixes the underlying problem - once that lands, the line in `resolver-test.js` can go. The one in the integration config is worth keeping regardless, since those tests shouldn't depend on what a previous run left in the temp directory.

## Benchmark
This cuts package scope lookups 3.7x, and takes 28% off first-pass resolution time, measured in isolation on our benchmark app (described in the test plan). 7 interleaved rounds. Resolver output is byte-identical.

| metric | #1944 | this diff |
|---|---|---|
| `getPackageForModule` calls per pass | 64k | **17k (3.7x fewer)** |
| first pass after startup (ms) | 119.6 | **85.7 (−28.3%)** |
| warm pass (ms) | 55.2 | **53.5 (−3.1%)** |
| after a source change (ms) | 63.0 | **60.9 (−3.4%)** |
| after a `package.json` change (ms) | 68.4 | **64.0 (−6.4%)** |

The first pass is where this lands, because that's where every distinct candidate path is a `PackageCache` miss and a walk up the tree.

## Test plan
```
yarn jest packages/metro-resolver packages/metro-file-map packages/metro/src/node-haste packages/metro/src/DeltaBundler/__tests__/resolver-test.js
yarn flow check
yarn verify-api-snapshots
```

**Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the resolutions a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds. These timings isolate resolution - no transformation, serialisation or file reads are included. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it.

Changelog: Internal
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant