demo: Add cache GC measurement harnesses (Node, browser, Android) by ntucker · Pull Request #4034 · reactive/data-client

ntucker · 2026-07-19T17:22:03Z

Summary

Add opt-in cache GC baselines for Node (examples/benchmark start:gc), Chromium (examples/benchmark-react bench:gc), and release Hermes Android (examples/benchmark-native)
Wire BuildManifest / provenance verification so measurements bind to verified build identity rather than live HEAD
Document scenario axes, timing boundaries, and validation gates in example READMEs and the GC plan

Test plan

yarn install succeeds with the new examples/benchmark-native workspace
Node: yarn workspace example-benchmark build && yarn workspace example-benchmark start:gc:verify then a short start:gc filter run
Browser: yarn workspace example-benchmark-react build && yarn workspace example-benchmark-react test:gc-metrics && yarn workspace example-benchmark-react test:gc-provenance
Browser: yarn workspace example-benchmark-react bench:gc --scenario unique-1000 --samples 1
Native: yarn workspace example-benchmark-native typecheck && yarn workspace example-benchmark-native test && yarn workspace example-benchmark-native lint
Confirm CircleCI setup still strips example workspaces (including the new index)

Made with Cursor

vercel · 2026-07-19T17:22:05Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs-site	Ignored	Preview	Sep 13, 2026 4:39pm UTC

changeset-bot · 2026-07-19T17:22:07Z

⚠️ No Changeset found

Latest commit: 3412517

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

cursor

Stale Bugbot comment from a previous run.

cursor · 2026-07-19T17:23:44Z

+SIDECAR_BUILD_ID="$(node -e 'console.log(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).buildId)' "${SIDECAR}")"
+SIDECAR_DIGEST="$(node -e 'console.log(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).sourceDigest)' "${SIDECAR}")"
+SIDECAR_APK_SHA="$(node -e 'console.log(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).apkSha256)' "${SIDECAR}")"
+SIDECAR_ID="$(node -e 'console.log(JSON.parse(require("fs").readFileSync(process.argv[1],"utf8")).sidecarId)' "${SIDECAR}")"


Node eval argv index wrong

High Severity

Inline node -e snippets treat process.argv[1] as the first path argument, but with -e Node puts -e at index 1 and script arguments at index 2 onward. Sidecar verification, digest reads, and report provenance steps read or require the wrong paths, so host collection fails or skips checks.

Additional Locations (1)

examples/benchmark-native/scripts/collect-report.sh#L186-L209

^{Reviewed by Cursor Bugbot for commit 30bdad5. Configure here.}

github-actions · 2026-07-19T17:24:14Z

Size Change: 0 B

Total Size: 81 kB

ℹ️ View Unchanged

Filename	Size
`examples/test-bundlesize/dist/App.js`	1.46 kB
`examples/test-bundlesize/dist/polyfill.js`	307 B
`examples/test-bundlesize/dist/rdcClient.js`	10.9 kB
`examples/test-bundlesize/dist/rdcEndpoint.js`	8.07 kB
`examples/test-bundlesize/dist/react.js`	59.6 kB
`examples/test-bundlesize/dist/webpack-runtime.js`	784 B

_{compressed-size-action}

github-actions

Benchmark React

Details

Benchmark suite	Current: `3412517`	Previous: `3fa47c8`	Ratio
`data-client: getlist-100`	`208.33` ops/s (`± 4.6%`)	`137.94` ops/s (`± 4.5%`)	`0.66`
`data-client: getlist-500`	`66.01` ops/s (`± 5.9%`)	`44.05` ops/s (`± 3.8%`)	`0.67`
`data-client: update-entity`	`606.62` ops/s (`± 10.5%`)	`384.62` ops/s (`± 9.3%`)	`0.63`
`data-client: update-user`	`500` ops/s (`± 9.2%`)	`333.33` ops/s (`± 8.6%`)	`0.67`
`data-client: getlist-500-sorted`	`71.44` ops/s (`± 9.3%`)	`43.11` ops/s (`± 9.4%`)	`0.60`
`data-client: update-entity-sorted`	`434.78` ops/s (`± 6.4%`)	`312.5` ops/s (`± 7.9%`)	`0.72`
`data-client: update-entity-multi-view`	`476.19` ops/s (`± 7.2%`)	`327.96` ops/s (`± 7.5%`)	`0.69`
`data-client: list-detail-switch-10`	`21.37` ops/s (`± 9.9%`)	`9.9` ops/s (`± 9.4%`)	`0.46`
`data-client: update-user-10000`	`112.36` ops/s (`± 15.8%`)	`72.2` ops/s (`± 14.8%`)	`0.64`
`data-client: invalidate-and-resolve`	`60.98` ops/s (`± 6.7%`)	`37.67` ops/s (`± 5.6%`)	`0.62`
`data-client: unshift-item`	`303.03` ops/s (`± 5.2%`)	`219.81` ops/s (`± 6.0%`)	`0.73`
`data-client: delete-item`	`400` ops/s (`± 4.8%`)	`285.71` ops/s (`± 3.8%`)	`0.71`
`data-client: move-item`	`250` ops/s (`± 8.4%`)	`177.01` ops/s (`± 7.9%`)	`0.71`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Benchmark

Details

Benchmark suite	Current: `3412517`	Previous: `93555f8`	Ratio
`normalizeLong`	`522` ops/sec (`±3.93%`)	`447` ops/sec (`±4.75%`)	`0.86`
`normalizeLong Values`	`477` ops/sec (`±0.22%`)	`408` ops/sec (`±1.51%`)	`0.86`
`normalizeLong Scalar`	`438` ops/sec (`±3.07%`)	`352` ops/sec (`±3.69%`)	`0.80`
`normalizeLong Scalar update`	`1096` ops/sec (`±0.25%`)	`895` ops/sec (`±0.68%`)	`0.82`
`denormalizeLong`	`245` ops/sec (`±5.63%`)	`233` ops/sec (`±6.00%`)	`0.95`
`denormalizeLong Values`	`228` ops/sec (`±4.64%`)	`213` ops/sec (`±4.94%`)	`0.93`
`denormalizeLong donotcache`	`1339` ops/sec (`±0.41%`)	`1002` ops/sec (`±0.64%`)	`0.75`
`denormalizeLong Values donotcache`	`920` ops/sec (`±0.47%`)	`737` ops/sec (`±0.59%`)	`0.80`
`denormalizeLong Scalar donotcache`	`1539` ops/sec (`±1.25%`)	`1073` ops/sec (`±0.13%`)	`0.70`
`denormalizeShort donotcache 500x`	`1731` ops/sec (`±0.15%`)	`1437` ops/sec (`±0.29%`)	`0.83`
`denormalizeShort 500x`	`685` ops/sec (`±5.72%`)	`639` ops/sec (`±6.97%`)	`0.93`
`denormalizeShort 500x withCache`	`8792` ops/sec (`±0.16%`)	`6834` ops/sec (`±5.52%`)	`0.78`
`queryShort 500x withCache`	`4268` ops/sec (`±0.08%`)	`3206` ops/sec (`±0.97%`)	`0.75`
`buildQueryKey All`	`65806` ops/sec (`±0.48%`)	`58478` ops/sec (`±1.39%`)	`0.89`
`query All withCache`	`7853` ops/sec (`±3.03%`)	`5828` ops/sec (`±2.46%`)	`0.74`
`denormalizeLong with mixin Entity`	`222` ops/sec (`±7.47%`)	`209` ops/sec (`±7.50%`)	`0.94`
`denormalizeLong withCache`	`7970` ops/sec (`±1.43%`)	`7517` ops/sec (`±0.32%`)	`0.94`
`denormalizeLong withCache (Scalar churn)`	`7971` ops/sec (`±0.76%`)	`7491` ops/sec (`±0.24%`)	`0.94`
`denormalizeLong Values withCache`	`8261` ops/sec (`±1.33%`)	`5132` ops/sec (`±1.60%`)	`0.62`
`denormalizeLong Scalar withCache`	`9567` ops/sec (`±0.26%`)	`7648` ops/sec (`±0.98%`)	`0.80`
`denormalizeLong Scalar update withCache`	`6824` ops/sec (`±0.50%`)	`4074` ops/sec (`±0.24%`)	`0.60`
`denormalizeLong All withCache`	`8286` ops/sec (`±0.26%`)	`6058` ops/sec (`±0.18%`)	`0.73`
`denormalizeLong Query-sorted withCache`	`8216` ops/sec (`±2.70%`)	`6098` ops/sec (`±1.48%`)	`0.74`
`denormalizeLongAndShort withEntityCacheOnly`	`1929` ops/sec (`±1.45%`)	`1748` ops/sec (`±0.19%`)	`0.91`
`denormalize bidirectional 50`	`4905` ops/sec (`±9.83%`)	`4498` ops/sec (`±10.41%`)	`0.92`
`denormalize bidirectional 50 donotcache`	`54888` ops/sec (`±0.42%`)	`42385` ops/sec (`±1.43%`)	`0.77`
`getResponse`	`6305` ops/sec (`±5.33%`)	`4418` ops/sec (`±4.05%`)	`0.70`
`getResponse (null)`	`12054407` ops/sec (`±0.38%`)	`10236651` ops/sec (`±0.70%`)	`0.85`
`getResponse (clear cache)`	`211` ops/sec (`±9.34%`)	`203` ops/sec (`±7.07%`)	`0.96`
`getSmallResponse`	`4937` ops/sec (`±0.20%`)	`3543` ops/sec (`±0.24%`)	`0.72`
`getSmallInferredResponse`	`3723` ops/sec (`±1.05%`)	`2852` ops/sec (`±1.79%`)	`0.77`
`getResponse Collection`	`6266` ops/sec (`±3.78%`)	`4306` ops/sec (`±4.05%`)	`0.69`
`get Collection`	`4199` ops/sec (`±0.32%`)	`2707` ops/sec (`±0.19%`)	`0.64`
`get Query-sorted`	`7083` ops/sec (`±3.75%`)	`5052` ops/sec (`±1.47%`)	`0.71`
`setLong`	`545` ops/sec (`±0.27%`)	`467` ops/sec (`±0.60%`)	`0.86`
`setLongWithMerge`	`306` ops/sec (`±0.37%`)	`257` ops/sec (`±0.45%`)	`0.84`
`setLongWithSimpleMerge`	`316` ops/sec (`±0.46%`)	`272` ops/sec (`±0.80%`)	`0.86`
`setSmallResponse 500x`	`1104` ops/sec (`±2.22%`)	`926` ops/sec (`±1.48%`)	`0.84`

This comment was automatically generated by workflow using github-action-benchmark.

codecov · 2026-07-19T17:26:44Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.84%. Comparing base (44cb7e6) to head (3412517).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #4034   +/-   ##
=======================================
  Coverage   97.84%   97.84%           
=======================================
  Files         156      156           
  Lines        3057     3057           
  Branches      612      612           
=======================================
  Hits         2991     2991           
  Misses         18       18           
  Partials       48       48

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

cursor · 2026-07-19T17:33:14Z

Verification: `node -e` / `process.argv` indexing in `collect-report.sh`

Checked the reported bug that inline node -e snippets treat process.argv[1] as the first path argument incorrectly (claiming -e occupies argv[1] and script args start at argv[2]).

Result: the issue does not exist — current indexing is correct.

With node -e, -e and the eval string go in process.execArgv, not process.argv. Script arguments start at process.argv[1]:

// node -e '…' foo bar
argv:     ["…/node", "foo", "bar"]
execArgv: ["-e", "…"]

Confirmed on Node 14, 16, 18, and 22 (this package requires >= 22.11.0).

Replayed the exact snippets from examples/benchmark-native/scripts/collect-report.sh against mock sidecar/report files:

Sidecar identity verification (lines 82–90) — OK
Digest/field reads (lines 92–95) — OK
Report provenance attach (lines 187–209) — OK

Shifting to argv[2] as the first path would break these steps (path becomes undefined). No code change needed.

…droid Establish opt-in, provenance-verified baselines for data-client cache GC interaction cost before changing GCPolicy behavior. Co-authored-by: Cursor <cursoragent@cursor.com>

cursor

Stale Bugbot comment from a previous run.

ntucker · 2026-09-13T16:24:12Z

Staff review — CHANGE_THIS_PR

Demo/measurement harnesses look directionally right (timerless BenchmarkGCPolicy, prepare vs timed boundaries, gc/no-gc controls, BuildManifest provenance). One merge blocker before Android baselines are trustworthy:

Release collect cannot retrieve reports on typical physical devices

Authoritative path is release-Hermes on a physical device (README + plan). BenchNativeModule.writeReport writes gc-report.json under private filesDir, and scripts/collect-report.sh only pulls via run-as … cat. The release buildType is not debuggable (AGP default; no override), so run-as fails on normal user builds. Wait can even pass via logcat REPORT_READY, then fail on pull. Emulator/userdebug exceptions don’t count for the mid-range physical gate.

Please fix collect before relying on this harness: write the post-run report to an adb-pullable path (e.g. externalFilesDir) and update collect-report.sh + README. Prefer that over release { debuggable true } so we don’t soften release-Hermes measurement semantics; debuggable true is only a last resort with an explicit caveat (this APK is already debug-signed).

Follow-ups (not blocking)

Align examples/benchmark-native react-native / @react-native/* from 0.86.0 → 0.86.2 to match root/packages/react and drop dual lockfile trees (declared scaffold vs root gradle already tends toward 0.86.2).
No shared harness package yet — plan already says v1 is vocabulary-only; extract after real cross-harness drift.
CircleCI npm pkg delete workspaces[N] correctly adds [7] for this insert, but remains index-fragile; later delete/set by name.
Soften plan wording that implies Android already measures navigation/gestures; v1 is a synthetic interaction/frame probe.
BuildManifest complexity is fine for baseline integrity; consolidate later if the three copies drift.

Bugbot: the older node -e / argv finding is a false positive (already verified in-thread). The tip finding on release run-as is real and matches the change request above.

Adversarial pass: SURVIVE on collect (prefer externalFilesDir); RN pin downgraded to FOLLOW_UP.

Write the post-run report to app-specific external storage and a Downloads mirror so adb pull works on non-debuggable release devices. Co-authored-by: Nathaniel Tucker <me@ntucker.me>

Co-authored-by: Nathaniel Tucker <me@ntucker.me>

ntucker · 2026-09-13T16:31:42Z

Staff review — LGTM (prior CHANGE_THIS_PR cleared)

Tip 66dcaa7 fixes the release-collect blocker: report goes to externalFilesDir, Q+ mirrors via MediaStore Downloads, collect-report.sh adb pulls (no run-as), README documents why not release { debuggable true }, and the script contract test locks it in. Preferable path over softening release debuggability — good.

No new CHANGE_THIS_PR. Prior follow-ups still stand after merge (RN 0.86.0→0.86.2 align, shared harness later, CircleCI workspace index-by-name, plan wording on gestures, BuildManifest consolidate-if-drift). Optional later: if MediaStore insert fails on Q+, still emit REPORT_READY for the externalFilesDir write so wait/pull degrade gracefully — not a merge blocker.

Hold squash until Ready + Bugbot green on this tip + any required E2E you care about for the harness. I will not merge from here.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 3d0077e. Configure here.}

Verify Downloads and externalFilesDir copies are gone before start so a leftover file cannot complete wait for the wrong scenario. Co-authored-by: Nathaniel Tucker <me@ntucker.me>

cursor Bot reviewed Jul 19, 2026

View reviewed changes

github-actions Bot reviewed Jul 19, 2026

View reviewed changes

demo: Add cache GC measurement harnesses across Node, browser, and An…

58e10a4

…droid Establish opt-in, provenance-verified baselines for data-client cache GC interaction cost before changing GCPolicy behavior. Co-authored-by: Cursor <cursoragent@cursor.com>

ntucker force-pushed the demo/gc-benchmark-harnesses branch from 30bdad5 to 58e10a4 Compare September 13, 2026 16:08

cursor Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread examples/benchmark-native/scripts/collect-report.sh Outdated

cursoragent and others added 2 commits September 13, 2026 16:29

fix: Collect release Android GC reports without run-as

66dcaa7

Write the post-run report to app-specific external storage and a Downloads mirror so adb pull works on non-debuggable release devices. Co-authored-by: Nathaniel Tucker <me@ntucker.me>

fix: Ignore comment mentions of run-as in collect-script test

3d0077e

Co-authored-by: Nathaniel Tucker <me@ntucker.me>

cursor Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread examples/benchmark-native/scripts/collect-report.sh

fix: Refuse stale Android GC reports after collect clear

3412517

Verify Downloads and externalFilesDir copies are gone before start so a leftover file cannot complete wait for the wrong scenario. Co-authored-by: Nathaniel Tucker <me@ntucker.me>

Uh oh!

Uh oh!

Conversation

ntucker commented Jul 19, 2026

Summary

Test plan

Uh oh!

vercel Bot commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot Bot commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 19, 2026

Choose a reason for hiding this comment

Node eval argv index wrong

Uh oh!

github-actions Bot commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark React

Uh oh!

github-actions Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark

Uh oh!

codecov Bot commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cursor Bot commented Jul 19, 2026

Verification: node -e / process.argv indexing in collect-report.sh

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ntucker commented Sep 13, 2026

Staff review — CHANGE_THIS_PR

Release collect cannot retrieve reports on typical physical devices

Follow-ups (not blocking)

Uh oh!

ntucker commented Sep 13, 2026

Staff review — LGTM (prior CHANGE_THIS_PR cleared)

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel Bot commented Jul 19, 2026 •

edited

Loading

changeset-bot Bot commented Jul 19, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

github-actions Bot commented Jul 19, 2026 •

edited

Loading

github-actions Bot left a comment •

edited

Loading

github-actions Bot left a comment •

edited

Loading

codecov Bot commented Jul 19, 2026 •

edited

Loading

Verification: `node -e` / `process.argv` indexing in `collect-report.sh`

cursor Bot left a comment •

edited

Loading