Skip to content

feat(snapshot): disclose a cut capture on every platform; raise the iOS bridge node cap to 5000 - #2510

Open
thymikee wants to merge 1 commit into
mainfrom
fix/snapshot-truncation-disclosure
Open

feat(snapshot): disclose a cut capture on every platform; raise the iOS bridge node cap to 5000#2510
thymikee wants to merge 1 commit into
mainfrom
fix/snapshot-truncation-disclosure

Conversation

@thymikee

Copy link
Copy Markdown
Member

Part of #2492 (task 5). Two changes, both measured first.

One disclosure for a cut capture, every platform

Every backend already reports the same fact — truncated: true on the snapshot — but only in JSON. The text path said nothing: an agent read a screen missing its footer, tab bar, or the items after a long list as complete. The backends walk the tree in document order, so what falls off is what comes last, on screen or not.

truncatedCaptureWarning in capture-kit renders one warning from that flag, and the single cross-platform warnings assembly in the snapshot runtime calls it. Android helper (5000-node cap), iOS Simulator bridge, XCTest runner, and the web provider all flow through it. The limit and dimension stay backend-side; the copy names neither, and tells the agent what to do: navigate or scroll so fewer elements render, re-run, use screenshot for the rest.

Producers that cannot observe truncation keep truncated unknown (#2188 invariant 5) and get no warning.

iOS Simulator bridge node cap 1500 → 5000

Measured on iPhone 17 Pro with a synthetic unvirtualized list in the fixture dev client (8 regular snapshots per cell, bridge acquire duration from --debug diagnostics):

tree cap 1500 3000 5000 10000
600 rows (~3.5k raw nodes) 969 ms, truncated 976 ms, truncated 962 ms, complete 970 ms, complete
1500 rows (~8k raw nodes) 2378 ms, truncated 2344 ms, truncated 2404 ms, complete

The native read fetches the whole tree regardless of the cap; the cap only stops conversion. Cost is the tree itself (~0.25 ms per raw node). At 1500 the screen's on-screen footer (drawn last) was absent; at 5000 present. 5000 matches the Android helper. The 4 MB response bound did not bite at 8k nodes.

Not here

…OS bridge node cap to 5000

Every backend sets truncated: true when it cuts a capture at one of its
limits, but only JSON carried it. One shared warning now renders from that
flag in the cross-platform warnings assembly and tells the agent what fell
off (what comes last in document order) and what to do.

The iOS Simulator AX bridge cap moves from 1500 to 5000 nodes, the Android
helper's bound. Measured on a synthetic 600-row screen, acquisition time did
not change with the cap while the 1500 cut dropped the on-screen footer.
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-2510/

Built to branch gh-pages at 2026-09-12 05:58 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.50 MB 4.50 MB +407 B
Package (unpacked) 4.50 MB 4.50 MB +407 B
Package (download) 1.32 MB 1.32 MB +162 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.2 ms 26.5 ms +0.3 ms
CLI --help 74.0 ms 73.6 ms -0.4 ms

@thymikee

Copy link
Copy Markdown
Member Author

The new warning misclassifies incomplete captures at a552b16. quality-warnings.ts:43-47 treats every truncated flag as a backend-limit cut, but Android presentation failures and iOS accessibility-unavailable responses also set that flag. Those failures now get misleading advice about missing later elements and scrolling. Use wording that describes incomplete capture without inventing its cause, or select limit-specific advice from typed evidence; cover a real sparse/presentation-failure result. Current-head checks pass and there are no conflicts. The separate bridge publish-versus-fallback policy remains deferred, as the description states.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant