Skip to content

feat(fold): pose a foldable iPhone simulator through Device Hub and verify it with CoreDevice - #2717

Open
thymikee wants to merge 2 commits into
mainfrom
feat/foldable-pose
Open

thymikee wants to merge 2 commits into
mainfrom
feat/foldable-pose

Conversation

@thymikee

Copy link
Copy Markdown
Member

Summary

Adds agent-device fold <closed|half-open|open> and client.command.fold({ pose }) for the iPhone Duo simulator. ADR 0025 concluded no official host API sets the hinge pose and left it to the operator. This PR sets it the way the operator does: the macOS helper presses the pose control in the Xcode Device Hub window through accessibility (the earlier "zero accessibility nodes" finding was System Events seeing the trampolined app as pid 0), and the Apple owner trusts only CoreDevice's read-back of devicectl device motion hinge-angle. closed and open are the hinge end stops; half-open is reported only once two consecutive reads agree the hinge stopped, because the first read after pressing Book caught it at 175.1°.

The helper resolves Device Hub from the process table, reopens its window when a headless boot left none, and selects the simulator through the sidebar row keyed TableRow.Device.<udid>, so twins sharing a name cannot be confused. The response reports the verified pose, the angle, and the lit panel's point size, so an agent sees its refs are stale.

agent-device fold open        # Folded to open (hinge 180°, LCD-1 lit at 669x951pt); refs ... are stale
agent-device fold half-open   # Folded to half-open (hinge 130°, ...)
await client.command.fold({ pose: 'closed' })

setFoldPose is a host-side device-runtime operation (no runner); every other platform and provider states a refusal cell. 70 files touched; the gross diff exceeds the 1,000-line budget because of the ADR update, tests, and the Swift helper subcommand.

Validation

Tested at 37e7778: pnpm check:affected --run passes (format, lint, typecheck, layering, di-seams, fallow, mcp-metadata, build, package, integration-node, macos-coverage, vitest-related 698 files, integration-progress, replay-compat, daemon-wire-compat, affected-selector).

Live, through the built CLI on an iOS 27.1 iPhone Duo simulator booted headlessly (Device Hub showed no window for it): open → 180° on LCD-1, screenshot 951x669; half-open → 130°; closed → 0° on LCD, screenshot 466x678; each 10–16s. An iPhone 17 simulator refuses with UNSUPPORTED_OPERATION (single-panel-device) before anything is pressed. Not exercised: a second Xcode running its own Device Hub, and physical foldables (refused by the leaf fact).

🤖 Generated with Claude Code

thymikee and others added 2 commits September 20, 2026 22:41
…erify it with CoreDevice

Add `agent-device fold <closed|half-open|open>` and `client.command.fold({ pose })`.
ADR 0025 recorded that no official host API sets the hinge pose and left it to the
operator. The operator's own control is the pose action bar in the Xcode Device Hub
window, whose controls are ordinary accessibility buttons; the macOS helper gains a
`device-hub pose` subcommand that finds Device Hub in the process table (LaunchServices
registers the trampolined app with pid -1, which is why System Events saw no nodes),
reopens its window with a reopen event when it shows none, selects the simulator through
the sidebar row keyed `TableRow.Device.<udid>`, and presses the control.

The press is not the evidence. The Apple owner reads the hinge angle back through
`devicectl device motion hinge-angle` until it agrees with the request: closed is 0°, open
is 180°, and half-open is any angle between them reported only once two consecutive reads
agree the hinge stopped moving, because a hinge sweeps through half-open angles on its way
to either stop (the live run read 175.1° one stream after pressing Book, 130° after it
settled). The stream never ends on its own, so each read is bounded by devicectl's smallest
`--timeout` of five seconds. The response carries the verified pose, the angle, and the lit
panel's point size, so an agent sees its refs are stale without another capture.

`setFoldPose` is a host-side device-runtime operation rather than an interactor method: it
needs no XCUITest runner, and the leaf fact admits iPhone/iPad simulators while the
operation itself refuses a single-panel simulator from CoreDevice's display table. Every
other platform and provider states its own refusal cell.

Verified through the built CLI on an iOS 27.1 iPhone Duo simulator: open → 180° on LCD-1
(669x951pt, screenshot 951x669), half-open → 130°, closed → 0° on LCD (466x678pt,
screenshot 466x678), from a Device Hub that showed no window for the device; an iPhone 17
refuses with UNSUPPORTED_OPERATION before anything is pressed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… and conformance tables

The contracts export snapshot names the new `fold-runtime` subpath, the six-platform
coverage declaration and its Android evidence row classify the command, the runtime-binding
conformance table resolves it, and the provider output guard drives it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-2717/

Built to branch gh-pages at 2026-09-20 20:46 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.67 MB 4.69 MB +22.8 kB
Package (unpacked) 4.67 MB 4.69 MB +22.8 kB
Package (download) 1.39 MB 1.40 MB +6.9 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 17.0 ms 17.1 ms +0.1 ms
CLI --help 49.1 ms 45.9 ms -3.2 ms

@thymikee

Copy link
Copy Markdown
Member Author

This looks close, but one refusal path contradicts itself and the coverage gate is failing because of where the pose vocabulary got exported.

At 37e7778, awaitHingePose in packages/platform-apple/src/foldable/pose.ts refuses once the 4-attempt budget runs out, even when the last reading already maps to the requested pose (https://github.com/callstack/agent-device/blob/37e7778/packages/platform-apple/src/foldable/pose.ts#L129). For fold half-open, the loop needs two consecutive reads within 0.5 deg, and the ADR's own live sequence (180 -> 175.1 -> 130 -> 130) spends all 4 attempts to get there, so one extra stale read anywhere in that chain ends the loop with observedPose === requestedPose === 'half-open' and the command still reports COMMAND_FAILED, naming the very angle it asked for. The test in pose.test.ts locks this in: it reads 170/150/120/90 and asserts a refusal at hingeAngleDegrees: 90, which is itself half-open. Can you state the rule directly — a fold refusal must never report an observed pose equal to the requested pose — and either return success when budget exhaustion lands on the requested pose, or give "right pose, still settling" its own reason and message, with the test split to cover both cases?

The Coverage job fails on 11 eager-closure-budget assertions, all tracing to one new edge: packages/contracts/src/facades/device.ts -> packages/contracts/src/fold-pose.ts (https://github.com/callstack/agent-device/blob/37e7778/packages/contracts/src/facades/device.ts). That pulls fold-pose exports onto the import path of packages/platform-android/src/mechanics.ts, provider-limrun, provider-webdriver, session-journal, and src/cli.ts, none of which can pose a hinge. Can the pose vocabulary move to a module those entries already evaluate, or be reached only through the @agent-device/contracts/fold-runtime subpath this PR already adds, so only fold's actual readers pay for it?

Is the Apple-only per-platform seam here the same shape as actionButton, or does fold need something narrower — PlatformRuntimeOperations is a total union, every public command declares a device-runtime use, and the Android/webdriver/limrun one-liners look like the existing totality rule rather than new surface? If so the seam itself isn't the problem, just its placement above.

Not blocking, take or leave: parseHingeAngleSample returns the first Angle: line of the devicectl stream rather than the last, so every read can be about five seconds stale — which is also what consumes the retry budget behind the refusal above (https://github.com/callstack/agent-device/blob/37e7778/packages/platform-apple/src/core/hinge-angle.ts#L65). Smaller ones alongside it: SetFoldPoseInput.execution is copied from the orientation runtime but nothing reads it, appleFoldFact gates on hasAppleActionButton() which only coincides with hinge-capable devices today, fold inherits the generic 90s timeout although its own worst case can sum past it, the Android refusal says emulators "expose no hinge control" when foldable AVDs do have a posture control this project simply does not drive yet, the daemon test for an invalid pose asserts a bare .rejects.toThrow() with no error code, and noInventedPoseCommand still forbids the unfold/unfolded aliases that parseFoldPose now accepts.

The PR body says 70 files touched where the diff shows 78, and describes pnpm check:affected --run as passing at a head where Coverage is failing on the eager-closure edge above; both are worth correcting so the record matches the current state.

I could not run anything on macOS, so the Device Hub press, the trampolined-pid discovery, the reopen, and the sidebar row selection are unverified from the diff, and the 313-line Swift helper has no test covering them. The live angles and timings in the PR body are narrative without an attached run, so I checked the code paths they describe but not the readings themselves. Line references above are pinned to 37e7778.

Before this merges, the fold-pose exports need to move off packages/contracts/src/facades/device.ts so Coverage passes without rebaselining the budget, and the half-open refusal needs to stop firing when the observed angle already matches the requested pose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant