Conversation
The computer-use API has shipped in all three SDKs but never reached the CLI, so the only way to drive a sandbox desktop from a shell was to talk to the REST API by hand. The Claude Code plugin does exactly that in its cos driver, which is the one place it bypasses this binary — and the other integrations, which only shell out, cannot do it at all. `sandbox desktop` turns on ingress, waits for the desktop stack to come up, and prints a noVNC link. `sandbox computer` drives that desktop: screenshot, screen, cursor, windows, move, click, type, key, open, plus a hidden raw escape hatch for the routes not wrapped here. Three things carried over from cos because each one costs a debugging session to rediscover: - The desktop stack starts after the sandbox reports running, and nothing upstream polls for it, so every caller writes the wait itself. - fc answers 409 desktop_unavailable both while the desktop is booting and when an action fails on a live desktop. ComputerError.Retryable encodes which codes a waiter should keep trying, so the readiness wait neither gives up on a booting desktop nor spins on a permanent failure. - Driving a non-desktop image fails early with the fix, rather than a bare 501 from the first call. Flags are re-scanned by hand because urfave stops parsing them at the first positional argument, so `computer screenshot my-box --out shot.png` otherwise writes to the default path without reporting anything wrong — the same workaround `sandbox edit` makes for --ingress. `--out` has no short alias: -o is taken by the global output-format flag. Verified against a live desktop:1 sandbox: link minted, Chrome driven to a page by chord, typing and Return, pointer landed on the target link, and every error path checked.
`sandbox desktop my-box --screen screen-1` dropped --screen, and --wait with it, because urfave stops parsing flags at the first positional. The computer subcommands already re-scanned their arguments by hand; desktop read its flags straight off the context and so kept the bug. Route desktop through the same parser and teach it --wait. A duration it cannot parse now keeps the declared default instead of zeroing, which would have turned the readiness wait into a single attempt. Tests cover both flag positions, the equals form, the short alias, and operands surviving around a flag.
CI runs golangci-lint, which this was not developed against. Two classes: a type assertion on an error in the test, which breaks once anything wraps it, and three `err` shadows in runDesktop. Assert with errors.As, and assign to the existing err instead of redeclaring it.
pratikbin
added a commit
to NodeOps-app/createos-plugins
that referenced
this pull request
Sep 16, 2026
The CLI grew `sandbox desktop` and `sandbox computer`, so the REST layer this driver carried for them has no reason to exist. Delete api(), api_auth(), api_check() and desktop_wait(), and shell out like every other verb. That layer was the one place cos bypassed the CLI, and the cost of it was a second implementation of things the CLI already knew: its own auth precedence, its own error-code map, its own readiness poll. It also held a false premise — a comment claiming the CLI does not read CREATEOS_API_KEY, which it has (cmd/root/root.go). The same code now serves pi, opencode and herdr, which could not do computer-use at all. cos keeps what the CLI cannot know: which box belongs to this project, and creating one on a desktop image when there is none. The rootfs guard stays too, because the CLI tells you to create a new sandbox, and for a project box the fix is to replace the one you have. Two details preserved for callers: - `cos computer screenshot -o file` still works. The CLI spells it --out, since -o is its global output-format flag, so cos translates. - `cos resume` now says to re-run `cos desktop` on a box that had one. A resumed desktop answers reads before its input stack is back, so a click can 409 for a moment after the screen route already succeeds. Requires a createos CLI carrying those commands. cos probes for them and tells the user to upgrade rather than failing with "unknown command". Blocked on NodeOps-app/createos-cli#94.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
createos sandbox desktopandcreateos sandbox computer— the computer-use surface that already exists in all three SDKs but never reached the CLI.Why
Computer-use shipped in
fc-sdk,createos-go-sdkandcreateos-python-sdk, but not here. That left the CLI the only client that couldn't drive a desktop — so the Claude Code plugin'scosdriver talks to the REST API directly for these two verbs, the single place it bypasses this binary. It carries its own auth precedence and error-code map to do it.Every other integration (
pi-extension,opencode-plugin,herdr-plugin) only shells out to this CLI, so none of them can do computer-use at all today. This closes that.Follows ADR-0001 in
createos-plugin, which records the intent to fold the driver's engine into this CLI and keep the plugins thin.Commands
sandbox desktop [<sandbox>]computer screenshotcomputer screen/cursor/windowscomputer move/clickcomputer type/keykeypresses a chordcomputer opencomputer rawThree behaviors carried over from
cosEach of these costs a debugging session to rediscover, so they're encoded rather than left to callers:
running, and nothing upstream polls for it. Every caller ends up writing this wait; now it's insandbox desktop.desktop_unavailableboth while the desktop is booting and when an action fails on a healthy desktop.ComputerError.Retryable()encodes which codes a waiter should keep retrying, so the wait neither abandons a booting desktop nor spins for the full timeout on a permanent failure. This is what the tests pin.Flag parsing
parseComputerArgsre-scans arguments by hand because urfave/cli v2 stops parsing flags at the first positional. Without it,computer screenshot my-box --out shot.pngsilently writes to the default path and reports success — found while testing. Same workaroundsandbox editalready makes for--ingress.--outdeliberately has no short alias:-ois the global output-format flag.Verification
Against a live
desktop:1sandbox (created, driven, destroyed):desktop→ ingress enabled, readiness wait, noVNC link with expiry and a "anyone with this link can control it" warningdevbox:1sandbox before touching ingressscreen1280×800,cursor,windows(28)screenshotverified by eye — Chrome on example.comopen→key ctrl l→type→key Returnnavigated toexample.com/index.htmlmove 300 337landed on the target link (hand cursor, status bar confirmed)raw GET clipboardreaches unwrapped routesWire format cross-checked against
createos-go-sdkrather than trustingcosalone:screen_id,/computer/screen,/cursor,/mouse/move,/mouse/click,/keyboard/type{text},/keyboard/press{keys},/open{target},/screens/{id}/connectall match.gofmt,go vetclean; 71 tests pass.golangci-lintandgosecwere not available on the machine this was developed on.Draft — open questions
GET/PUT /computer/clipboardand it works (raw GET clipboardreturns{"text":""}). For agent use it beatstypefor long strings — one call instead of per-character input. Worth a first-classcomputer clipboardcommand in this PR, or separate?desktopcreate sandboxes?cos desktopcreates one if none exists, but that relies on its per-repo statefile, which is plugin state and shouldn't move into a stateless CLI. Currentlycreate --rootfs desktop:1thendesktop <ref>. A--createflag would restore the one-liner../screenshot.png.cosused its state directory; the CLI has none.Mesh
coscan dropapi()/api_auth()and shell out like every other verb