From d20d764d9e83de10f863ca6fd5b90cadbb7b1d14 Mon Sep 17 00:00:00 2001 From: pratikbin <68642400+pratikbin@users.noreply.github.com> Date: Wed, 16 Sep 2026 13:21:51 +0530 Subject: [PATCH] refactor(cos): delegate desktop and computer to the createos CLI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CLI grew `sandbox desktop` and `sandbox computer`, so the REST layer this driver carried for them has no reason to exist. Delete api(), api_auth(), api_check() and desktop_wait(), and shell out like every other verb. That layer was the one place cos bypassed the CLI, and the cost of it was a second implementation of things the CLI already knew: its own auth precedence, its own error-code map, its own readiness poll. It also held a false premise — a comment claiming the CLI does not read CREATEOS_API_KEY, which it has (cmd/root/root.go). The same code now serves pi, opencode and herdr, which could not do computer-use at all. cos keeps what the CLI cannot know: which box belongs to this project, and creating one on a desktop image when there is none. The rootfs guard stays too, because the CLI tells you to create a new sandbox, and for a project box the fix is to replace the one you have. Two details preserved for callers: - `cos computer screenshot -o file` still works. The CLI spells it --out, since -o is its global output-format flag, so cos translates. - `cos resume` now says to re-run `cos desktop` on a box that had one. A resumed desktop answers reads before its input stack is back, so a click can 409 for a moment after the screen route already succeeds. Requires a createos CLI carrying those commands. cos probes for them and tells the user to upgrade rather than failing with "unknown command". Blocked on NodeOps-app/createos-cli#94. --- packages/claude-code-plugin/README.md | 2 +- packages/claude-code-plugin/scripts/cos | 209 ++++-------------- .../skills/using-createos-sandbox/SKILL.md | 35 +-- packages/codex-plugin/scripts/cos | 209 ++++-------------- .../skills/using-createos-sandbox/SKILL.md | 35 +-- 5 files changed, 103 insertions(+), 387 deletions(-) diff --git a/packages/claude-code-plugin/README.md b/packages/claude-code-plugin/README.md index 49d465c..81140c4 100644 --- a/packages/claude-code-plugin/README.md +++ b/packages/claude-code-plugin/README.md @@ -266,7 +266,7 @@ The two halves compose: the URL lets **you** watch and take over in a browser wh | **Raw pixels** | Coordinates are unscaled X11 pixels of that screen. Read the bounds from `cos computer screen`. | | **Needs ingress** | `desktop` enables it for you. `unexpose` turns it off and kills the link. | -> `desktop` and `computer` are the only `cos` commands that call the CreateOS REST API directly — the `createos` CLI has no computer or desktop command yet. Everything else shells out to the CLI as usual. Auth is reused as-is: `CREATEOS_API_KEY` or `~/.createos/.token` go out as `X-Api-Key`, a browser session's JWT as `X-Access-Token`. +> `desktop` and `computer` shell out to `createos sandbox desktop` and `createos sandbox computer`, like every other `cos` verb. `cos` adds only the project-box resolution: which box belongs to this directory, and creating one on a desktop image when there is none. Ops `cos` does not wrap — clipboard, window manipulation, screen resize — are reachable with `createos sandbox computer --help`. These commands need a `createos` CLI new enough to carry them; `cos` checks and tells the user to upgrade if not. ``` /createos-sandbox:desktop [-s shape] [-S screen-N] diff --git a/packages/claude-code-plugin/scripts/cos b/packages/claude-code-plugin/scripts/cos index d9d6a9a..331bf21 100755 --- a/packages/claude-code-plugin/scripts/cos +++ b/packages/claude-code-plugin/scripts/cos @@ -698,96 +698,23 @@ cmd_unexpose(){ } # ══════════════════════════ desktop / computer use ════════════════════════════ -# `createos` has no computer or desktop command, so these two verbs are the only -# place cos talks to the sandbox REST API directly instead of shelling out to the -# CLI. Everything else in this script stays CLI-driven; when the CLI grows a -# `sandbox computer` group, delete api()/api_auth() and shell out like the rest. +# Both verbs shell out to `createos sandbox desktop` and `createos sandbox +# computer`, like every other verb here. The REST layer this script used to +# carry is gone: the CLI owns the readiness wait, the error-code mapping and +# the noVNC link now, so every host that shells out gets them, not just this one. # -# Auth mirrors the CLI exactly: an api key goes in X-Api-Key, an OAuth JWT in -# X-Access-Token. fc rejects Bearer on user-facing routes, and it rejects a JWT -# sent under X-Api-Key ("invalid api key") — the two are not interchangeable. -api_base(){ printf '%s' "${CREATEOS_SANDBOX_URL:-https://api.sb.createos.sh}"; } - -api_auth(){ - # Precedence MUST match the CLI's (createos-cli cmd/root/root.go): an OAuth - # session wins, an api key is the fallback. Inverting it authenticates these - # direct calls as a different identity than every CLI-driven verb — and the - # symptom is a 404 on a box cos itself just created, which reads like the box - # is missing rather than like an auth mismatch. - local exp at - if [ -f "$CREATEOS_DIR/.oauth" ]; then - exp=$(jq -r '.expires_at // 0' "$CREATEOS_DIR/.oauth" 2>/dev/null || echo 0) - numeric "$exp" || exp=0 - # cos deliberately does not implement OAuth refresh. The CLI already refreshes - # in its pre-flight and rewrites ~/.createos/.oauth, so poke it and re-read - # rather than carrying a second, subtly different refresh implementation. - if [ "$(date +%s)" -ge "$((exp - 60))" ]; then - "$CLI" -o json sandbox ls >/dev/null 2>&1 || true - fi - at=$(jq -r '.access_token // empty' "$CREATEOS_DIR/.oauth" 2>/dev/null || true) - [ -n "$at" ] && { printf 'X-Access-Token: %s' "$at"; return 0; } - fi - # NOTE: the CLI itself does NOT read CREATEOS_API_KEY (it has no such env var), - # so this branch works for these REST calls but not for any CLI-driven verb. - [ -n "${CREATEOS_API_KEY:-}" ] && { printf 'X-Api-Key: %s' "$CREATEOS_API_KEY"; return 0; } - [ -f "$CREATEOS_DIR/.token" ] && { printf 'X-Api-Key: %s' "$(tr -d '\r\n' <"$CREATEOS_DIR/.token")"; return 0; } - die "not signed in — run 'cos auth'" -} - -# Map the computer API's error codes onto something actionable. Worth doing by -# hand: `desktop_unavailable` is fc's catch-all for every X failure, so the raw -# message alone never tells you whether the desktop is still booting or the -# action itself failed on a live desktop. -api_check(){ local code=$1 out=$2 what=$3 msg - case "$code" in 2??) return 0;; esac - msg=$(printf '%s' "$out" | jq -r '.message // .error // empty' 2>/dev/null || true) - case "$code" in - 000) die "no response from $(api_base) — network down, or CREATEOS_SANDBOX_URL points somewhere wrong";; - 401|403) die "auth rejected (HTTP $code). The API key or browser session is invalid or expired. - Ask the user to re-run 'createos login' in their own terminal, or export CREATEOS_API_KEY.";; - 404) die "not found (HTTP 404): ${msg:-$what} - Either the box is gone, or it has no such screen — computer-use needs a desktop image ('cos desktop').";; - 409) case "$msg" in - *ingress*) die "409: $msg — 'cos desktop' turns ingress on for you";; - *desktop_unavailable*|*) die "desktop did not answer (HTTP 409 ${msg:-desktop_unavailable}). - fc returns this both while the desktop is still booting AND when an action fails on a live desktop. - If the box just came up, 'cos desktop' waits for readiness — run that first.";; - esac;; - 429) die "rate limited (429) — the control plane caps concurrent screenshots. Retry in a second.";; - 501) die "501 desktop_tools_unavailable — this rootfs has no desktop tools installed. Recreate with: cos down && cos desktop";; - *) die "API error HTTP $code on $what${msg:+: $msg}";; - esac -} - -# api [json-body] → prints the response payload, unwrapped from -# fc's JSend envelope. Callers pipe it through jq for the fields they want. -api(){ local method=$1 path=$2 body=${3:-} hdr code out tmp - hdr=$(api_auth) || exit 1 - tmp=$(mktemp) - local args=(-sS -X "$method" -H "$hdr" -o "$tmp" -w '%{http_code}' --max-time 60) - [ -n "$body" ] && args+=(-H 'Content-Type: application/json' -d "$body") - code=$(curl "${args[@]}" "$(api_base)$path" 2>/dev/null) || code=000 - out=$(cat "$tmp"); rm -f "$tmp" - api_check "$code" "$out" "$method $path" - printf '%s' "$out" | jq -c '.data // .' 2>/dev/null || printf '%s' "$out" -} - -# The desktop stack (Xvfb → XFCE → x11vnc → websockify) starts AFTER the box -# reports `running`, so every computer call 404s or 409s for the first while. -# Neither fc nor the SDK polls for this — every caller has to, so cos does it here. -desktop_wait(){ local id=$1 screen=$2 to=${3:-120} hdr code i=0 said=0 - hdr=$(api_auth) || exit 1 - while [ "$i" -lt "$to" ]; do - code=$(curl -sS -o /dev/null -w '%{http_code}' --max-time 10 -H "$hdr" \ - "$(api_base)/v1/sandboxes/$id/computer/screen?screen_id=$screen" 2>/dev/null) || code=000 - case "$code" in - 2??) [ "$said" = 1 ] && echo "cos: desktop ready" >&2; return 0;; - 401|403|501) api_check "$code" '' "desktop readiness";; - esac - [ "$said" = 0 ] && { echo "cos: waiting for the desktop stack to come up (up to ${to}s)…" >&2; said=1; } - sleep 2; i=$((i+2)) - done - die "desktop did not come up within ${to}s on $id ($screen) — check: cos run 'pgrep -a Xvfb; pgrep -a websockify'" +# cos keeps only what the CLI cannot know: which box belongs to this project, +# and how to create one on a desktop image when there is none. + +# The CLI grew these commands after this script shipped, so probe for them +# rather than failing with a bare "unknown command". Same shape as the +# `sandbox sync --mode` probe further down. +require_cli_computer(){ + "$CLI" sandbox computer --help >/dev/null 2>&1 && return 0 + die "the installed createos CLI has no 'sandbox computer' command. + Desktop and computer-use moved out of cos and into the CLI. Upgrade it: + createos upgrade + or re-run the installer from https://createos.sh" } # ───────────────────────────── desktop: graphical box + a live noVNC URL @@ -796,9 +723,10 @@ cmd_desktop(){ local shape=s-2vcpu-4gb screen=screen-0; local OPTIND=1 o while getopts "s:S:h" o; do case $o in s) shape=$OPTARG;; S) screen=$OPTARG;; - h) echo "cos desktop [-s shape] [-S screen-N] desktop:1 box + ingress + live noVNC URL"; return 0;; + h) echo "cos desktop [-s shape] [-S screen-N] desktop:1 box + a live noVNC URL"; return 0;; *) die "usage: cos desktop [-s shape] [-S screen-N]";; esac; done shift $((OPTIND-1)) + require_cli_computer local id; id=$(state_get id) if [ -z "$id" ] || ! box_live "$id"; then @@ -807,8 +735,9 @@ cmd_desktop(){ id=$(state_get id); [ -n "$id" ] || die "box creation did not record an id" else box_resume_if_paused "$id" - # Never silently drive a non-desktop box: every computer call would 501 with - # a much less obvious message than saying so here. + # The CLI refuses a non-desktop box too, but it tells you to create a new + # sandbox. That is the wrong move for a project box — there is one per + # directory, so the fix is to replace this one. local rf; rf=$("$CLI" -o json sandbox get "$id" 2>/dev/null | jq -r '.rootfs // empty') case "$rf" in *desktop*) :;; @@ -818,36 +747,26 @@ cmd_desktop(){ fi wait_running "$id" 30 || die "box $id not running" - "$CLI" sandbox edit "$id" --ingress on >/dev/null 2>&1 || die "failed to enable ingress on $id" - desktop_wait "$id" "$screen" - - local conn url exp - conn=$(api GET "/v1/sandboxes/$id/computer/screens/$screen/connect") - url=$(printf '%s' "$conn" | jq -r '.url // empty') - exp=$(printf '%s' "$conn" | jq -r '.expires_at // "?"') - [ -n "$url" ] || die "connect returned no URL — ingress is off on $id (fc only mints one when ingress is enabled)" state_set desktop_screen "$screen" - { - echo "cos: desktop $screen on $id — open in a browser:" - echo "cos: ⚠ anyone with this link can drive the desktop. The token expires $exp, and re-running" - echo "cos: 'cos desktop' mints a fresh one (which invalidates this link for NEW connections)." - } >&2 - echo "$url" + # The CLI turns ingress on, waits for the desktop stack, and prints the URL + # together with its expiry and the warning that the link is the credential. + "$CLI" sandbox desktop "$id" --screen "$screen" } # ───────────────────────────── computer: drive that desktop from the agent side computer_usage(){ cat <<'EOF' cos computer drive the desktop in the project box (needs `cos desktop` first) - screenshot [-o file] capture PNG (default: $STATE_DIR/screenshot.png), prints the path + screenshot [-o file] capture PNG, prints the path screen screen geometry {width,height} cursor cursor position {x,y} + windows list windows on the screen move move the pointer click [ ] click (optionally move there first) type type a string key ... press a chord, e.g. cos computer key ctrl l open open a target in the desktop browser - windows list windows on the screen - raw [json] any other computer endpoint (path relative to .../computer) +Each op runs `createos sandbox computer ` against this project's box; run +`createos sandbox computer --help` for the ops not listed here. Screen defaults to screen-0; override per call with COS_SCREEN=screen-N. Coordinates are raw X11 pixels of that screen — match them against `cos computer screen`. EOF @@ -856,66 +775,22 @@ EOF cmd_computer(){ local op=${1:-}; shift || true case "$op" in ''|-h|--help|help) computer_usage; return 0;; esac + require_cli_computer local id; id=$(state_get id) [ -n "$id" ] || die "no active box — run 'cos desktop' first" [ "$(box_status "$id")" = paused ] && die "box $id is paused — 'cos resume' first" local screen=${COS_SCREEN:-$(state_get desktop_screen)}; screen=${screen:-screen-0} - local base="/v1/sandboxes/$id/computer" q="screen_id=$screen" - - case "$op" in - screenshot) - local out="$STATE_DIR/screenshot.png" hdr code - [ "${1:-}" = "-o" ] && { out=${2:?-o needs a path}; shift 2; } - hdr=$(api_auth) || exit 1 - # Screenshot is the one computer route that returns bytes, not JSON, so it - # bypasses api() entirely — the PNG is passed through from fc verbatim. - code=$(curl -sS -H "$hdr" -o "$out" -w '%{http_code}' --max-time 60 \ - "$(api_base)$base/screenshot?$q" 2>/dev/null) || code=000 - case "$code" in - 2??) :;; - *) local body; body=$(cat "$out" 2>/dev/null || true); rm -f "$out"; api_check "$code" "$body" "screenshot";; - esac - echo "cos: screenshot → $out ($(wc -c <"$out" | tr -d ' ') bytes) — open it with the Read tool" >&2 - echo "$out";; - screen) api GET "$base/screen?$q";; - cursor) api GET "$base/cursor?$q";; - windows) api GET "$base/windows?$q";; - move) - numeric "${1:-}" && numeric "${2:-}" || die "usage: cos computer move " - api POST "$base/mouse/move?$q" "$(jq -nc --argjson x "$1" --argjson y "$2" '{x:$x,y:$y}')" >/dev/null - echo "cos: moved to $1,$2" >&2;; - click) - if [ $# -ge 2 ]; then - numeric "$1" && numeric "$2" || die "usage: cos computer click [ ]" - api POST "$base/mouse/click?$q" "$(jq -nc --argjson x "$1" --argjson y "$2" '{x:$x,y:$y}')" >/dev/null - echo "cos: clicked $1,$2" >&2 - else - api POST "$base/mouse/click?$q" '{}' >/dev/null - echo "cos: clicked at the current cursor position" >&2 - fi;; - type) - [ $# -ge 1 ] || die "usage: cos computer type " - # Unquoted multi-word text arrives as separate argv entries; join before - # sending so `cos computer type hello world` types the space too. - local text="$*" - api POST "$base/keyboard/type?$q" "$(jq -nc --arg t "$text" '{text:$t}')" >/dev/null - echo "cos: typed ${#text} chars" >&2;; - key) - [ $# -ge 1 ] || die "usage: cos computer key ... e.g. cos computer key ctrl l" - api POST "$base/keyboard/press?$q" "$(jq -nc '{keys:$ARGS.positional}' --args "$@")" >/dev/null - echo "cos: pressed $*" >&2;; - open) - [ $# -ge 1 ] || die "usage: cos computer open " - api POST "$base/open?$q" "$(jq -nc --arg t "$1" '{target:$t}')" >/dev/null - echo "cos: opened $1" >&2;; - raw) - # Escape hatch: cos wraps the handful of ops an agent loop actually needs, - # not all ~30 computer routes. Everything else goes through here. - local m=${1:?usage: cos computer raw [json]} p=${2:?path required}; shift 2 - case "$p" in /*) :;; *) p="$base/$p";; esac - api "$m" "$p" "${1:-}";; - *) die "unknown computer op '$op' — run 'cos computer help'";; - esac + + # cos has always spelled the screenshot path -o. The CLI spells it --out, + # because -o is the CLI's global output-format flag. Translate rather than + # break every caller that already passes -o. + if [ "$op" = screenshot ]; then + local -a mapped=(); local a + for a in ${@+"$@"}; do case "$a" in -o) mapped+=(--out);; *) mapped+=("$a");; esac; done + set -- ${mapped[@]+"${mapped[@]}"} + fi + + "$CLI" sandbox computer "$op" "$id" --screen "$screen" ${@+"$@"} } # ───────────────────────────── cluster: N boxes on one private network (by-name DNS) @@ -1091,6 +966,10 @@ cmd_resume(){ esac box_resume_if_paused "$id" echo "cos: $id running. Re-run 'cos sync' / 'cos tunnel' / 'cos expose' if you had them up." >&2 + # A resumed desktop box answers reads (screen, screenshot) before its input + # stack is back, so a click can still 409 for a moment. `cos desktop` waits + # and mints a fresh link — the old one is expired anyway. + [ -n "$(state_get desktop_screen)" ] && echo "cos: this box had a desktop — re-run 'cos desktop' before driving it again." >&2 } # ───────────────────────────── template: build a custom rootfs from a Dockerfile diff --git a/packages/claude-code-plugin/skills/using-createos-sandbox/SKILL.md b/packages/claude-code-plugin/skills/using-createos-sandbox/SKILL.md index 3969c3d..53a87e2 100644 --- a/packages/claude-code-plugin/skills/using-createos-sandbox/SKILL.md +++ b/packages/claude-code-plugin/skills/using-createos-sandbox/SKILL.md @@ -169,42 +169,21 @@ cos computer open https://example.com cos computer click 640 400 cos computer type 'hello' cos computer key ctrl l # a chord -cos computer help # every op, plus `raw` for the rest of the API +cos computer help # every op cos wraps ``` -The two halves are independent and useful together: the URL lets the **user** watch and take over in a browser, while `cos computer` lets **you** act. `desktop:1` ships the same five agent CLIs `devbox:1` does, so "run an agent on a box and let the user watch the screen" needs no extra setup — see Pattern E and `references/coding-agents.md` for pointing them at a provider. +The two halves are independent and useful together: the URL lets the **user** watch and take over in a browser, while `cos computer` lets **you** act. `desktop:1` also ships the Claude Code, Codex, Pi, OpenCode and Cursor CLIs, so "run an agent on a box and let the user watch the screen" needs no extra setup. + +Both verbs are thin wrappers over `createos sandbox desktop` and `createos sandbox computer`. cos adds only the project-box resolution — which box belongs to this directory, and creating one on a desktop image when there is none. Anything cos does not wrap is reachable directly: run `createos sandbox computer --help` for the full op list, including clipboard, window manipulation and screen resize. A box you address by name or id needs no project state, so `createos sandbox computer screenshot ` works on any desktop box, not just this project's. Things that will bite you if you skip them: - **Take a screenshot before you click, and after.** You are driving blind otherwise — nothing in this API confirms that a click landed on what you meant. - **Coordinates are raw X11 pixels** of that screen, with no scaling or DPI translation anywhere. Read the bounds from `cos computer screen` rather than assuming 1280x800. -- **The desktop boots after the box reports `running`.** `cos desktop` polls for readiness; a bare `cos up -r desktop:1` does not, and every computer call will fail until the stack is up. -- **A `409` is ambiguous by design.** fc returns `desktop_unavailable` both while the desktop is still coming up and when an action fails on a perfectly healthy desktop, so never read it as "the box is broken". +- **The desktop boots after the box reports `running`.** `cos desktop` waits for readiness; a bare `cos up -r desktop:1` does not, and every computer call will fail until the stack is up. +- **A `409` is ambiguous by design.** fc returns `desktop_unavailable` both while the desktop is still coming up and when an action fails on a perfectly healthy desktop, so never read it as "the box is broken". The CLI retries the first case for you during `cos desktop`. - **The noVNC link is a bearer URL** — anyone holding it can drive the desktop, and the token expires. Say so when handing it over, and don't paste it anywhere it will outlive the box. -- This is the one place `cos` calls the CreateOS REST API directly, because the `createos` CLI has no computer or desktop command yet. Everything else still goes through the CLI. - -## Pattern E — hand the job to another coding agent - -`devbox:1` ships five agent CLIs — `claude`, `codex`, `opencode`, `pi`, `cursor-agent` — so "have a different agent do this in a box" needs no install. Each can be pointed at **OpenRouter, an OpenAI-compatible provider, or an Anthropic-compatible one** (with real exceptions, below), so this does not require the user to hold an Anthropic subscription. - -```bash -export OPENROUTER_API_KEY=sk-or-... # in the user's own shell -cos agent -m openai/gpt-5.6-luna -o . claude . 'fix the failing tests' -cos agent -P anthropic -m claude-sonnet-4-5 -o . pi . 'add type hints' -cos agent -P https://gw.example.com/v1 -k MY_KEY -o . codex . 'port this to v2' -``` - -`cos agent ` stages the directory, wires the agent to the provider, runs it headless with its permission gate off (the microVM is the isolation), and destroys the box. Flags: `-P` provider (default `openrouter`), `-m` model, `-k` the env var holding the key, plus every `offload` flag. - -What to get right: - -- **`-o .` or the work is lost.** Without it the box is destroyed with the agent's edits inside and you keep only the transcript. Run it on a clean tree so `git diff` shows exactly what changed. -- **Keys come from the user's shell via `-v`/`-k`, never from the conversation.** Same rule as the CreateOS key — asking them to paste a provider key writes it to the transcript. -- **Two agents can't be repointed the way you'd assume.** `cursor-agent` runs only on Cursor's own service — no third-party provider path exists. `codex` speaks only the OpenAI **Responses** wire, so a plain Chat-Completions gateway is rejected at config load. `opencode` and `pi` will talk to anything. -- **An agent box holds an API key**, so `-p openrouter` (or `-p openai` / `-p anthropic`) is worth reaching for, composed with whatever registries the task itself needs: `-p openrouter -p npm`. - -The per-agent env blocks, the wire-protocol matrix, and the traps (claude needs `IS_SANDBOX=1` as root; `openrouter.ai/api` vs `/api/v1`) → **`references/coding-agents.md`**. - +- **These need a recent `createos` CLI.** Desktop and computer-use moved out of cos into the CLI, so an older binary has no `sandbox computer` command. `cos` says so and tells the user to upgrade rather than failing obscurely. ## Scratch box and data disks ```bash diff --git a/packages/codex-plugin/scripts/cos b/packages/codex-plugin/scripts/cos index d9d6a9a..331bf21 100755 --- a/packages/codex-plugin/scripts/cos +++ b/packages/codex-plugin/scripts/cos @@ -698,96 +698,23 @@ cmd_unexpose(){ } # ══════════════════════════ desktop / computer use ════════════════════════════ -# `createos` has no computer or desktop command, so these two verbs are the only -# place cos talks to the sandbox REST API directly instead of shelling out to the -# CLI. Everything else in this script stays CLI-driven; when the CLI grows a -# `sandbox computer` group, delete api()/api_auth() and shell out like the rest. +# Both verbs shell out to `createos sandbox desktop` and `createos sandbox +# computer`, like every other verb here. The REST layer this script used to +# carry is gone: the CLI owns the readiness wait, the error-code mapping and +# the noVNC link now, so every host that shells out gets them, not just this one. # -# Auth mirrors the CLI exactly: an api key goes in X-Api-Key, an OAuth JWT in -# X-Access-Token. fc rejects Bearer on user-facing routes, and it rejects a JWT -# sent under X-Api-Key ("invalid api key") — the two are not interchangeable. -api_base(){ printf '%s' "${CREATEOS_SANDBOX_URL:-https://api.sb.createos.sh}"; } - -api_auth(){ - # Precedence MUST match the CLI's (createos-cli cmd/root/root.go): an OAuth - # session wins, an api key is the fallback. Inverting it authenticates these - # direct calls as a different identity than every CLI-driven verb — and the - # symptom is a 404 on a box cos itself just created, which reads like the box - # is missing rather than like an auth mismatch. - local exp at - if [ -f "$CREATEOS_DIR/.oauth" ]; then - exp=$(jq -r '.expires_at // 0' "$CREATEOS_DIR/.oauth" 2>/dev/null || echo 0) - numeric "$exp" || exp=0 - # cos deliberately does not implement OAuth refresh. The CLI already refreshes - # in its pre-flight and rewrites ~/.createos/.oauth, so poke it and re-read - # rather than carrying a second, subtly different refresh implementation. - if [ "$(date +%s)" -ge "$((exp - 60))" ]; then - "$CLI" -o json sandbox ls >/dev/null 2>&1 || true - fi - at=$(jq -r '.access_token // empty' "$CREATEOS_DIR/.oauth" 2>/dev/null || true) - [ -n "$at" ] && { printf 'X-Access-Token: %s' "$at"; return 0; } - fi - # NOTE: the CLI itself does NOT read CREATEOS_API_KEY (it has no such env var), - # so this branch works for these REST calls but not for any CLI-driven verb. - [ -n "${CREATEOS_API_KEY:-}" ] && { printf 'X-Api-Key: %s' "$CREATEOS_API_KEY"; return 0; } - [ -f "$CREATEOS_DIR/.token" ] && { printf 'X-Api-Key: %s' "$(tr -d '\r\n' <"$CREATEOS_DIR/.token")"; return 0; } - die "not signed in — run 'cos auth'" -} - -# Map the computer API's error codes onto something actionable. Worth doing by -# hand: `desktop_unavailable` is fc's catch-all for every X failure, so the raw -# message alone never tells you whether the desktop is still booting or the -# action itself failed on a live desktop. -api_check(){ local code=$1 out=$2 what=$3 msg - case "$code" in 2??) return 0;; esac - msg=$(printf '%s' "$out" | jq -r '.message // .error // empty' 2>/dev/null || true) - case "$code" in - 000) die "no response from $(api_base) — network down, or CREATEOS_SANDBOX_URL points somewhere wrong";; - 401|403) die "auth rejected (HTTP $code). The API key or browser session is invalid or expired. - Ask the user to re-run 'createos login' in their own terminal, or export CREATEOS_API_KEY.";; - 404) die "not found (HTTP 404): ${msg:-$what} - Either the box is gone, or it has no such screen — computer-use needs a desktop image ('cos desktop').";; - 409) case "$msg" in - *ingress*) die "409: $msg — 'cos desktop' turns ingress on for you";; - *desktop_unavailable*|*) die "desktop did not answer (HTTP 409 ${msg:-desktop_unavailable}). - fc returns this both while the desktop is still booting AND when an action fails on a live desktop. - If the box just came up, 'cos desktop' waits for readiness — run that first.";; - esac;; - 429) die "rate limited (429) — the control plane caps concurrent screenshots. Retry in a second.";; - 501) die "501 desktop_tools_unavailable — this rootfs has no desktop tools installed. Recreate with: cos down && cos desktop";; - *) die "API error HTTP $code on $what${msg:+: $msg}";; - esac -} - -# api [json-body] → prints the response payload, unwrapped from -# fc's JSend envelope. Callers pipe it through jq for the fields they want. -api(){ local method=$1 path=$2 body=${3:-} hdr code out tmp - hdr=$(api_auth) || exit 1 - tmp=$(mktemp) - local args=(-sS -X "$method" -H "$hdr" -o "$tmp" -w '%{http_code}' --max-time 60) - [ -n "$body" ] && args+=(-H 'Content-Type: application/json' -d "$body") - code=$(curl "${args[@]}" "$(api_base)$path" 2>/dev/null) || code=000 - out=$(cat "$tmp"); rm -f "$tmp" - api_check "$code" "$out" "$method $path" - printf '%s' "$out" | jq -c '.data // .' 2>/dev/null || printf '%s' "$out" -} - -# The desktop stack (Xvfb → XFCE → x11vnc → websockify) starts AFTER the box -# reports `running`, so every computer call 404s or 409s for the first while. -# Neither fc nor the SDK polls for this — every caller has to, so cos does it here. -desktop_wait(){ local id=$1 screen=$2 to=${3:-120} hdr code i=0 said=0 - hdr=$(api_auth) || exit 1 - while [ "$i" -lt "$to" ]; do - code=$(curl -sS -o /dev/null -w '%{http_code}' --max-time 10 -H "$hdr" \ - "$(api_base)/v1/sandboxes/$id/computer/screen?screen_id=$screen" 2>/dev/null) || code=000 - case "$code" in - 2??) [ "$said" = 1 ] && echo "cos: desktop ready" >&2; return 0;; - 401|403|501) api_check "$code" '' "desktop readiness";; - esac - [ "$said" = 0 ] && { echo "cos: waiting for the desktop stack to come up (up to ${to}s)…" >&2; said=1; } - sleep 2; i=$((i+2)) - done - die "desktop did not come up within ${to}s on $id ($screen) — check: cos run 'pgrep -a Xvfb; pgrep -a websockify'" +# cos keeps only what the CLI cannot know: which box belongs to this project, +# and how to create one on a desktop image when there is none. + +# The CLI grew these commands after this script shipped, so probe for them +# rather than failing with a bare "unknown command". Same shape as the +# `sandbox sync --mode` probe further down. +require_cli_computer(){ + "$CLI" sandbox computer --help >/dev/null 2>&1 && return 0 + die "the installed createos CLI has no 'sandbox computer' command. + Desktop and computer-use moved out of cos and into the CLI. Upgrade it: + createos upgrade + or re-run the installer from https://createos.sh" } # ───────────────────────────── desktop: graphical box + a live noVNC URL @@ -796,9 +723,10 @@ cmd_desktop(){ local shape=s-2vcpu-4gb screen=screen-0; local OPTIND=1 o while getopts "s:S:h" o; do case $o in s) shape=$OPTARG;; S) screen=$OPTARG;; - h) echo "cos desktop [-s shape] [-S screen-N] desktop:1 box + ingress + live noVNC URL"; return 0;; + h) echo "cos desktop [-s shape] [-S screen-N] desktop:1 box + a live noVNC URL"; return 0;; *) die "usage: cos desktop [-s shape] [-S screen-N]";; esac; done shift $((OPTIND-1)) + require_cli_computer local id; id=$(state_get id) if [ -z "$id" ] || ! box_live "$id"; then @@ -807,8 +735,9 @@ cmd_desktop(){ id=$(state_get id); [ -n "$id" ] || die "box creation did not record an id" else box_resume_if_paused "$id" - # Never silently drive a non-desktop box: every computer call would 501 with - # a much less obvious message than saying so here. + # The CLI refuses a non-desktop box too, but it tells you to create a new + # sandbox. That is the wrong move for a project box — there is one per + # directory, so the fix is to replace this one. local rf; rf=$("$CLI" -o json sandbox get "$id" 2>/dev/null | jq -r '.rootfs // empty') case "$rf" in *desktop*) :;; @@ -818,36 +747,26 @@ cmd_desktop(){ fi wait_running "$id" 30 || die "box $id not running" - "$CLI" sandbox edit "$id" --ingress on >/dev/null 2>&1 || die "failed to enable ingress on $id" - desktop_wait "$id" "$screen" - - local conn url exp - conn=$(api GET "/v1/sandboxes/$id/computer/screens/$screen/connect") - url=$(printf '%s' "$conn" | jq -r '.url // empty') - exp=$(printf '%s' "$conn" | jq -r '.expires_at // "?"') - [ -n "$url" ] || die "connect returned no URL — ingress is off on $id (fc only mints one when ingress is enabled)" state_set desktop_screen "$screen" - { - echo "cos: desktop $screen on $id — open in a browser:" - echo "cos: ⚠ anyone with this link can drive the desktop. The token expires $exp, and re-running" - echo "cos: 'cos desktop' mints a fresh one (which invalidates this link for NEW connections)." - } >&2 - echo "$url" + # The CLI turns ingress on, waits for the desktop stack, and prints the URL + # together with its expiry and the warning that the link is the credential. + "$CLI" sandbox desktop "$id" --screen "$screen" } # ───────────────────────────── computer: drive that desktop from the agent side computer_usage(){ cat <<'EOF' cos computer drive the desktop in the project box (needs `cos desktop` first) - screenshot [-o file] capture PNG (default: $STATE_DIR/screenshot.png), prints the path + screenshot [-o file] capture PNG, prints the path screen screen geometry {width,height} cursor cursor position {x,y} + windows list windows on the screen move move the pointer click [ ] click (optionally move there first) type type a string key ... press a chord, e.g. cos computer key ctrl l open open a target in the desktop browser - windows list windows on the screen - raw [json] any other computer endpoint (path relative to .../computer) +Each op runs `createos sandbox computer ` against this project's box; run +`createos sandbox computer --help` for the ops not listed here. Screen defaults to screen-0; override per call with COS_SCREEN=screen-N. Coordinates are raw X11 pixels of that screen — match them against `cos computer screen`. EOF @@ -856,66 +775,22 @@ EOF cmd_computer(){ local op=${1:-}; shift || true case "$op" in ''|-h|--help|help) computer_usage; return 0;; esac + require_cli_computer local id; id=$(state_get id) [ -n "$id" ] || die "no active box — run 'cos desktop' first" [ "$(box_status "$id")" = paused ] && die "box $id is paused — 'cos resume' first" local screen=${COS_SCREEN:-$(state_get desktop_screen)}; screen=${screen:-screen-0} - local base="/v1/sandboxes/$id/computer" q="screen_id=$screen" - - case "$op" in - screenshot) - local out="$STATE_DIR/screenshot.png" hdr code - [ "${1:-}" = "-o" ] && { out=${2:?-o needs a path}; shift 2; } - hdr=$(api_auth) || exit 1 - # Screenshot is the one computer route that returns bytes, not JSON, so it - # bypasses api() entirely — the PNG is passed through from fc verbatim. - code=$(curl -sS -H "$hdr" -o "$out" -w '%{http_code}' --max-time 60 \ - "$(api_base)$base/screenshot?$q" 2>/dev/null) || code=000 - case "$code" in - 2??) :;; - *) local body; body=$(cat "$out" 2>/dev/null || true); rm -f "$out"; api_check "$code" "$body" "screenshot";; - esac - echo "cos: screenshot → $out ($(wc -c <"$out" | tr -d ' ') bytes) — open it with the Read tool" >&2 - echo "$out";; - screen) api GET "$base/screen?$q";; - cursor) api GET "$base/cursor?$q";; - windows) api GET "$base/windows?$q";; - move) - numeric "${1:-}" && numeric "${2:-}" || die "usage: cos computer move " - api POST "$base/mouse/move?$q" "$(jq -nc --argjson x "$1" --argjson y "$2" '{x:$x,y:$y}')" >/dev/null - echo "cos: moved to $1,$2" >&2;; - click) - if [ $# -ge 2 ]; then - numeric "$1" && numeric "$2" || die "usage: cos computer click [ ]" - api POST "$base/mouse/click?$q" "$(jq -nc --argjson x "$1" --argjson y "$2" '{x:$x,y:$y}')" >/dev/null - echo "cos: clicked $1,$2" >&2 - else - api POST "$base/mouse/click?$q" '{}' >/dev/null - echo "cos: clicked at the current cursor position" >&2 - fi;; - type) - [ $# -ge 1 ] || die "usage: cos computer type " - # Unquoted multi-word text arrives as separate argv entries; join before - # sending so `cos computer type hello world` types the space too. - local text="$*" - api POST "$base/keyboard/type?$q" "$(jq -nc --arg t "$text" '{text:$t}')" >/dev/null - echo "cos: typed ${#text} chars" >&2;; - key) - [ $# -ge 1 ] || die "usage: cos computer key ... e.g. cos computer key ctrl l" - api POST "$base/keyboard/press?$q" "$(jq -nc '{keys:$ARGS.positional}' --args "$@")" >/dev/null - echo "cos: pressed $*" >&2;; - open) - [ $# -ge 1 ] || die "usage: cos computer open " - api POST "$base/open?$q" "$(jq -nc --arg t "$1" '{target:$t}')" >/dev/null - echo "cos: opened $1" >&2;; - raw) - # Escape hatch: cos wraps the handful of ops an agent loop actually needs, - # not all ~30 computer routes. Everything else goes through here. - local m=${1:?usage: cos computer raw [json]} p=${2:?path required}; shift 2 - case "$p" in /*) :;; *) p="$base/$p";; esac - api "$m" "$p" "${1:-}";; - *) die "unknown computer op '$op' — run 'cos computer help'";; - esac + + # cos has always spelled the screenshot path -o. The CLI spells it --out, + # because -o is the CLI's global output-format flag. Translate rather than + # break every caller that already passes -o. + if [ "$op" = screenshot ]; then + local -a mapped=(); local a + for a in ${@+"$@"}; do case "$a" in -o) mapped+=(--out);; *) mapped+=("$a");; esac; done + set -- ${mapped[@]+"${mapped[@]}"} + fi + + "$CLI" sandbox computer "$op" "$id" --screen "$screen" ${@+"$@"} } # ───────────────────────────── cluster: N boxes on one private network (by-name DNS) @@ -1091,6 +966,10 @@ cmd_resume(){ esac box_resume_if_paused "$id" echo "cos: $id running. Re-run 'cos sync' / 'cos tunnel' / 'cos expose' if you had them up." >&2 + # A resumed desktop box answers reads (screen, screenshot) before its input + # stack is back, so a click can still 409 for a moment. `cos desktop` waits + # and mints a fresh link — the old one is expired anyway. + [ -n "$(state_get desktop_screen)" ] && echo "cos: this box had a desktop — re-run 'cos desktop' before driving it again." >&2 } # ───────────────────────────── template: build a custom rootfs from a Dockerfile diff --git a/packages/codex-plugin/skills/using-createos-sandbox/SKILL.md b/packages/codex-plugin/skills/using-createos-sandbox/SKILL.md index 3969c3d..53a87e2 100644 --- a/packages/codex-plugin/skills/using-createos-sandbox/SKILL.md +++ b/packages/codex-plugin/skills/using-createos-sandbox/SKILL.md @@ -169,42 +169,21 @@ cos computer open https://example.com cos computer click 640 400 cos computer type 'hello' cos computer key ctrl l # a chord -cos computer help # every op, plus `raw` for the rest of the API +cos computer help # every op cos wraps ``` -The two halves are independent and useful together: the URL lets the **user** watch and take over in a browser, while `cos computer` lets **you** act. `desktop:1` ships the same five agent CLIs `devbox:1` does, so "run an agent on a box and let the user watch the screen" needs no extra setup — see Pattern E and `references/coding-agents.md` for pointing them at a provider. +The two halves are independent and useful together: the URL lets the **user** watch and take over in a browser, while `cos computer` lets **you** act. `desktop:1` also ships the Claude Code, Codex, Pi, OpenCode and Cursor CLIs, so "run an agent on a box and let the user watch the screen" needs no extra setup. + +Both verbs are thin wrappers over `createos sandbox desktop` and `createos sandbox computer`. cos adds only the project-box resolution — which box belongs to this directory, and creating one on a desktop image when there is none. Anything cos does not wrap is reachable directly: run `createos sandbox computer --help` for the full op list, including clipboard, window manipulation and screen resize. A box you address by name or id needs no project state, so `createos sandbox computer screenshot ` works on any desktop box, not just this project's. Things that will bite you if you skip them: - **Take a screenshot before you click, and after.** You are driving blind otherwise — nothing in this API confirms that a click landed on what you meant. - **Coordinates are raw X11 pixels** of that screen, with no scaling or DPI translation anywhere. Read the bounds from `cos computer screen` rather than assuming 1280x800. -- **The desktop boots after the box reports `running`.** `cos desktop` polls for readiness; a bare `cos up -r desktop:1` does not, and every computer call will fail until the stack is up. -- **A `409` is ambiguous by design.** fc returns `desktop_unavailable` both while the desktop is still coming up and when an action fails on a perfectly healthy desktop, so never read it as "the box is broken". +- **The desktop boots after the box reports `running`.** `cos desktop` waits for readiness; a bare `cos up -r desktop:1` does not, and every computer call will fail until the stack is up. +- **A `409` is ambiguous by design.** fc returns `desktop_unavailable` both while the desktop is still coming up and when an action fails on a perfectly healthy desktop, so never read it as "the box is broken". The CLI retries the first case for you during `cos desktop`. - **The noVNC link is a bearer URL** — anyone holding it can drive the desktop, and the token expires. Say so when handing it over, and don't paste it anywhere it will outlive the box. -- This is the one place `cos` calls the CreateOS REST API directly, because the `createos` CLI has no computer or desktop command yet. Everything else still goes through the CLI. - -## Pattern E — hand the job to another coding agent - -`devbox:1` ships five agent CLIs — `claude`, `codex`, `opencode`, `pi`, `cursor-agent` — so "have a different agent do this in a box" needs no install. Each can be pointed at **OpenRouter, an OpenAI-compatible provider, or an Anthropic-compatible one** (with real exceptions, below), so this does not require the user to hold an Anthropic subscription. - -```bash -export OPENROUTER_API_KEY=sk-or-... # in the user's own shell -cos agent -m openai/gpt-5.6-luna -o . claude . 'fix the failing tests' -cos agent -P anthropic -m claude-sonnet-4-5 -o . pi . 'add type hints' -cos agent -P https://gw.example.com/v1 -k MY_KEY -o . codex . 'port this to v2' -``` - -`cos agent ` stages the directory, wires the agent to the provider, runs it headless with its permission gate off (the microVM is the isolation), and destroys the box. Flags: `-P` provider (default `openrouter`), `-m` model, `-k` the env var holding the key, plus every `offload` flag. - -What to get right: - -- **`-o .` or the work is lost.** Without it the box is destroyed with the agent's edits inside and you keep only the transcript. Run it on a clean tree so `git diff` shows exactly what changed. -- **Keys come from the user's shell via `-v`/`-k`, never from the conversation.** Same rule as the CreateOS key — asking them to paste a provider key writes it to the transcript. -- **Two agents can't be repointed the way you'd assume.** `cursor-agent` runs only on Cursor's own service — no third-party provider path exists. `codex` speaks only the OpenAI **Responses** wire, so a plain Chat-Completions gateway is rejected at config load. `opencode` and `pi` will talk to anything. -- **An agent box holds an API key**, so `-p openrouter` (or `-p openai` / `-p anthropic`) is worth reaching for, composed with whatever registries the task itself needs: `-p openrouter -p npm`. - -The per-agent env blocks, the wire-protocol matrix, and the traps (claude needs `IS_SANDBOX=1` as root; `openrouter.ai/api` vs `/api/v1`) → **`references/coding-agents.md`**. - +- **These need a recent `createos` CLI.** Desktop and computer-use moved out of cos into the CLI, so an older binary has no `sandbox computer` command. `cos` says so and tells the user to upgrade rather than failing obscurely. ## Scratch box and data disks ```bash