th-8e3087: SmoothFlow engine e2e — real daemon, fake-agent manifests, hooks contract, th flow - #562
Open
brentrager wants to merge 7 commits into
Open
th-8e3087: SmoothFlow engine e2e — real daemon, fake-agent manifests, hooks contract, th flow#562brentrager wants to merge 7 commits into
brentrager wants to merge 7 commits into
Conversation
… hooks contract, th flow
Brent asked for cmux/orca-style tests. Lane A had engine unit tests, the app
its XCTests, the phones codec tests — nothing booted the real thing and
drove it like a client. This adds crates/smooth-daemon/tests/flow_e2e: a REAL
smooth-daemon per test (isolated HOME, ephemeral port, private tmux server,
no lock / tailscale / relay / gateway creds), driven over the flow WS, the
HTTP siblings, the `th` binary and POST /api/flow/hooks. The agent under
test is tests/fixtures/fake-agent, installed through harness manifests the
way any coding CLI is, in four flavours — hooks / learned id / native /
scrape — one per state source the engine supports. 24 tests, ~70 s wall.
The suite found four engine bugs on its first run, fixed here:
- tmux rewrites control characters under a non-UTF-8 locale, so the
tab-joined `#{pane_dead}\t#{pane_dead_status}` query read `1_2` and a dead
pane was NEVER detected for any daemon not started from a shell (launchd,
CI). Every supervision rule was inert there. `|`-joined + pure parsers.
- `{daemon_url}` was rendered from the requested address, so a port-0 daemon
sent every pane's hooks to http://127.0.0.1:0. Resolved before the router
is built.
- A resumed or prompt-less harness sat in `starting` forever: SessionStart
flipped the source to `hooks` (stopping the scraper) and mapped to
nothing. SessionStart on a starting row is now idle.
- A `held` row (duplicate-resume guard) was flapped by the supervisor, which
saw its dead tmux session as a crash and scheduled a resume the guard then
refused, every backoff. Held rows are left alone.
CI: tmux on the Linux leg + SMOOTH_E2E_STRICT=1 so a skip is a failure;
Windows compiles the suite empty. docs/Engineering/SmoothFlow-Testing.md is
the map (survey, layers, per-test table, the rig, fake-agent, runtimes, gaps).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q5YUmVQ5obDT27oLjgt7AL
brentrager
enabled auto-merge (squash)
September 9, 2026 14:32
🦋 Changeset detectedLatest commit: 22939e2 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
… hooks contract, th flow
Brent asked for cmux/orca-style tests. Lane A had engine unit tests, the app
its XCTests, the phones codec tests — nothing booted the real thing and
drove it like a client. This adds crates/smooth-daemon/tests/flow_e2e: a REAL
smooth-daemon per test (isolated HOME, ephemeral port, private tmux server,
no lock / tailscale / relay / gateway creds), driven over the flow WS, the
HTTP siblings, the `th` binary and POST /api/flow/hooks. The agent under
test is tests/fixtures/fake-agent, installed through harness manifests the
way any coding CLI is, in four flavours — hooks / learned id / native /
scrape — one per state source the engine supports. 24 tests, ~70 s wall.
The suite found four engine bugs on its first run, fixed here:
- tmux rewrites control characters under a non-UTF-8 locale, so the
tab-joined `#{pane_dead}\t#{pane_dead_status}` query read `1_2` and a dead
pane was NEVER detected for any daemon not started from a shell (launchd,
CI). Every supervision rule was inert there. `|`-joined + pure parsers.
- `{daemon_url}` was rendered from the requested address, so a port-0 daemon
sent every pane's hooks to http://127.0.0.1:0. Resolved before the router
is built.
- A resumed or prompt-less harness sat in `starting` forever: SessionStart
flipped the source to `hooks` (stopping the scraper) and mapped to
nothing. SessionStart on a starting row is now idle.
- A `held` row (duplicate-resume guard) was flapped by the supervisor, which
saw its dead tmux session as a crash and scheduled a resume the guard then
refused, every backoff. Held rows are left alone.
CI: tmux on the Linux leg + SMOOTH_E2E_STRICT=1 so a skip is a failure;
Windows compiles the suite empty. docs/Engineering/SmoothFlow-Testing.md is
the map (survey, layers, per-test table, the rig, fake-agent, runtimes, gaps).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q5YUmVQ5obDT27oLjgt7AL
… a state transition A real CLI without credentials sits on onboarding in the fresh HOME — a pane no scrape pattern matches, so `starting` is the engine's honest reading; th code reports natively only once it can boot. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q5YUmVQ5obDT27oLjgt7AL
…ness-flake fix Also lands the rig's own daemon.addr (a second daemon deliberately does not advertise itself since #546) and the th-code providers.json opt-in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…'s hold sticks The duplicate-resume guard was racy end to end, not just in the tick: kill, supervise_one and POST /api/flow/hooks each read a session row and then wrote it, so a `held` written between another reader's read and its write was silently overwritten (~1 in 3 under load). The observed sequence: kill --resume holds row A, A's dying agent posts its last Stop, the hook — resolved before the hold landed — writes A back to idle, and the next supervision pass reads A's dead tmux session as an unexpected crash. All three paths now take the same per-session lock (supervision try_locks and skips: a kill owns the row) and re-read the row under it, holding drops any pending relaunch backoff, and a hook naming a held row is ignored — its harness session id belongs to the live process that holds it, not to this row. 15/15 green on the previously 1-in-3 flaky e2e test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The remote tip was an older squash of this same work on an older base. This worktree carries it plus the main merge and the rule-4 race fix, so the tree is kept as-is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The per-session lock already fixes this path, but nothing pinned it: the bug was pure ordering — hook read the row BEFORE relaunch wrote Starting, saw the pre-kill idle, and HookOutcome::Started's `state == Starting` test declined, so a resumed row sat at `starting` until something timed out. The new test simulates the kill's slow half by holding the session lock rather than sleeping out KILL_GRACE, so it pins the ordering, not the timing; it fails on the pre-lock read and passes now. Narrowing kill's critical section so a hook can never be delayed past its own client timeout is deferred to th-786fdc: the attempt destabilised shell_lifecycle (8 failures in 16 runs against 0 in 16), with the shell logging out at the second-client attach — before kill is reached at all — which points at a separate race in the PTY detach/attach bridge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Brent: "can we write similar tests to orca / cmux". Lane A of th-6ac036 had engine unit tests, the macOS app its XCTests, the phones codec/reducer tests — nothing booted the real daemon and drove it the way a client,
th flowor a harness's hook script does.Solution
crates/smooth-daemon/tests/flow_e2e— a REALsmooth-daemonper test (isolated HOME, ephemeral port, private tmux server, no lock / tailscale / relay / gateway creds), driven over the flow WS, the HTTP siblings, thethbinary andPOST /api/flow/hooks. The agent under test istests/fixtures/fake-agent, installed through harness manifests like any coding CLI, in four flavours — hooks / learned id / native / scrape — one per state source the engine supports.24 tests, ~70 s wall (2 slow ones wait real backoff / usage-limit windows):
--resume <id>; the three-resume give-up; learned-id binding from the first hook; the duplicate-resume guard (held)th flowls/new/snapshot/send/inbox/approve/handoff/kill --json+ human lines;th harness list/add/hide/unhide/order/showflow.hello, the WS broadcast andth harness list;th harness add <path>of a custom manifest then a session on it; real claude/opencode/codex/th-code launches (opt-in,SMOOTH_E2E_REAL_HARNESSES=1)~/.smooth(daemon.addr,daemon.lock,operator-token,flow.dbbyte- and mtime-identical), teardown leaves nothing, andflow_e2e_server(the mac lane's host) is the same engineEngine bugs the suite found on day one (fixed here, each with a unit test)
#{pane_dead}\t#{pane_dead_status}query read1_2, so a dead pane was NEVER detected for any daemon not started from a shell (launchd, CI). Every supervision rule was inert there.|-joined formats + pure parsers.{daemon_url}rendered from the requested address — a port-0 daemon sent every pane's hooks tohttp://127.0.0.1:0. Resolved before the router is built.startingforever —SessionStartflipped the source tohooks(stopping the scraper) but mapped to nothing.SessionStarton a starting row ⇒ idle.heldrow flapped — the supervisor saw its dead tmux session as a crash and scheduled a resume the guard then refused, every backoff. Held rows are left alone.CI
pr-checks.yml:tmuxon the Linux leg andSMOOTH_E2E_STRICT=1so a missing tmux/bash/curl/thfails the job instead of passing it having run nothing. Windows compiles the suite empty (#![cfg(unix)]). The suite runs inside the normalcargo nextest run --profile ci.Verification
cargo nextest run -p smooai-smooth-daemon --test flow_e2e: 24/24, 68.5 s wall on a dev box at load 25–40cargo test -p smooai-smooth-flow --lib: 75/75 (new: pane parsers, SessionStart ⇒ idle, held rows survive a tick, prefs test made PATH-independent)cargo test -p smooai-smooth-daemon --lib operator flow_route: green (new: ephemeral port resolution)cargo fmt --check,cargo clippy(no new warnings),pnpm format:checkDocs
docs/Engineering/SmoothFlow-Testing.md— the cmux/orca survey, the layers table, a per-test table, the rig, the fake-agent contract, runtimes, the CI split rule, gaps. Linked fromdocs/Architecture/SmoothFlow.md§ Testing.Pearl: th-8e3087 (epic th-6ac036). Sibling: th-a58a97 (#548, the macOS XCUITest lane).
🤖 Generated with Claude Code
https://claude.ai/code/session_01Q5YUmVQ5obDT27oLjgt7AL