feat(engine): outbox-ordered turn.submit saga with an operation journal

[Zerlight]

Summary

Phase 1 of CODE-627 — Conversation turn graph & immutable attachment store. Linear: https://linear.app/arcbox/issue/CODE-629/featengine-outbox-ordered-turnsubmit-saga-with-an-operation-journal

Stack: #507 ← this PR (ruocheng/code-629, base ruocheng/code-628) ← #509. Merge bottom-up; this PR's diff is only its own commits.

One mutation primitive for every turn-starting affordance: the outbox-ordered turn.submit saga (replay → admit → persist intent → provider work under hard timeouts → dispatch → cutover) with an operation journal that makes every outcome committed-or-failed and idempotent by operationId. Legacy history.branch writes the tree non-destructively from day one; boot recovery resolves open operations; run sealing and history binding address runs by id.

Commits

• feat(engine): address run sealing and history binding by runId
• feat(engine): persist turn intents for every turn-starting input
• test(engine): cover legacy-input turn tracking
• feat(engine): implement the outbox-ordered turn.submit saga
• test(engine): cover the turn.submit saga
• feat(engine): record legacy rewrites in the turn tree and recover open operations at boot
• fix(engine): persist automation prompt turns
• refactor(daemon): share the conversation database connection
• fix(engine): resolve turn intents on interrupted or defected exits
• test(engine): cover interrupted dispatch, timed-out start, and errored predecessors
• fix(engine): split submit launch and dispatch timeouts with a running-turn rescue
• fix(engine): gate resolve side effects on the store transition
• fix(conversation): restore persist and resolve after the shared-connection rebase
• fix(engine): scope turn.submit replay to its session and keep the conversation-reported flag on a dispatch failure

Verification

Every commit passed pnpm check:ci and pnpm test at its own tip; the stack tip (1d942a62; the same tree as the originally gated 73fc5ff2 plus the two review fixes below on 628/629) is at pnpm check:ci 0 errors, pnpm test 3408 passed / 1 skipped. Adversarial reviewers (one per axis, isolated read-only worktrees) reviewed the branch; each P1/P2 was reproduced with a failing test or a probe step before its fix — the round-by-round record is in the Linear issue's comments.

Checklist

• pnpm check:ci and pnpm test both pass (no Rust changes)
• I ran the affected surface and observed the change working — engine exercised through its harness tests and the dev-mock parity suite; the live-daemon run against real providers is recorded per issue in Linear (the claude edit path still needs a paid turn, opencode its binary)
• Wire: frames added under the still-unshipped wire 80 (no further bump until release)
• New code and assets are my own work
• Docs and comments are updated where behavior changed (AGENTS.md and module docs in this branch)

[pullfrog]

Your Claude subscription has hit its usage limit. It resets at 11:30am (UTC). Re-trigger Pullfrog after the reset, or add an ANTHROPIC_API_KEY repo secret — Pullfrog routes around an exhausted subscription automatically when one is present.

Add repo secret → · Model settings → · Setup docs → · Ask in Discord →

  ^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[linear-code]

CODE-629

[greptile-apps]

Greptile Summary

This PR introduces the durable, operation-journaled turn.submit saga, tracks legacy and automation inputs as conversation turns, adds boot recovery and run-addressed lifecycle handling, and shares the daemon database connection between session and conversation persistence.

• Persists turn intent before provider launch and dispatch, then records a terminal operation result.
• Adds graph revision and active-leaf updates after accepted dispatches.
• Tracks provider lifecycle events against explicit run IDs and recovers interrupted operations at boot.
• Refactors daemon migrations and session/conversation stores onto a shared SQLite client.
• The SQLite conversation implementation currently diverges from the engine store contract, and session deletion retains a stale field reference.

Confidence Score: 1/5

The PR is not safe to merge because the daemon conversation store violates required interfaces and saga semantics, session deletion references a nonexistent field, and legacy run IDs are not stable across saves.

Normal daemon turn submission cannot obtain a store-assigned turn or a successful transition result, late operation resolvers can overwrite committed outcomes, session deletion does not compile against the refactored orchestrator, and missing run IDs are regenerated on every save.

Files Needing Attention: apps/daemon/src/conversation-store.ts, packages/host/engine/src/session/orchestrator.ts, apps/daemon/src/session-store.ts

Important Files Changed

Filename	Overview
apps/daemon/src/conversation-store.ts	Refactors conversation persistence onto the shared database, but breaks intent assignment and terminal-operation compare-and-set contracts.
packages/host/engine/src/conversation/turn-service.ts	Implements durable intent, commit, failure, settlement, and boot-recovery orchestration while depending on return semantics absent from the daemon store.
packages/host/engine/src/session/lifecycle-service.ts	Adds replay-first admission, phase-specific timeouts, cold resume, fresh-root launch, and operation resolution for turn submission.
packages/host/engine/src/session/orchestrator.ts	Adds run-addressed dispatch and lifecycle tracking but leaves session deletion referencing the removed conversation-store field.
apps/daemon/src/session-store.ts	Moves the session store to the shared client but remints legacy missing run IDs without stabilizing the in-memory record.
apps/daemon/src/db/database.ts	Centralizes SQLite setup, migration-ledger reconciliation, migration execution, and connection ownership.

Comments Outside Diff (1)

1. packages/host/engine/src/session/orchestrator.ts, line 138-146 (link)

   Deletion uses removed field

   The constructor field was renamed from conversations to turns, but delete() still reads and calls conversations. That property no longer exists, so the engine fails type-checking. If emitted without type-checking, every session.delete fails before removing the conversation graph and session record.

_{Reviews (1): Last reviewed commit: "fix(engine): gate resolve side effects o..." | Re-trigger Greptile}

[greptile-apps]

Greptile has paused reviews on this repository — it used its 100 free open-source review credits for this billing period. Reviews resume automatically on September 23. To continue before then, an organization admin can keep reviews running past the free credits — those bill as normal usage.

[pullfrog]

Your Claude subscription has hit its usage limit. It resets at 11:30am (UTC). Re-trigger Pullfrog after the reset, or add an ANTHROPIC_API_KEY repo secret — Pullfrog routes around an exhausted subscription automatically when one is present.

Add repo secret → · Model settings → · Setup docs → · Ask in Discord →

  ^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[Copilot]

🟡 Changes recommended

turn.submit replay is not currently scoped to sessionId, allowing cross-session operationId replay (and potential information leakage) unless guarded.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Implements Phase 1 of the conversation turn graph execution path in the engine by introducing an outbox-ordered turn.submit saga backed by a durable operation journal, while also ensuring legacy agent.input paths persist turn intents so the graph never misses turn-starting inputs.

Changes:

• Add ConversationTurnService to persist turn intents + operation journal entries, commit running turns, resolve failures, and recover open operations at boot.
• Route turn.submit through a submit saga with admission gating, durable commit points, hard timeouts, and idempotent replay by operationId.
• Rework run bookkeeping to address runs by runId (history binding, sealing, event gating) and share a single daemon DB connection for session + conversation stores.

File summaries

File	Description
packages/host/engine/src/session/session-record-registry.ts	Address run mutations by runId, add graph revision/leaf commit helper, and make beginRun return a stable runId.
packages/host/engine/src/session/session-input-dispatcher.ts	Persist legacy turn intents, coordinate with open-operation gating, and ensure failures resolve stored intents.
packages/host/engine/src/session/session-event-processor.ts	Gate session-scoped straggler events by current run; settle/record turn/run outcomes via ConversationTurnService.
packages/host/engine/src/session/orchestrator.ts	Thread runId into live sessions, integrate ConversationTurnService, and ensure teardown settles turns and seals runs by id.
packages/host/engine/src/session/live-session.ts	Attach runId to each live adapter instance for run-addressed bookkeeping.
packages/host/engine/src/session/lifecycle-service.ts	Implement the turn.submit saga (admit/persist/launch/dispatch/resolve) and wire it into lifecycle paths (including rewrites).
packages/host/engine/src/failure.ts	Add busy request code and introduce causeToRequestFailure for interrupt/defect exit mapping.
packages/host/engine/src/engine.ts	Instantiate and wire ConversationTurnService, switch conversation request handling to lifecycle-backed saga, and run boot recovery.
packages/host/engine/src/conversation/turn-service.ts	New durable turn/operation journal service: intent persistence, commit/resolve, running-turn tracking, and boot recovery.
packages/host/engine/src/conversation/request-handler.ts	Handle turn.submit by invoking the lifecycle saga and replying with turn.submitted / stored request.failed.
packages/host/engine/src/automation/turn-watcher.ts	Add onDispatchAccepted hook and ensure it runs once after dispatch acceptance or terminal outcome.
packages/host/engine/src/tests/session-record-registry.test.ts	New tests for run addressing semantics (sealRun, bindHistoryId, isCurrentRun, caller-minted runId).
packages/host/engine/src/tests/engine-turn-tracking.test.ts	New tests for legacy input turn tracking, recovery behavior, and commit/resolve idempotence.
packages/host/engine/src/tests/engine-turn-submit.test.ts	New tests for the turn.submit saga: replay, admission, timeouts, relaunch/resume behavior, and failure handling.
packages/host/engine/src/tests/engine-session-lifecycle.test.ts	Update test harness construction to pass runId into LiveSession.
packages/host/engine/src/tests/engine-schedule.test.ts	Ensure scheduled prompts are recorded as completed turns; extend harness to use conversation/resource stores.
packages/host/engine/src/tests/engine-conversation-stubs.test.ts	Update stubs to only assert loud failures for still-unimplemented read/projection requests.
apps/daemon/tests/integration/session-store.test.ts	Update integration tests to use the shared openDaemonDatabase() connection lifecycle.
apps/daemon/src/session-store.ts	Refactor session store to accept a shared Drizzle client rather than owning its own SQLite connection/migrations.
apps/daemon/src/index.ts	Open a shared daemon database connection and pass its client to session + conversation stores.
apps/daemon/src/db/schema.ts	Update comments to reflect shared connection ownership for conversation/session tables.
apps/daemon/src/db/database.ts	New shared DB opener that runs migration ledger reconciliation + migrations once per owned connection.
apps/daemon/src/database-migrations.ts	New single source of truth for daemon migrations folder resolution.
apps/daemon/src/conversation-store.ts	Refactor conversation store to accept the shared Drizzle client instead of owning its own connection.
apps/daemon/src/tests/session-store.test.ts	Update unit tests to manage DB lifetime via openDaemonDatabase().
apps/daemon/src/tests/resource-store.test.ts	Adjust resource store tests to coexist with the new shared DB setup for session persistence.
apps/daemon/src/tests/conversation-store.test.ts	Update conversation store tests to use the shared DB client and explicitly reopen/close databases.
apps/daemon/AGENTS.md	Document the new shared graph/session DB connection ownership and migration workflow.

Review details

• Files reviewed: 28/28 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Zerlight]

Review round (2026-09-07). Verified every Copilot and Greptile finding against the branch.

Fixed — bc956547: turn.submit replay is scoped to the requesting session (an operation id from another session is refused with typed invalid_request), and a dispatch failure that the input dispatcher mirrored into the conversation now replies request.failed with reportedInConversation, so the client does not raise it a second time; a replay after a disconnect deliberately omits the flag because no live event backs it. Both covered by tests in engine-turn-submit.test.ts.

Already fixed before this round — the three Greptile P1s (store contract, terminal-result overwrite, legacy run-id drift) and the "deletion uses removed field" item outside the diff all described the tree at 89a97b9c, which an earlier rebase had left inconsistent; the repair commit now sits at the end of this branch (92f52fba), so the tip you see today is what CI checks.

Stack tip after this round: 1d942a62; all gates green at the tip, and the intermediate PR tips typecheck.

[pullfrog]

Your Claude subscription has hit its usage limit. It resets at 11:30am (UTC). Re-trigger Pullfrog after the reset, or add an ANTHROPIC_API_KEY repo secret — Pullfrog routes around an exhausted subscription automatically when one is present.

Add repo secret → · Model settings → · Setup docs → · Ask in Discord →

  ^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.