feat(engine): conversation graph reads and turn-scoped projection

[Zerlight]

Summary

Phase 1 of CODE-627 — Conversation turn graph & immutable attachment store. Linear: https://linear.app/arcbox/issue/CODE-631/featengine-conversation-graph-reads-and-turn-scoped-projection

Stack: #509 ← this PR (ruocheng/code-631, base ruocheng/code-630) ← #511. Merge bottom-up; this PR's diff is only its own commits.

conversation.graph.get and cursor-paged conversation.read compose the turn-scoped projection: host user rows from the ConversationStore, provider output through the attribution gate (fingerprint-verified positional alignment, active-lineage only), the live tail from the journal with an honest truncation marker, and the prompt-only history-unavailable fallback. Structured read cursors reject drift as a typed conflict. The dev mock answers the same frames.

Commits

• feat(schema): page conversation.read by items with a final-page watermark
• feat(engine): track evicted in-flight streams in the live journal
• feat(engine): compose the turn-scoped conversation projection
• feat(engine): serve conversation.graph.get and conversation.read
• feat(workbench): answer conversation graph and read frames in the dev mock host
• fix(engine): sort the read tail by stamp and carry cwd on projection history reads
• fix(engine): gate projection attribution on the active lineage and fingerprints
• fix(engine): safe journal-less watermark and honest truncation above the cut
• fix(engine): structured conversation cursors and loud mock parameter gaps

Verification

Every commit passed pnpm check:ci and pnpm test at its own tip; the stack tip (1d942a62; the same tree as the originally gated 73fc5ff2 plus the two review fixes below on 628/629) is at pnpm check:ci 0 errors, pnpm test 3408 passed / 1 skipped. Adversarial reviewers (one per axis, isolated read-only worktrees) reviewed the branch; each P1/P2 was reproduced with a failing test or a probe step before its fix — the round-by-round record is in the Linear issue's comments. The dev mock host serves the frames the renderer consumes, so the projection was exercised end to end in dev:mock.

Checklist

• pnpm check:ci and pnpm test both pass (no Rust changes)
• I ran the affected surface and observed the change working — engine exercised through its harness tests and the dev-mock parity suite; the live-daemon run against real providers is recorded per issue in Linear (the claude edit path still needs a paid turn, opencode its binary)
• Wire: frames added under the still-unshipped wire 80
• New code and assets are my own work
• Docs and comments are updated where behavior changed (AGENTS.md and module docs in this branch)

[pullfrog]

Your Claude subscription has hit its usage limit. It resets at 11:30am (UTC). Re-trigger Pullfrog after the reset, or add an ANTHROPIC_API_KEY repo secret — Pullfrog routes around an exhausted subscription automatically when one is present.

Add repo secret → · Model settings → · Setup docs → · Ask in Discord →

  ^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[linear-code]

CODE-631

[greptile-apps]

Greptile Summary

This PR implements turn-scoped conversation graph reads and paginated conversation projections across the schema, engine, and development mock.

• Composes host-owned user rows with fingerprint-gated provider history and a stamped live-journal tail.
• Adds structured paging cursors, final-page watermarks, graph summaries, and history-unavailable placeholders.
• Tracks evicted in-flight streams and refreshes provider-history caches after turn settlement.
• Adds engine, projection, journal, schema-contract, and mock-host coverage.

Confidence Score: 2/5

The PR is not safe to merge until failed-turn output is preserved and paginated reads reliably disclose truncation and remain pinned to one provider-history snapshot.

Failed turns can silently lose streamed output, oversized tails can be finalized after silently dropping state, and provider-history changes between pages can shift item offsets without invalidating the cursor.

Files Needing Attention: packages/host/engine/src/conversation/projection-service.ts

Important Files Changed

Filename	Overview
packages/host/engine/src/conversation/projection-service.ts	Introduces the core graph/read projection, attribution, live-tail merge, and pagination; failed-turn output, tail truncation, and snapshot drift need correction.
packages/host/engine/src/conversation/live-journal.ts	Adds bounded tracking for chunk streams whose earlier deltas were evicted.
packages/host/engine/src/conversation/request-handler.ts	Replaces unsupported read stubs with projection-backed graph and conversation responses.
packages/host/engine/src/session/history-service.ts	Adds settlement-aware cache freshness metadata used by projection history reads.
packages/foundation/schema/src/wire/conversation.ts	Extends graph/read frames with summaries, placeholders, item unions, paging cursors, and final-page-only watermarks.
packages/client/workbench/src/mock/dev-mock-host.ts	Adds minimal turn graph, submission, graph-read, and prompt-only projection behavior to the development host.

Sequence Diagram

sequenceDiagram
  participant Client
  participant Handler as ConversationRequestHandler
  participant Projection as ConversationProjectionService
  participant Store as ConversationStore
  participant History as Provider History
  participant Journal as Live Journal

  Client->>Handler: conversation.read(session, leaf, cursor)
  Handler->>Projection: read(request)
  Projection->>Store: load graph and root-to-leaf turns
  Projection->>History: read provider corpus
  Projection->>Projection: fingerprint-gated turn attribution
  Projection->>Journal: snapshot events above durable cut
  Projection->>Projection: compose host rows + history + live tail
  Projection->>Projection: page by offset and byte budget
  Projection-->>Handler: events, cursor or final watermark
  Handler-->>Client: conversation.read.result

Loading

Comments Outside Diff (1)

1. packages/host/engine/src/conversation/projection-service.ts, line 2053-2065 (link)

   Cursor allows snapshot drift

   The cursor pins the graph shape and terminal-turn count, but not the provider-history snapshot, even though each page recomposes the durable projection. Provider history can gain final rows after a turn settles, and the history cache can expire between pages without changing any cursor field. The cursor is then accepted against a differently shaped projection, so its item offset can skip or duplicate conversation items instead of returning a conflict.

_{Reviews (1): Last reviewed commit: "fix(engine): structured conversation cur..." | Re-trigger Greptile}

[greptile-apps]

Greptile has paused reviews on this repository — it used its 100 free open-source review credits for this billing period. Reviews resume automatically on September 23. To continue before then, an organization admin can keep reviews running past the free credits — those bill as normal usage.

[pullfrog]

Your Claude subscription has hit its usage limit. It resets at 11:30am (UTC). Re-trigger Pullfrog after the reset, or add an ANTHROPIC_API_KEY repo secret — Pullfrog routes around an exhausted subscription automatically when one is present.

Add repo secret → · Model settings → · Setup docs → · Ask in Discord →

  ^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[Copilot]

🟡 Changes recommended

The CODE-35 backstop/deduping logic can drop still-open interactive asks (and their responding state) from conversation.read under tail trimming/truncation conditions.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Implements Phase 1 of the engine-side “conversation turn graph & turn-scoped projection” read surfaces by adding host-composed conversation.graph.get and cursor-paged conversation.read, including provider-history attribution gating, a bounded live tail merge, and dev-mock parity.

Changes:

• Add ConversationProjectionService to compose a root→leaf projection from durable turns/prompts, provider history (fingerprint-gated), and the bounded live journal tail (watermark-merged).
• Add wire handlers for conversation.graph.get and conversation.read, plus schema updates for paged read items (events + placeholders) and graph turn inputSummary.
• Extend journal/history plumbing (in-flight chunk eviction tracking; history cache freshness bound) and add/refresh tests + dev mock responses.

File summaries

File	Description
packages/host/engine/src/session/orchestrator.ts	Exposes live-session interactive requests for projection/backstop reads.
packages/host/engine/src/session/history-service.ts	Adds freshAfter cache-bypass bound and tracks builtAt for event-cache entries.
packages/host/engine/src/engine.ts	Wires ConversationProjectionService into the engine runtime and request handler.
packages/host/engine/src/conversation/turn-service.ts	Exposes prompts and tracks last terminal-settle timestamps for cache freshness.
packages/host/engine/src/conversation/request-handler.ts	Handles conversation.graph.get and conversation.read requests via projection service.
packages/host/engine/src/conversation/projection-service.ts	New: composes graph/read projections, paging, cursor integrity, and live-tail merging.
packages/host/engine/src/conversation/live-journal.ts	Tracks evicted in-flight stream keys so headless chunk tails can be cleared.
packages/host/engine/src/tests/engine-conversation-stubs.test.ts	Removes obsolete “unimplemented stubs fail loudly” test now that reads are implemented.
packages/host/engine/src/tests/engine-conversation-read.test.ts	New: end-to-end tests for graph/read, paging, watermark behavior, freshness, and tail merge.
packages/host/engine/src/tests/conversation-projection.test.ts	New: focused unit tests for projection attribution, tail semantics, paging, and cursor integrity.
packages/host/engine/src/tests/conversation-live-journal.test.ts	Extends tests for in-flight stream eviction/clearing and keying logic.
packages/foundation/schema/tests/contract/wire/conversation.test.ts	Updates wire contract tests for new read/graph result shapes and validation.
packages/foundation/schema/src/wire/index.ts	Re-exports new conversation wire types/schemas (read items, placeholders, graph turns).
packages/foundation/schema/src/wire/conversation.ts	Adds placeholder/read-item schemas, graph turn inputSummary, and optional watermark for non-final pages.
packages/client/workbench/src/mock/dev-mock-host.ts	Adds dev-mock responses for turn submit + graph/read parity and minimal graph storage.
packages/client/workbench/src/mock/tests/dev-mock-conversation.test.ts	New: dev-mock conversation parity tests for submit/graph/read and loud parameter failures.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[pullfrog]

Your Claude subscription has hit its usage limit. It resets at 11:30am (UTC). Re-trigger Pullfrog after the reset, or add an ANTHROPIC_API_KEY repo secret — Pullfrog routes around an exhausted subscription automatically when one is present.

Add repo secret → · Model settings → · Setup docs → · Ask in Discord →

  ^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Zerlight]

Review round (2026-09-07). Verified every Copilot and Greptile finding against the branch. Fixes landed on the top of the stack (#517) because the code moved: the attribution gate lives in lineage-attribution.ts since #511 and the read path evolved through #512.

Fixed — 9b7e5f9e: a mid-run failed turn that left provider rows no longer breaks attribution (it was worse than reported — the whole lineage read as placeholders); the gate now aligns around failed turns from the start, renders a failed turn's partial output under it when the corpus proves every failed turn left a row, consumes nothing when none did, and keeps the prefix before the first failed turn when the footprint is ambiguous. Fork cuts use the next verified row. Four tests.

Fixed — ed32870b: the read cursor also pins the durable item count, so a provider corpus that gained or lost rows between pages yields a typed conflict instead of a shifted offset (Greptile's outside-diff item); the CODE-35 backstop replays the prompt-response-status of an open ask when the tail no longer carries it; the mock formats an empty command argument like the daemon.

Declined, with the reasoning on the threads: tail trimming is unreachable in production (journal cap 10 MiB, page budget ~16 MiB — now stated in a constraint comment on the budget), which also covers the trim half of the backstop concern.