feat(schema,daemon): persist the conversation turn tree and prompt records - #507
feat(schema,daemon): persist the conversation turn tree and prompt records#507Zerlight wants to merge 12 commits into
Conversation
…ry persist atomic
|
Your Claude subscription has hit its usage limit. It resets at 11:30am (UTC). Re-trigger Pullfrog after the reset, or add an Add repo secret → · Model settings → · Setup docs → · Ask in Discord →
|
There was a problem hiding this comment.
🟡 Changes recommended
There are correctness/concurrency issues in the daemon persistence layer and a schema-load edge case that can cause session record parsing failures.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR lays the Phase 1 foundation for durable conversation history by introducing a host-authoritative conversation turn tree (turn identity, prompt records, provider bindings, operation journal), adding corresponding wire frames, and persisting the data in the daemon via a single-connection SQLite-backed ConversationStore.
Changes:
- Added new schema models (
ConversationTurn,PromptRecord,ProviderTurnBinding,ConversationOperation) plus new primitives (TurnId,PromptId,RunId, etc.) and session-record fields (graphRevision,activeLeafTurnId, fork provenance). - Introduced additive wire protocol frames for turn submission and reading/observing conversation graphs; bumped
WIRE_PROTOCOL_VERSIONto 80 and added delivery scoping forconversation.graph.changed. - Implemented a
ConversationStoreseam (in-memory default + daemon SQLite implementation) and wired it into engine runtime lifecycle, including deletion cleanup.
File summaries
| File | Description |
|---|---|
| packages/host/engine/tests/integration/run-command.test.ts | Increased timeout to reduce flake in timeout/kill integration test. |
| packages/host/engine/src/wire/request-router.ts | Routed new conversation request kinds to a dedicated handler. |
| packages/host/engine/src/session/session-record-registry.ts | Added runId minting and ensured runs always get a durable id. |
| packages/host/engine/src/session/orchestrator.ts | Ensured session deletion also deletes conversation-store session data. |
| packages/host/engine/src/session/lifecycle-service.ts | Minted runId at session creation/import/start and initialized graphRevision. |
| packages/host/engine/src/index.ts | Re-exported ConversationStore surface and related types/errors. |
| packages/host/engine/src/engine.ts | Added ConversationStore dependency with in-memory default; registered handler. |
| packages/host/engine/src/deps.ts | Added optional conversationStore dependency for engine embedders/daemon. |
| packages/host/engine/src/conversation/request-handler.ts | Added stub handler that fails loudly for new wire kinds. |
| packages/host/engine/src/conversation/conversation-store.ts | Defined ConversationStore interface + in-memory implementation. |
| packages/host/engine/src/tests/engine-session-records.test.ts | Updated fixtures for runId + graphRevision. |
| packages/host/engine/src/tests/engine-session-lifecycle.test.ts | Added coverage ensuring session delete cleans conversation rows and GC’s prompts. |
| packages/host/engine/src/tests/engine-conversation-stubs.test.ts | Added test ensuring conversation requests are refused (not silently dropped). |
| packages/host/engine/src/tests/conversation-store.test.ts | Added in-memory store tests for atomic intent persistence, gating, GC, and resolve semantics. |
| packages/foundation/schema/tests/contract/wire/session.test.ts | Added compatibility tests for v79 session frames and legacy runs without runId. |
| packages/foundation/schema/tests/contract/wire/conversation.test.ts | Added contract tests for new conversation wire frames and delivery scoping. |
| packages/foundation/schema/src/wire/payload.ts | Added payload refinement for turn.submit parent/revision guard coupling. |
| packages/foundation/schema/src/wire/message.ts | Bumped wire protocol version to 80. |
| packages/foundation/schema/src/wire/index.ts | Exported conversation wire types/schemas. |
| packages/foundation/schema/src/wire/delivery.ts | Added session-scoped delivery for conversation.graph.changed. |
| packages/foundation/schema/src/wire/conversation.ts | Added conversation-related wire variants and schemas. |
| packages/foundation/schema/src/wire/agent.ts | Extended agent.event with optional run/turn attribution + watermark coords. |
| packages/foundation/schema/src/model/session/record.ts | Added fork provenance, run identity/base, and graph fields to session record model. |
| packages/foundation/schema/src/model/primitives.ts | Added new branded primitives for turn/prompt/run/attachment/operation ids. |
| packages/foundation/schema/src/model/index.ts | Exported new conversation model. |
| packages/foundation/schema/src/model/conversation.ts | Added conversation model schemas (turns, prompts, operations, watermark). |
| packages/foundation/schema/src/model/tests/conversation.test.ts | Added schema tests for conversation model invariants. |
| packages/client/workbench/src/mock/dev-mock-host.ts | Updated mock session import to include graphRevision. |
| apps/daemon/tests/integration/session-store.test.ts | Updated session-store integration tests for runId persistence. |
| apps/daemon/src/session-store.ts | Persisted runId, fork provenance, and graph fields; made missing runId a hard error on save. |
| apps/daemon/src/index.ts | Wired daemon to inject a SQLite-backed ConversationStore. |
| apps/daemon/src/db/schema.ts | Added new conversation tables + indexes and extended session/session_runs schema for new fields. |
| apps/daemon/src/conversation-store.ts | Implemented single-connection SQLite ConversationStore with transactional intent persistence and GC. |
| apps/daemon/src/tests/session-store.test.ts | Added tests for fork provenance round-trip and save-time refusal of runId-less runs. |
| apps/daemon/src/tests/conversation-store.test.ts | Added comprehensive SQLite conversation-store round-trip and concurrency/GC tests. |
| apps/daemon/drizzle/meta/0012_snapshot.json | Drizzle snapshot update for latest migration state. |
| apps/daemon/drizzle/meta/0011_snapshot.json | Drizzle snapshot update for prior migration state. |
| apps/daemon/drizzle/meta/0010_snapshot.json | Drizzle snapshot introducing new session/session_runs columns. |
| apps/daemon/drizzle/meta/_journal.json | Drizzle journal updated with new migrations. |
| apps/daemon/drizzle/0012_overjoyed_risque.sql | Added unique/partial indexes for open-op and sibling ordinal constraints. |
| apps/daemon/drizzle/0011_flippant_bastion.sql | Added conversation turn-tree tables. |
| apps/daemon/drizzle/0010_chilly_swordsman.sql | Added runId/baseTurnId + fork provenance + graph fields; backfilled existing runs with runId. |
Review details
- Files reviewed: 42/42 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Greptile SummaryThis PR introduces the schema and durable-storage foundation for conversation turn graphs, prompt records, provider bindings, operation journaling, and explicit session run identity.
Confidence Score: 4/5The PR appears safe to merge with non-blocking persistence-hardening follow-ups for graph ancestry, immutable prompts, and turn-state upserts. No currently reachable production request failure was established because conversation requests remain unsupported, but the new public persistence seam permits cross-session ancestry and inconsistent immutable-record behavior that should be corrected before building the submit saga on it. Files Needing Attention: apps/daemon/src/conversation-store.ts
|
| Filename | Overview |
|---|---|
| apps/daemon/src/conversation-store.ts | Adds transactional SQLite persistence for turns, prompts, bindings, operations, and prompt GC; relational and immutable-record invariants need strengthening. |
| apps/daemon/src/db/schema.ts | Adds conversation tables, graph indexes, session graph metadata, and run identity columns. |
| packages/host/engine/src/conversation/conversation-store.ts | Defines the conversation persistence seam and in-memory implementation, including atomic intent and terminal-operation transitions. |
| packages/foundation/schema/src/model/conversation.ts | Defines turn, prompt, provider-binding, operation, and watermark contracts. |
| packages/foundation/schema/src/wire/conversation.ts | Adds additive wire-v80 conversation submit, graph, read, and change frames. |
| apps/daemon/src/session-store.ts | Persists graph metadata, fork provenance, run IDs, and base-turn IDs with legacy migration support. |
| packages/host/engine/src/conversation/request-handler.ts | Routes the declared conversation surface to explicit typed unsupported responses until later implementation phases. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
C[Client conversation frame] --> R[Conversation request handler]
R -->|Phase 1| U[Typed unsupported reply]
R -. future submit saga .-> S[ConversationStore.persistTurnIntent]
S --> T[(conversation_turns)]
S --> P[(prompts)]
S --> O[(conversation_operations)]
T --> B[(provider_turn_bindings)]
T -->|prompt_id| P
T -->|parent_turn_id| T
D[Session deletion] --> G[Conversation graph and prompt GC]
G --> X[Session record deletion]
Reviews (1): Last reviewed commit: "fix(conversation): report the resolve tr..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
Your Claude subscription has hit its usage limit. It resets at 11:30am (UTC). Re-trigger Pullfrog after the reset, or add an Add repo secret → · Model settings → · Setup docs → · Ask in Discord →
|
|
Review round (2026-09-07). Verified every Copilot and Greptile finding against the branch. Fixed — Declined, with the reasoning on each thread: the turn/operation insert order (admission is gated by an explicit open-operation check inside one synchronous transaction; the unique indexes are backstops), Stack tip after this round: |
Summary
Phase 1 of CODE-627 — Conversation turn graph & immutable attachment store. Linear: https://linear.app/arcbox/issue/CODE-628/featschemadaemon-persist-the-conversation-turn-tree-and-prompt-records
Stack:
master← this PR (ruocheng/code-628, basemaster) ← #508. Merge bottom-up; this PR's diff is only its own commits.Phase 1 foundation: host-authoritative turn identity and durable user prompts. zod schemas for
ConversationTurn,PromptRecord, andProviderTurnBinding;SessionRecordgainsactiveLeafTurnId/graphRevisionand runs gain an explicitrunId/baseTurnId; additive wire frames (turn.submit,conversation.graph.get/.changed,conversation.read) under wire 80; drizzle tables behind oneConversationStoreconnection; the legacy null-prompt migration renders migrated sessions exactly as before.Commits
Verification
Every commit passed
pnpm check:ciandpnpm testat its own tip; the stack tip (1d942a62; the same tree as the originally gated73fc5ff2plus the two review fixes below on 628/629) is atpnpm check:ci0 errors,pnpm test3408 passed / 1 skipped. Adversarial reviewers (one per axis, isolated read-only worktrees) reviewed the branch; each P1/P2 was reproduced with a failing test or a probe step before its fix — the round-by-round record is in the Linear issue's comments.Checklist
pnpm check:ciandpnpm testboth pass (no Rust changes)AGENTS.mdand module docs in this branch)