Sans-IO engine and harness product

[vinniefalco]

Sans-IO engine and harness product

All paths are relative to the PromptForge repository root. Repository facts are cited by path; every fact is included so a reader without the conversation can act on this plan.

Status: decomposed into 51 steps and reviewed against every AGENTS.md in the repository on 2026-09-18 (see the "AGENTS.md review" entry in the Decision Record). No step has started. Execution begins at Step 1; Steps 1-7 touch no engine crate and may run on a branch alongside Steps 8-40.

Product Requirements

PromptForge runs Markdown prompts whose sections contain Lua. Today the component that runs them (the executor) mixes a pure scheduler with the machinery that talks to the outside world: it spawns tokio tasks for network calls, builds an HTTP client, holds callbacks into the host, and implements two author features (models.loop and fanout) in Rust in ways that block or serialize other work. This plan splits it into an engine that never touches the outside world and a harness product that does all of it, records everything, and is what Workshop and Papergate depend on. The engine's whole host interface becomes four functions.

Terms used throughout

• Prompt: a Markdown file with front matter, an H1 title, and H2 sections; each section holds prose and fenced Lua blocks. Source: crates/promptforge/parser/.

• Section VM: the sandboxed Lua state created fresh for each section entry. No Lua state survives leaving a section.

• Chain: one line of section execution inside a run (walk the sections in order, jump, fall through). The scheduler keeps an arena of chains. A call(target) starts a child chain and blocks the caller; fanout starts many.

• Yield and shim: Lua cannot suspend across a call into Rust, so every suspending author function (models.infer, call, tools.call, ...) is a few lines of Lua that coroutine.yield a request table and receive an answer. The scheduler validates the yield, acts on it, and resumes the coroutine. Source: crates/promptforge/lua/src/coro.rs, __impl_coro.lua.

• Structural yield: a request the scheduler answers by itself (start a chain, wait for a task). Leaf yield: a request that needs the outside world (a model round, a tool call, operator input, a store operation, a timer).

• Effect: a leaf yield turned into a value the engine returns to its host; the host performs it and returns an EffectAnswer. The term is from algebraic effect handlers: the engine performs an effect, the harness handles it.

• Event: something the engine reports (a section started, a model replied, a tool ran). Returned as values alongside effects; replaces today's Observer callback trait.

• Task: a chain started with spawn and tracked by id so its owner can wait on it, inspect it, or cancel it. fanout becomes a Lua function over tasks.

• Engine: the promptforge-* crates after this plan. Pure: no async, no network, no clock, no callbacks.

• Harness: the new harness-* product. Owns tokio, performs every effect, keeps the run log, supervises sessions.

• var: the author's clipboard table that rolls forward across sections within a chain and is discarded when a call chain ends. Tasks follow var.

• Store and claims: the run-scoped virtual filesystem authors reach through store.*, with a claims model that detects two concurrent identities touching one path. Source: crates/shared-vfs/, crates/promptforge/store/.

• Capability: a host-installed bundle of tools a prompt declares in front matter (web search, shell, ...). Source today: crates/promptforge-api-types/src/capabilities.rs.

• Workshop: the desktop product (crates/workshop/). Papergate: an external consumer of the executor in its own repository.

• Problem and users:

  • The scheduler (crates/promptforge-api-runtime/src/execute/scheduler.rs) is already a coroutine driver with all Lua on one thread, but four things keep it from being a pure state machine: models.loop runs as Rust async on the driver thread holding the section VM across network waits, so while one fanout arm's loop waits on a model no other arm can be resumed (documented in dispatch_loop); fanout is about 600 lines of scheduler-internal join code; leaf I/O is tokio::spawned inside the crate and collected on a channel; the HTTP client, capability registry, input broker, observer, debug capture, and ui() snapshot are all host callbacks reaching into or out of the run.
  • Users: prompt authors (Lua surface), models running inside prompts (tool surface), the Workshop desktop app and Papergate (host surface), and maintainers who need a testable core.

• Goals:

  • The engine crates (promptforge-api-runtime, promptforge-api-types, everything under crates/promptforge/) declare no dependency on tokio, tokio-util, async-trait, or reqwest, enforced by a test on declared manifest dependencies.
  • The engine's host interface is Run::new, Run::step, Run::resume, Run::cancel. step returns the effects to perform and the events produced; nothing in the engine awaits, blocks, reads a clock, or calls a host callback.
  • Fanout arms that run models.loop interleave at every model round and tool call, so N arms have up to N model rounds in flight.
  • Authors gain spawn, when_any, when_all, ready, status, note, events, cancel under a tasks namespace; fanout keeps its signature and semantics.
  • Models gain background tasks and a bounded blocking wait, in the same shape Cursor gives its own agent: start, cancel, inspect, await with a timeout, results pushed as messages.
  • A harness-* product owns the effect loop, every performer, session supervision, and a Turso-backed log of every effect, answer, and event in one ordered stream.
  • workshop-sessions is dissolved into the harness; Workshop and Papergate depend on harness-api.

• Non-goals:

  • Replaying a recorded run, or resuming a cancelled task by re-execution. The log is written so both are possible later; nothing reads it back into the engine. Their contracts are recorded under Deferred.
  • A live clock in Lua (now()), an author-visible timer task, Lua string-hash seed control, parallel Lua within one run, the compactor framework, any gateway product change.

• Success criteria:

  • cargo check -p promptforge-api-runtime succeeds, and cargo test -p build-xtask proves that its non-dev dependency tables, and those of every crate under crates/promptforge/, contain none of the forbidden crates.
  • A plain #[test] drives a three-arm fanout to completion with a serial performer, feeding answers in reverse order, with no tokio and no mock HTTP server.
  • Two runs with the same seed, started_at, and answers produce identical effect and event sequences, including every Provenance and sys.id, on prompts that avoid Lua pairs.
  • Workshop's agent integration suites pass against harness-api with import and construction changes only; their assertions are unchanged.
  • Every existing engine test suite passes through a test-support driver (which adapts the returned event stream to the recording observers those suites install), or is rewritten at prompt level where it called the deleted Rust loop directly.

• Constraints:

  • One thread runs every chain step; the scheduler is unreachable from Lua; the coroutine global is stripped after the shims capture yield (crates/promptforge/lua/src/coro.rs).
  • Yield cannot cross the C boundary, so every author-visible suspending function is a Lua shim, never an mlua callback.
  • Claims model: a chain's store access is spawned from its parent's at chain start and released at chain end before any waiter resumes; a run result is never delivered while an in-flight store operation still holds an access clone.
  • Typed errors never flatten: when an answer fails, the scheduler keeps the typed error and substitutes it when the shim's Lua error() surfaces as the coroutine's failure.
  • Product matrix (crates/build-xtask/src/product.rs): families by name prefix, private containers with one named public door, shared-* depends on no product. New structural checks need explicit approval; the user approved four in this plan (2026-09-18): the harness family row, the engine manifest test, the retired-symbol source scan, and the harness clippy.toml check. Extending the existing 500-line ceiling check to harness-* crates is a scope change to a check that already exists, not a new check.
  • Workspace lints: unsafe_code forbidden, unwrap_used/expect_used denied, pedantic clippy; files under 500 lines; flat source directories; Cargo features gate real constraints only.
  • {{ }} prose substitution stays data-only; no call syntax is added.

• Open questions: None.

Functional Specification

Three actors see the change. Prompt authors keep every function they have and gain a tasks namespace for background work with timeouts. Models inside a prompt gain five tools, enabled by the author, for starting, inspecting, awaiting, and cancelling background tasks, with results arriving as messages. Hosts drive a run by asking the engine what it needs, doing it, and handing the result back; the harness is the one host in the workspace and Workshop talks to it.

• Actors and workflows:
  • Author, existing surface unchanged: models.infer(handle?, prompt), models.loop(handle?, messages, compactor?), call(target, input?), fanout(worker, collection), tools.call(alias_or_tool, args), user_input(), store.*. models.loop and fanout are now written in Lua but behave the same, except as listed under acceptance criteria.
  • Author, new tasks namespace (available in every section and in the H1 pass):
    • spawn(target, opts?) -> Task starts a chain over section target and returns at once. opts.input overrides the chain's args; opts.item becomes the item global and {{ item }} in the target; opts.index becomes sys.index. The caller's var seeds the chain. Depth is the caller's plus one, capped as call is.
    • tasks.when_any(set, opts?) -> Task, ok, result waits until the first task in set finishes and returns which one, whether it succeeded, and its final text or error value. opts.timeout (seconds) returns nil if nothing finished in time; the tasks keep running.
    • tasks.when_all(set, opts?) -> results, timed_out waits for every task and returns { task, ok, result } per member in input order. It never raises because a member failed; the author decides. With a timeout, unfinished members are absent and timed_out is true.
    • tasks.ready(task), tasks.pending(filter?), tasks.cancel(task): non-blocking check, list of the caller's live tasks (optionally by origin author or model), abort.
    • tasks.status(task) -> table: target, origin, state (running/done/cancelled/abandoned), ok, current section, what it is blocked on (chat, tool_call, user_input, store, timer, tasks, call, or nil), turns, owned tasks, depth, latest note. No elapsed time, because the engine has no clock.
    • tasks.note(text): from inside a task, publish a one-line progress note visible in status.
    • tasks.events(task, opts?) -> sequence: the task's content events so far ({ kind, section, turn, text }), answered from the harness's log, so unbounded; opts.last = n for the most recent n.
    • Task is a plain table { task = id } with no methods (Lua host handles are methodless per crates/promptforge/lua/AGENTS.md and archdoc A9; every operation is a tasks.* namespace function); every tasks.* function accepts the table or the bare integer, so a handle stored in var works unchanged.
  • Author, ownership rules (tasks follow var): only the spawning chain may wait on, inspect, or cancel a task, with one addition: a chain may call tasks.status, tasks.events, and tasks.note on its own task (sys.taskid), which is how a task reports progress and how an agent reads its own history. Tasks survive jump and fall-through, keep running while the owner is blocked in call or a wait, transfer from the H1 pass to the main walk, and end when their owner's call chain or spawned chain ends. A chain that ends with live author-spawned tasks fails with an error naming them; aborting a chain aborts everything it owns.
  • Model, enabled by the author calling tools.allow_tasks(targets?) in a section (targets optionally restricts which sections may be started):
    • task { target, input? } starts a background chain and returns Task id=N started.
    • task_cancel { id }, task_status { id }, task_events { id, last? } mirror the author functions; status is trusted, events are marked untrusted because they contain another chain's model output.
    • await_tasks { timeout? } blocks the model's tool call until one of its tasks finishes or the timeout passes, returning the finished tasks' results, or timed out; tasks 3, 5 still running. With no tasks and a timeout it is a sleep. With neither it returns nothing to wait for.
    • Results the model did not await are appended as messages (Task id=N (## Heading) completed: ..., failed: ..., was canceled: the author cancelled it after an explicit tasks.cancel, was abandoned: <why the owner ended> with the reason the section ended, the tool loop was exhausted, or the owner failed) before its next model round. A chain ending with live model-started tasks abandons them and records TaskAbandoned; the model is not told because it has no next round in that chain.
  • Harness (host of the engine): parse the prompt; resolve declared capabilities and assemble a tool catalog; build a RunContext with a fresh random seed and the wall-clock start time; call Run::new; loop: step, perform each returned effect on tokio, log every effect, answer, and event, resume each answer as it arrives, until Done.
  • Workshop: opens sessions through harness-api, renders the event and delta streams, supplies operator input, builds the ui() snapshot, and pushes the gateway binding (base URL, key, generation) to the harness whenever the gateway it supervises is started or replaced; the harness rebuilds its capability registry on each push, as today's EffectExecutor does on a generation change. Papergate: switches its dependency from the engine to harness-api and supplies its own gateway binding the same way.
• Inputs and outputs:
  • Engine in: a parsed prompt (shared through Arc), its args string, and a RunContext (name, seed, started_at, limits, current model, per-run VFS, tool catalog, filled tool and model bindings, ui snapshot, cancel handle). Engine out per step: a list of (EffectId, Effect) to perform and a list of Events produced, or the final RunResult with the last events.
  • Effects: Chat (one model round: binding, messages, the concrete list of advertised tool schemas, options), ToolCall (tool id, alias, JSON args), UserInput, Store (a store operation with the chain's access handle), Timer (seconds), TaskEvents (task id, optional last-n). Answers mirror them, plus Dropped meaning the host will not perform this effect. Each effect has a serializable projection, EffectRecord, which is the effect minus live handles (the store access); the log stores records, and only records deserialize.
  • Harness out: session events and streaming deltas to its client; a Turso database with runs and records tables.
• States and validation:
  • A run is Pending (effects may be outstanding) until Done. Done is never reported while any issued effect is unanswered; the host must answer or drop every effect first. This preserves the claims rule that a store operation's access handle is released before the result is delivered.
  • A task is running, done (result undelivered), delivered, cancelled (someone called cancel on it), or abandoned (its owner chain ended while it was live, so the engine ended it). abandoned is a distinct terminal state so the log and the model notice can tell "was stopped on purpose" from "lost its owner". Waiting on a delivered task is an error; cancelling anything is idempotent.
  • {{ }} paths must resolve to JSON data; a function, userdata, or thread is rejected. sys holds only data fields (when, id, model, index, taskid); sys.now is removed.
  • Structural identity is deterministic: every chain (the main walk, a call child, a spawned task) has a hierarchical chain id, its parent chain's id extended by the parent's local child counter, with call children and spawns sharing that counter; a task's id is its chain's id; a section's sys.id is its chain's id extended by the chain's local entry counter. Two runs with the same inputs produce the same ids regardless of how their chains interleave, and a call child never collides with its parent because they are different chains.
• Errors and recovery:
  • Every failure that reaches Lua is a table { kind, message, ... } whose tostring is the message, so pcall callers that print it see no change and callers that branch can read kind. Kinds: tool_loop_exhausted, context_exhausted (with reason), empty_model_reply (with finish_reason), out_of_scope_tool, unbound_tool, tool, task_not_owned, task_consumed, tasks_live, cancelled, lua, internal.
  • A model-issued tool call whose tool fails yields the failure text to the model as an untrusted result and the loop continues; a script-issued tools.call whose tool fails raises at the call site. Model tasks that outlive their owner are abandoned (ended by the engine, recorded as TaskAbandoned); author tasks that outlive their owner are a hard error. The two principals are treated differently on purpose: the author's leak is a bug, the model's is recoverable.
  • Store claims violations still end the run without resuming Lua. Cancellation from the host aborts every chain and reports Cancelled once outstanding effects are answered or dropped.
• Security and privacy behavior:
  • Trust boundaries are unchanged: bound tool output and any text from another chain's model (task results, task_events) is wrapped as untrusted before a model sees it; task_status is trusted because it is scheduler fact.
  • The engine holds no credentials and opens no connections; the harness holds the model client and capability implementations, as workshop-sessions does today.
  • The run log contains model inputs and outputs verbatim, as the current JSONL session log does; it lives in the same state directory.
• Acceptance criteria:
  • Behavior changes an author can observe, all accepted: fanout arms running loops interleave instead of serializing; models.loop counts against the Lua instruction quota (a few hundred instructions per round); a section ending with live author tasks fails (no existing prompt spawns tasks); pcall error values are tables (nothing in prompts/ or tests compares one to a string); fanout iterates a hash-shaped collection in sorted key order instead of undefined order; ui() is the snapshot taken at run start (the documented contract already says a change takes effect on the next run); sys.now is removed and two guide sentences change (guide/src/language/04-lua-globals-and-store.md); sys.id values change form but remain unique within a run.
  • Everything else authors can observe is unchanged, including every error message text that fanout and models.loop produce today.

Technical Design

The engine keeps the scheduler's existing shape (one thread, a chain arena, yield and resume) and removes everything that reached outside it. Two author features move from Rust into the Lua shim file so that every network wait inside them becomes an ordinary yield. A task arena replaces the fanout join machinery. The host boundary becomes four methods on Run, exchanging effects and events as serializable values. The harness is a new product family that performs effects on tokio, logs them to Turso, and absorbs the session machinery from Workshop.

flowchart LR
    WS["Workshop"] --> HAPI["harness-api"]
    PG["Papergate"] --> HAPI
    HAPI --> Runner["harness runner"]
    Runner -->|"step()"| Engine["Run"]
    Engine -->|"effects, events"| Runner
    Runner -->|"resume(id, a)"| Engine
    Runner --> Caps["capabilities"]
    Runner --> Models["model client"]
    Runner --> Log["Turso run log"]
    Engine --> Sched["scheduler"]
    Sched --> VM["section VM"]
    VM -->|yield| Sched

Loading

• Architecture:
  • Engine (promptforge-api-runtime and its private crates under crates/promptforge/): a deterministic state machine. Given the same RunContext and the same sequence of answers it produces the same effects, events, and ids. It performs no I/O, reads no clock, and holds no host trait objects.
  • Harness (harness-*): the engine's only production host. It owns the tokio runtime, one performer per effect kind, the model HTTP client (moved from crates/promptforge/model-client/src/client/), the capability registry and first-party capabilities (moved from crates/promptforge-api-types/src/capabilities.rs and crates/promptforge/{web,webfetch,web-search}/), the input wait registry and supervisor (moved from crates/workshop/sessions/), and the run log.
  • Family rules added to crates/build-xtask/src/product.rs: harness-* may depend on promptforge-api-runtime, promptforge-api-types, gateway-api, gateway-api-discovery, and shared-*, never on workshop-* or private gateway-* crates; workshop-* may depend on harness-api; promptforge-* and gateway-* never depend on harness-*. Container crates/harness/ is private with harness-api as its door, the same shape as crates/promptforge/ with promptforge-api-runtime.
  • Layout: crates/harness-api/ (public), crates/harness/runner/ (effect loop, performer traits, cancellation, supervision), crates/harness/models/ (model client), crates/harness/capabilities/ (registry, activation, and the Capability, Tool, and InputBroker traits; depends on no provider), crates/harness/{web,webfetch,web-search}/ (the first-party capabilities as harness-web, harness-webfetch, harness-web-search; the two providers depend on harness-capabilities for the Tool trait, and harness-sessions depends on all of them to register the first-party set), crates/harness/log/ (Turso), crates/harness/sessions/ (discovery, session state, waits, supervisor state machine).
• Modules and interfaces:

  • Run (crates/promptforge-api-runtime/src/execute/run.rs, new):

    pub struct Run { /* Arc<Prompt>, run state, scheduler, cancel flag, seed */ }
    pub enum Step {
        Pending { effects: Vec<(EffectId, Provenance, Effect)>, events: Vec<Event> },
        Done { result: RunResult, events: Vec<Event> },
    }
    // Every Event variant carries `provenance: Provenance` beside `execution` and `section`.
    pub enum Effect {
        Chat { binding: ModelBinding, messages: Vec<Message>, tools: Vec<ToolSchema>, options: CompletionOptions },
        ToolCall { tool: ToolId, alias: String, args: serde_json::Value },
        UserInput { execution: String, section: String },
        Store { access: Arc<Access>, op: StoreOp },
        Timer { seconds: f64 },
        TaskEvents { task: TaskId, last: Option<u32> },
    }
    pub enum EffectAnswer {
        Chat(Result<Completion, CompletionError>),
        ToolCall(Result<ToolOutput, ToolError>),
        UserInput(Result<InputOutcome, InputError>),
        Store(Result<StoreOutcome, shared_vfs::VfsError>),
        Timer,
        TaskEvents(Vec<Event>),
        Dropped,
    }
    impl Run {
        pub fn new(prompt: Arc<Prompt>, args: &str, ctx: RunContext) -> Run;
        pub fn step(&mut self) -> Step;
        pub fn resume(&mut self, id: EffectId, answer: EffectAnswer);
        pub fn cancel(&mut self);
    }

  • Run contract: step drains the ready queue and returns when no chain can proceed without an answer, or when the run is over. Pending with an empty effect list means "waiting on effects already issued." Done is withheld while any effect is unanswered. resume applies one answer, buffers that round's events, and re-queues the chain; an unknown id is an internal error; Dropped resumes the chain with a cancelled error and is itself an answer, so the harness writes an answer row for it and every effect in the log has exactly one answer. cancel sets a flag the Lua instruction hook already polls; the next step aborts every chain. Run is Send; one caller at a time; the thread may change between calls. The contract is incremental: one resume per arriving answer, then step, so an arm advances while its siblings' effects are still in flight.

  • Event (crates/promptforge-api-types/src/event.rs, replacing observe.rs and the Observer and DebugCapture traits): one enum. Lifecycle variants for run, section, model turn, tool call, input wait, and store operations (one per member of today's Observation enum), plus TaskStarted { task, target, origin, input, item, index, var } carrying the spawn seeds, TaskSucceeded, TaskFailed, TaskCancelled, TaskAbandoned (the owner ended first), and TaskResumed (reserved, unused until resume lands). Content variants: Thinking, AssistantReply, AssistantToolCalls, ToolResult, UserInput, TaskNotice, TaskNote. Debug variants: Request, Response. Every variant carries execution, section, and provenance; effects carry theirs in the Step::Pending tuple, (EffectId, Provenance, Effect), so the harness can write task_id and task_seq for every record without inspecting the payload.

  • Provenance { task: TaskId, seq: u32 } on every effect and event (not Origin: shared_vfs::observe::Origin already names the claims origin label one crate below, and TaskOrigin names the spawning principal; three Origins in one dependency chain would mislead): the nearest enclosing task (the main walk is task 0; a call child reports its parent's task, which is unambiguous because a call blocks its parent, so the two never interleave) and a per-task counter. This lets the harness slice its log by task and order within a task, and it is what a UI groups by; in durable-execution vocabulary it is the effect's replay key, and its doc comment says so. EffectId stays an opaque run-wide handle for in-flight correlation.

  • ReplayError { Nondeterminism, Fatal } in promptforge-api-types, defined now and unused until replay lands: Nondeterminism means a re-executed run or task issued an effect or event that disagrees with its record; Fatal means the record itself is malformed or internally inconsistent. The two are kept apart because the first is a property of the code under replay and the second of the log, and each demands a different remedy.

  • Flags (a #[repr(u32)] bitset, reserve-forever numbering) on RunContext and in the run record, empty in this plan. When a future engine change alters an exit rule or protocol detail, it runs the new behavior live and sets its flag, and a later replay honors the flag only if the original run recorded it. The gate function is deferred with replay; the field exists so the first such change has somewhere to record itself.

  • Event implements Serialize and Deserialize. Effect implements Serialize only through EffectRecord, its projection minus live handles (Effect::record(&self) -> EffectRecord); EffectRecord implements both. The store access handle cannot be deserialized into existence, and nothing in this plan reads an effect back into the engine, so the asymmetry costs nothing.

  • Protocol (crates/promptforge/lua/src/protocol.rs), the request vocabulary Lua yields: leaf requests Infer, Chat { messages, binding, tools }, ToolCall { alias, args, call_id }, UserInput, Store, TaskEvents; structural requests Call, Spawn { target, input, item, index, var, origin }, Timer { seconds }, WhenAny { tasks }, Ready, Status, Note, Cancel, Pending, DrainTaskNotices. Removed: Loop, Fanout, Mcp and their answers, parse_loop, parse_fanout, LuaFanoutResult, the registry-key plumbing that let Rust append to the author's message table, append_message_record, invoke_selected. A Chat from a section VM with tools: None means "the section's current tool scope, including local Lua tools"; the agent VM keeps passing an explicit list, and one arm serves both. ChatResult gains overflow (the request was refused before or by the provider as too large; no round ran) and reports an empty reply as a completed round with reply absent, so Lua applies the exit rules. ToolCall.call_id: Some marks a model-issued call: it always resumes with content (a tool's own failure becomes untrusted failure text), and ToolResult fires under that id; None is a script call and keeps today's behavior. A call to a local Lua tool is answered inside step on the parked chain's VM; no effect is issued. The five model built-ins (task, task_cancel, task_status, task_events, await_tasks) are recognized by name in the tool_call arm before alias lookup.

  • models.loop shim (crates/promptforge/lua/src/__impl_coro.lua): per round, drain pending model-task notices into messages; yield chat; on overflow call the compactor (default raises context_exhausted); on tool calls, yield one tool_call per call with its call_id, buffer the results, then append the assistant tool-call record and one tool record per result so the author's list never shows a half-answered batch; on a reply, append and return; on an empty reply with finish_reason == "stop" after at least one answered tool call, append an empty assistant record and return (the model's clean exit); otherwise raise empty_model_reply; after the iteration cap raise tool_loop_exhausted. New chunk captures beside yield and var_snapshot: max_tool_iterations, max_fanout_concurrency, compactors, raise(kind, fields), collection_members, render_item, drain_task_notices. The shim emits no events; the scheduler emits each round's events (turn advance, debug capture, turn completed or failed or truncated, thinking, reply or tool calls) when it applies the Chat answer, and rejects an out-of-scope tool name against the scope it advertised for that round.

  • fanout shim (same file): collection_members (array part in order, then hash part as { key, value } sorted by key); empty collection raises before any spawn; worker validation happens in the Spawn arm so its message is byte-identical; up to max_fanout_concurrency arms live, refilled on every when_any completion; results placed by collection index; tool_loop_exhausted in an arm becomes the incomplete stub (## <item>\n\nUNKNOWN\n\n(section incomplete: tool loop exhausted)) and the fanout continues; any other arm failure cancels the live arms and re-raises. when_all is not used because refill must happen between completions. The fanout cannot leak tasks: every arm is delivered by when_any or cancelled before the function returns or raises.

  • Scheduler (crates/promptforge-api-runtime/src/execute/scheduler.rs, split into scheduler.rs plus the scheduler/ directory holding dispatch.rs, tasks.rs, walk.rs in standard module layout, because a three-file group is a directory under the repository's flat-directory rule, and to stay under the 500-line ceiling): keeps chains, ready, pending, stack; adds tasks: HashMap<TaskId, TaskSlot { backing: Chain | Effect, owner, origin, target, state }> (an effect-backed slot is the internal timeout timer), and on each chain owner, waiting_on, advertised (the last round's tool scope), task_notices (undelivered model-task notices), note; keeps a run-level event buffer drained by step. Removes the fanout join tables and arm templates, the tokio channel, join handles, the abort bookkeeping, the lazy gateway client, and the run-global id counters. finish(chain) checks the chain's own live tasks (author-origin: the outcome becomes tasks_live; model-origin: abandoned, each slot set to Abandoned with a TaskAbandoned event carrying why the owner ended), then completes the chain's task slot and wakes a waiting owner or queues a notice. Terminal slots (Done, Cancelled, Abandoned) persist until their result is delivered or their owner ends, so status can report the terminal state; Cancelled and Abandoned are delivered to a waiter as ok = false with a cancelled or abandoned error value. abort_subtree also aborts every chain the aborted chain owns. A stall (nothing ready, nothing pending, nothing waiting) is an internal error. The H1 hand-off reassigns H1's tasks to the main walk chain. await_tasks reuses the WhenAny arm from the tool_call path: park on the chain's model tasks plus an optional timer, drain notices on wake, cancel an unfired timer, resume with the rendered text.

  • Identity: no run-global counters. Every chain has a hierarchical chain id: its parent chain's id extended by the parent's local child counter, which call children and spawned tasks share, so a parent and its call child are distinct chains with distinct ids. A task's id is its chain's id. A section's sys.id is its chain's id extended by the chain's local entry counter. EffectId is allocated from a run-wide counter because it is an opaque in-flight handle that need not reproduce. The encoding of chain ids and sys.id (packed integer or path string) is chosen after surveying what reads sys.id; the property is the requirement.

  • Tool bindings and Environment (crates/promptforge-api-runtime/src/execute/{bindings,environment}.rs): ToolBinding carries id, alias, schema, description, output kind, and conflicts, never an implementation. Environment keeps base_vfs, max_depth, and a ToolCatalog the host supplies; prepare builds the per-run VFS, fills tool slots by id against the catalog, fills model bindings against the current model, and reports Requirements. Capability activation and conflict checking leave the engine.

  • RunContext after the change: name, seed (u64, host-drawn; the nonce guard derives from it), flags (empty Flags), started_at (Timestamp, UTC milliseconds, rendered to RFC 3339 for sys.when by a std-only formatter), limits, model, vfs, tools (catalog), tool_bindings, model_bindings, ui (a JSON value snapshot), cancel (a sync CancelHandle: an AtomicBool parent-child tree with cancel, is_cancelled, child). Removed: observer, client, input_broker, on_delta, debug.

  • Harness effect loop (crates/harness/runner/): step; append events to the log; for each effect append it and spawn a performer (tokio::spawn, or spawn_blocking for Store) that sends (id, answer) on a channel; select! over the channel and the session's cancel; on an answer, append it and resume; on cancel, run.cancel(), abort in-flight performers, await blocking-pool store operations so their access handles release, answer each outstanding effect Dropped, then step to Done. Performers are one trait per effect kind: ChatPerformer (model client, streams deltas to the session), ToolPerformer (resolves the tool id against activated capabilities), InputPerformer (wait registry), StorePerformer, TimerPerformer (tokio::time::sleep; tokio's timer wheel multiplexes every pending sleep, so no harness-side heap is needed), TaskEventsPerformer (reads the log). Events are committed before the producing step's effects are issued, because a running task may read history through TaskEvents.

  • Harness run preparation: parse; resolve declared capabilities, check co-activation conflicts, activate with RunServices { vfs, cancel }, assemble the ToolCatalog and the id-to-implementation table; build RunContext with a fresh seed and started_at, both logged; Environment::prepare; fail on unmet requirements with today's model-readable notice; Run::new; loop.

  • harness-api exposes Harness (from config: agents path, state dir), Harness::set_gateway(binding) (base URL, key, generation; the client calls it at startup and on every gateway replacement, and the harness rebuilds its capability registry and model client when the generation changes, which is what EffectExecutor does today against workshop-gateway's snapshot), Session (launch, send input, cancel, close, subscribe to events and deltas), and the event and delta types clients render. The harness never depends on workshop-gateway; the binding is data pushed across the door.

  • Harness session lifecycle: a run is Alive, then Closing once cancel or close is requested (outstanding effects are being answered or dropped), then Closed once Run reports Done; the supervisor's pure transition reducer (moved from crates/workshop/sessions/src/agents/supervisor/transition.rs) gains one pure rule, effective_interrupt(interrupt, saw_terminal): a genuine terminal outcome that arrives before a late cancel or timeout wins, and the synthetic terminal frame for an interrupt is rendered in exactly one place. This replaces the hand-managed active_run.take(), finish_run, and generation bookkeeping in today's EffectExecutor. The reducer's matches stay wildcard-free so a new variant is a compile error, with a fixture-coverage test.

• File and public API changes:
  • promptforge-api-runtime: removes execute::run, Environment::run, execute/gateway.rs, GatewaySource, execute/tool_loop.rs, dispatch_loop, run_loop, the client module, and the dependencies tokio, async-trait, tracing, rand, time. Adds execute/run.rs. now_rfc3339_checked in execute/support.rs becomes an infallible formatter over Timestamp; Error::TimestampFormat goes. Public surface: Run, Step, Effect, EffectAnswer, EffectId, Event, Provenance, RunContext, RunLimits, Environment, Requirements, RunResult, RunError, Prompt, promptforge_version, types. A test-support feature provides a serial driver (drive(run, perform) -> (RunResult, Vec<Event>)), an adapter that replays a returned Vec<Event> into the recording-observer trait the existing suites install (so those suites compile and pass without rewriting their assertions), and, under dev-dependencies, a tokio driver with the existing axum mock gateway, so the current suites keep running while they migrate.
  • promptforge-api-types: removes tokio, tokio-util, async-trait, rand, the async Tool trait, and capabilities.rs (the InputBroker trait is in promptforge-api-runtime/src/input.rs and leaves from there); observe.rs becomes event.rs; events.rs (EventLog, RuntimeEvent, RuntimeEventKind) is deleted, its read-side role passing to the TaskEvents effect; adds Timestamp, Provenance, TaskId, ReplayError, Flags. Keeps ToolId, ToolSchema, ToolOutput, ToolError, InputOutcome, InputError, catalogs, metrics, untrusted guards.
  • promptforge-model-client: client/transport.rs, reqwest, and url move to crates/harness/models/; client/wire.rs (serde wire shapes, no HTTP) stays as vocabulary so the engine's own suites can speak to the mock gateway without depending on a harness crate. Vocabulary (Message, Completion, CompletionResult, CompletionError, ToolSchema, ToolCall, CompletionOptions, ModelBinding, metrics) stays.
  • promptforge-lua: removes tokio and runtime_events.rs (the agent-only runtime.events() view; tasks.events replaces it); dispatch_tool in src/dispatch.rs splits into the sync prepare_dispatch (counts, trust classification, nonce wrap, ToolResult event) used at resume, and the async race, which leaves. __impl_coro.lua gains the loop, fanout, tasks, and timeout shims and loses nothing authors call.
  • crates/promptforge/{web,webfetch,web-search}/ move to crates/harness/{web,webfetch,web-search}/ as harness-web, harness-webfetch, harness-web-search; the registry and activation move to crates/harness/capabilities/.
  • crates/workshop/sessions/ is deleted; its agents/supervisor/*, agents/lifecycle.rs, agents/environment.rs, agents/session.rs, input.rs, input-tool.rs, agent discovery, and the embedded chat.md move to crates/harness/sessions/; session-log.rs (JSONL) is deleted in Step 35 and its role is taken by crates/harness/log/ (Turso) in Step 48. agents.rs, agents/socket.rs, session.rs, session-menu.rs, relay.rs, relay-tests.rs, state.rs, and the protocol frames stay in Workshop (moved into workshop-server); workshop-server depends on harness-api.
  • crates/build-xtask/src/product.rs: the harness family, its matrix row, the container door, and a manifest test that every engine crate's non-dev dependency tables exclude tokio, tokio-util, async-trait, reqwest (declared dependencies, so workspace-hack unification is irrelevant). Two further guards in the same crate: a source-identifier scan over the engine crates (comments and strings stripped) that fails when a retired symbol reappears, seeded with install_agent_chat_shim, EventsSnapshot, install_runtime_events, GatewaySource, run_models_loop, LuaFanoutResult, Observer, DebugCapture; and a check that every crates/harness/ crate carries a clippy.toml whose disallowed-methods names tokio::spawn and tokio::task::spawn_blocking, so the harness spawns only through one instrumented wrapper in harness-runner that tags the task with its EffectId and Provenance.
  • Guide: guide/src/language/04-lua-globals-and-store.md loses the sys.now sentence and documents sys.id's hierarchical form and the tasks namespace; regenerate the assembled guide. Root AGENTS.md Roles and Structure gain the harness in Step 1 (so the authoritative doc never lags the tree); crate-level AGENTS.md files are rewritten in the step that moves or renames what they describe (Steps 27, 36, 37, 38, 47). Papergate's migration (its current engine calls and their harness-api replacements) is written as a note for its own repository.
• Data, persistence, failure, security, and privacy constraints:
  • Run log (crates/harness/log/, Turso, already a workspace dependency): runs (run_id, session_id, agent, prompt_hash, seed, flags, started_at, ended_at, outcome, final_text, error_kind, error_message) and records (run_id, seq per run, task_id, task_seq, kind in effect | answer | event, effect_id, payload JSON holding an EffectRecord, EffectAnswer, or Event, at), indexed on (run_id, task_id, task_seq). Append-only; seq is the loop's order, not the clock's. Session transcript views, Workshop reconnect, and TaskEventsPerformer read records where kind = 'event'. Nothing reads answer rows back into the engine.
  • Determinism delivered: given the same RunContext and answer sequence, effects, events, Provenances, and sys.ids are identical, except where author Lua iterates a table with pairs (Lua randomizes the string hash seed per state; control of it is deferred).
  • The engine reads no clock: started_at is an input, timeouts are Timer effects. Under a future replay no timer would sleep.
  • Cancellation aborts in-flight performers in the harness; the engine only observes a flag and drops. The claims rule holds because Done is withheld until every store effect is answered or dropped and the harness awaits blocking-pool store operations before dropping them.
  • Trust: tool output and cross-chain model text are nonce-wrapped as untrusted before a model reads them, as today; task_status is trusted.

Testing Plan

The existing engine suites (about 14,000 lines under crates/promptforge-api-runtime/src/execute/tests/ and tests/suite/, mostly #[tokio::test] against an axum mock gateway) remain the acceptance tests for the Lua loop and the Lua fanout, run through the test-support tokio driver. New engine behavior is tested with the serial sans-IO driver in plain #[test]s, which need no runtime and no HTTP. Harness crates get unit tests with fake performers and an in-memory Turso database; Workshop's integration suites run unchanged against harness-api.

• Unit:
  • promptforge-lua: the structured error table round-trips (kind, tostring, typed substitution when it surfaces as a coroutine failure); every ChatResult rendering including overflow; collection_members ordering; the tasks shims' argument handling.
  • Scheduler and tasks: spawn/when_any/when_all/ready/status/note/cancel semantics; when_all reporting a failed member without raising; both timeout outcomes for each wait (timer wins: nil or timed_out, members keep running, no tasks_live at chain end; member wins: timer cancelled and its effect dropped); status for a parked and a finished task; ownership errors; survival across jump; termination at call end; H1 transfer; the tasks_live message text; abort_subtree over owned tasks; stall detection with a waiting chain; hierarchical ids identical across two runs whose fanout arms finish in different orders.
  • Model tasks: a scripted mock model emitting task, task_cancel, task_status, task_events, await_tasks; notices delivered before the next round; await_tasks returning on completion and on timeout with the still-running list; nothing to wait for; canceled for you on loop exhaustion; cancellation recorded only as an event at chain end; author adoption via tasks.pending; allowlist rejection; sibling chains stepping while one is parked in await_tasks.
  • Run with the serial driver: the doc example; a three-arm fanout with answers fed in reverse order; Done withheld while a Store effect is outstanding and delivered after Dropped; the event stream matching the former observer sequence; the determinism property (same seed, started_at, answers: identical effects, events, Provenances, sys.ids); the batching-pairing property: the same answers delivered one per step and all at once per step (and in shuffled arrival order within a batch) produce identical effects, events, and ids, which is the engine-side analogue of Temporal's incremental-versus-replay pairing; a task whose owner ends first reports abandoned, not cancelled, in both its event and the model notice.
  • harness-runner with fake performers: log record order, cancellation drops outstanding effects, store answers awaited before Done, timers answered and aborted. harness-log: round-trip against in-memory Turso; per-task slice ordering.
• Integration and end-to-end:
  • execute/tests/{tool_loop,models_loop,exec_flow,model_and_reply,local_tools,tool_scoping,exit_rules,observations}.rs through the tokio test-support driver for the Lua loop; tool_loop.rs tests that called run_prose_inference directly are rewritten at prompt level.
  • fanout.rs and execute/tests/scheduler.rs for the Lua fanout: collection order, refill on any completion, fail-fast with exactly one terminal event per arm, exhausted stub, empty collection, list-section worker, nested fanout, claims violation across arms.
  • harness-sessions inherits workshop-sessions' suites (agents/tests.rs, input-tests.rs, transition-tests.rs) relocated. Workshop server suites (crates/workshop/server/tests/it/agents/*, chat_gate/*, realtime_relay.rs) unchanged against harness-api.
• Regression, security, and performance:
  • The build-xtask manifest test fails on any forbidden dependency in an engine crate; the retired-symbol scan fails on a seeded name reintroduced in a fixture and passes on a fixture where the same name appears only in a #[cfg(test)] module, a comment, or a string (the scan covers non-test engine sources only); the harness clippy.toml check fails on a harness crate missing the disallowed-methods entries; the family matrix fixtures cover the harness row and the container door.
  • Identity: a prompt whose section calls a child section produces distinct sys.ids for parent and child entries; a fanout inside a call child produces ids nested under the child's chain id.
  • Supervisor reducer: table tests for effective_interrupt (terminal before interrupt wins; interrupt before terminal renders the synthetic frame once) and a fixture-coverage test over every interrupt variant.
  • The models_loop criterion bench (crates/promptforge-api-runtime/benches/models_loop.rs) shows no round-overhead regression from the Lua loop.
  • Trust wrapping asserted on task results and task_events reaching a model.
• Exit criteria:
  • The workspace gate list in AGENTS.md: cargo fmt --all --check; both clippy invocations with -D warnings; cargo check -p gateway --no-default-features; nextest for the workspace set and the workshop set; doctests; rustdoc with -D warnings; cargo test -p build-xtask; cargo deny check; cargo hakari verify.
  • Stop and re-plan when an existing test's expected event order or error text changes for a reason not listed under Acceptance criteria, or after two consecutive failures with the same signature on one item.

Decision Record

• Decisions:
  • Invert control: the engine returns the work it needs instead of performing it. Rationale: the scheduler was already a yield/resume state machine with I/O bolted on at the leaves; returning leaf yields as values removes the runtime dependency and makes the engine a deterministic function of its inputs. User's words: "the executor runs until it reaches a point where it wants inference or a tool call, and then it returns to the caller and then the caller provides the service."
  • models.loop in Lua over chat and tool_call yields, not a Rust state machine on the chain. Rationale: the loop's state becomes coroutine locals, the VM is live between rounds because the yields are the coroutine suspending, the fanout-with-loop serialization disappears, and the compactor framework gets a natural home. User's words: "rewrite models.loop in Lua so it can yield as a coroutine more often."
  • spawn, when_any, when_all, cancel as Lua shims over a task arena; fanout rewritten in Lua on top. Rationale: the scheduler already held an open set of chains; the task shims expose it and about 600 lines of join code become forty lines of Lua. User's words: "Can we then reimplement fanout() in lua, in terms of spawn()?"
  • when_any over a set is the only scheduler wait primitive; when_all and join are Lua over it, and when_all never raises for a member's failure. Rationale: fanout's refill and fail-fast need "first of any"; a raising when_all would force a cancel-or-leak choice on the other members that is wrong for half the callers. User's words: "having tasks.when_any tasks.when_all instead of just tasks.wait."
  • Tasks follow var: survive jump and fall-through, end with call and spawned chains, transfer from H1 to the walk, owner-only access, plain-data handles. Rationale: authors already hold the var model, and it matches the chain arena exactly.
  • Author task leak is a hard error; model task leak is a soft cancel with a notice to the model. Rationale: mirrors the existing rule that a tool's failure is the model's result record rather than the run's error; the author's mistake is a bug, the model's is recoverable. User's words: "subagents spawned by Lua become a hard error on chain termination. While subagents spawned by the model ... become soft warnings."
  • Effects and events are values returned from step; no callbacks remain. Rationale: the engine becomes pure, the harness gets one ordered stream, tests assert on a vector; delivery granularity is one step, which is Lua-fast. User's words: "that Vec<EffectId,Effect> and Vec sounds amazing!"
  • Incremental step/resume, never a batch API; Done withheld while effects are outstanding, Dropped as the host's release. Rationale: fanout parallelism at the leaves depends on resuming one arm while others' effects are in flight; the claims rule needs no join handles inside the engine.
  • Tool bindings carry ids, not implementations; capabilities move to the harness. Rationale: the harness performs tool calls, so the engine never needs the async Tool trait, and async-trait leaves the engine's types.
  • Tokio lives in the harness. execute::run and Environment::run are removed rather than preserved. Rationale: their one production consumer is moving into the harness; test-support drivers serve the engine's own suites. User's words: "so where does the tokio go" and "the engine's public API has to change ... its not a big change."
  • A harness-* product rather than a layer inside promptforge-* or workshop-*. Rationale: its own dependency discipline (tokio, Turso, the model client, capabilities; never Workshop), its own public door, and a second consumer (Papergate), the test a separate engine-* product failed. User's words: "move workshop/sessions functionality into a new top-level product called harness-*."
  • The harness owns the effect loop, not a wrapper around run. Rationale: replay, resume, and a single ordered log only exist if the harness sees every effect and answer. User's words: "yes of course the harness owns the effect loop."
  • Turso for the run log. Rationale: already a workspace dependency, and the run history is the workload that justifies it. User's words: "I am 100% certain on Turso, the value-add is enormous."
  • {{ }} stays data-only; sys holds only data. Rationale: substitution is documented as data, and admitting calls opens prose to invoking tools. User's words: "I want the narrow rule. No function calls in {{ }}."
  • No clock in the engine: sys.when is an input, timeouts are Timer effects, sys.now is removed. Rationale: nothing reads sys.now; effects are for rare explicit reads. User's words: "We should ship without now() and only when we need it."
  • Timeouts as an option on the wait shims, with the timer internal. Rationale: the shim always cancels the timer, so no leak question and no new author-visible task kind. User's words: "What about when_any_with_timeout(60, {t1,t2,t3})?"
  • The model gets task, task_cancel, task_status, task_events, await_tasks, not when_any/when_all. Rationale: Cursor's own shape (fire-and-forget with pushed results plus a bounded blocking wait like AwaitShell); a model has no idle loop; await_tasks covers the defensive check-in. User's words: "it might want to defensively put a 60s checkup timer on it."
  • History lives in the harness, present state in the engine; tasks.events is an effect answered from the log. Rationale: unbounded there, free here, reached like every other external read; the step stream is pulled so it needs no buffer or backpressure. User's words: "there should not be a limit."
  • Replay hints in the engine, replay logic in the harness: Provenance on every effect and event, hierarchical deterministic ids, spawn seeds on TaskStarted, TaskResumed reserved. Rationale: cheap now and painful to retrofit into a log schema; the engine knows a task is new or revived (a structural fact) but not where a replayed prefix ends (a harness fact). User's words: "The engine can distinguish between 'new task starting' versus 'existing task resumed'."
  • Manifest test on declared dependencies rather than cargo tree on the closure. Rationale: immune to workspace-hack feature unification; the shape crates/shared-vfs/Cargo.toml already uses.
  • Adopt the ten ranked findings of the 2026-09-18 field study of six references (lash, everruns, paigasus-helikon, Temporal sdk-core, zed, str0m), with three timing adjustments: the behavior-flag gate, at-most-once claim/settle for tool effects, and the child token-budget auto-cancel wait for replay or for harness policy; the flag field and column land now so the first replay-breaking change has somewhere to record itself. Rationale: every ranked finding maps to a named deficit and all but one were confirmed by two or more references. User's words: "should we simply adopt all the findings?" and "Yep. Do you recommendation."
  • Take the field's names where the field converges and we have no local reason otherwise: Effect, Event, Nondeterminism and Fatal for the two replay error kinds, Abandoned for a task that lost its owner, perform for what the harness does to an effect. Keep ours where the name carries a semantic the field's does not: Run::step and Run::resume (a whole batch per step, and a Lua coroutine really resumes; the sans-IO poll_output/handle_input pair implies one item per call and a drain contract we do not have), when_any/when_all (chosen from C++; no convergent alternative in the field), Task, Provenance for the replay key (the field's word would be Origin, but shared-vfs already uses Origin for claims labels and TaskOrigin names the spawning principal), Dropped (the host declined an effect, a different thing from Abandoned). Rationale: shared vocabulary helps readers who know the references, but borrowing a name without its contract misleads them.
  • Retired-symbol scan and a harness-side tokio::spawn ban as guards, not conventions. Rationale: the subject's fingerprint found 90-plus "legacy engine" anchors and dead protocol arms that a review did not catch; everruns and lash make the same class of rule a test or a lint, and the repository already has the build-xtask harness to hold them.
  • Abandoned distinct from Cancelled. Rationale: a task that lost its owner and a task someone stopped are different facts for the log, the UI, and the model notice; lash keeps them apart for the same reason.
  • A pure "terminal beats late interrupt" rule and an Alive/Closing/Closed lifecycle in the harness supervisor. Rationale: paigasus-helikon reduces the race to one function with wildcard-free matches, str0m's three-state lifecycle makes the host's question "is it closed"; both replace bookkeeping the subject's EffectExecutor does by hand.
  • Deterministic fanout member order; FANOUT_ARM_* events retired for TASK_*; spawned chains use call's target resolution. Rationale: one rule per concept; fanout is no longer a scheduler concept.
  • Interim dependency shape (added at decomposition): between the step that retires execute::run and the step that deletes workshop-sessions, tokio is an optional dependency of promptforge-api-runtime enabled only by test-support, and the engine manifest guard exempts an optional dependency whose sole enabling feature is test-support; the deletion step moves tokio to dev-dependencies and removes the exemption. Rationale: a dev-dependency is invisible to workshop-sessions, so the plan's interim cannot run on a dev-only driver; the exemption is the smallest bend and ends with the interim.
  • client/wire.rs stays in promptforge-model-client as pure vocabulary; only client/transport.rs, reqwest, and url move to harness-models (added at decomposition). Rationale: the engine's own suites drive the axum mock gateway through a dev-only tokio driver that needs the wire types and a dev-dependency on reqwest; a dev-dependency on harness-models would violate the product matrix.
  • harness-api carries a temporary bridge module of re-exports (model client, capability registry and activation, moved session pieces) during the migration, removed when workshop-sessions is deleted (added at decomposition). Rationale: workshop-* may name only harness-api, so each move can be one small commit instead of one deletion commit that moves everything.
  • The three web crates move as siblings crates/harness/{web,webfetch,web-search}/ renamed with the harness- prefix, and crates/harness/capabilities/ holds the Tool trait, registry, and activation and depends on no provider; the providers depend on it for the trait, and harness-sessions is the one crate that depends on both sides and registers the first-party capabilities (added at decomposition; direction fixed at the AGENTS.md review because the reverse is a cycle: webfetch/src/tool.rs and web-search/src/web_search.rs implement Tool). Rationale: families are keyed by name prefix and the matrix has no nested containers; folding three crates into one would erase harness-webfetch's own test target for invariant A3.
  • tasks.events, task_events, and the TaskEvents effect land with the Run API, not with the model-tasks component (added at decomposition). Rationale: before Run exists the engine has no history source to answer them; the test drivers answer from their event buffer.
  • AGENTS.md review (2026-09-18, after decomposition): every AGENTS.md in the repository (root plus 27 crate files) was checked against every step. Decisions, each recorded where it applies in the text above: Task handles are methodless (lua AGENTS.md and archdoc A9 forbid colon methods on host handles; user chose to drop the methods rather than take an exception); the retired-symbol scan and the harness clippy.toml check received the explicit approval the root Engineering rule requires; the three-file scheduler split is a scheduler/ directory and protocol.rs is split before it is edited (flat-directory and 500-line rules); parentless kebab filenames became plain modules and loop.rs was renamed because loop is a keyword; harness-capabilities depends on no provider and the providers depend on it (the plan's original direction was a cycle); code moved out of workshop-sessions sheds workshop_registry and workshop-server registers the Harness handle at boot (matrix rule and workshop-server AGENTS.md); root AGENTS.md Roles and Structure are updated in Step 1 and every crate AGENTS.md in the step that changes what it describes, because AGENTS.md is authoritative and must not lag the tree; harness-* crates carry the ## Invariants marker and the ceiling check extends to them; EventLog, events.rs, runtime_events.rs, and the JSONL session log are deleted rather than adapted (user accepted that Workshop writes no JSONL between Steps 35 and 48); the runtime AGENTS.md store-scope rule is reworded around minted Access handles; the replay key is Provenance, not Origin, because shared_vfs::observe::Origin and TaskOrigin already hold that word; a chain may call status, events, and note on its own task.
  • Execution review (2026-09-18, after the AGENTS.md review): a full read of the step text for data flow and ambiguity fixed eight things: Provenance has a concrete home (a field on every Event, the middle element of the Step::Pending effect tuple); test_support::Performers is a struct of boxed async closures, so workshop-sessions implements no test-support trait in the interim; the Step 28 events-to-observer adapter moves into test_support at Step 35 when its last production consumer dies; Step 8 records the models_loop bench baseline that Step 14 compares against; the harness spawn-ban check also covers the door crate harness-api; drive_run's sink is typed as FnMut(Event) and deltas travel on a separate DeltaSink; every file in workshop-sessions is named in either the move-to-harness list or the stay-in-Workshop list, and only the two moved files shed workshop_registry; the async InputBroker trait is deleted at Step 47 once InputPerformer replaces its only implementor.
  • Run owns its prompt (Arc<Prompt>) rather than borrowing it. Rationale: the harness holds a Run across awaits for the run's whole life and stores it beside the prompt it came from; a borrowed prompt would make that pair self-referential. Added at review.
  • How the execution steps are cut: dependency order first; between independent steps the less risky one first (risk being files touched, public interface or persisted shape changed, existing test expectations changed); safe additive work front-loaded as far as dependencies allow; fine-grained steps with one narrow test each and light per-step testing; the rich suites concentrated in checkpoint steps that add no product code, placed at least at the Lua loop complete, tasks and fanout complete, model tasks complete, the Run API complete with the engine dependency-free, the harness runner and log complete, and workshop-sessions deleted. Rationale: small commits are easy to review and revert, and a regression surfaces at a known checkpoint rather than anywhere. User's words: "ordered in dependency order, and for tie breaker from least risky to most risky. Front load the safe stuff as much as possible. Use fine grained steps but if you do that then go VERY light on the testing. Bake well-defined more rich test checkpoints into the plan."
  • This plan supersedes the earlier "Harness API crate" plan (workspace plan file harness_api_crate_7eaf6056), which created harness-api on the callback Observer design with a per-run Turso record ordered by causal position, section name, and chain id, plus a workshop-runs crate and a Run-button table. What carries over from it: PaperGate lives at wg21-paperflow/crates/papergate and path-depends on promptforge-core, a crate that no longer exists, so its migration note (Step 50) starts from a broken dependency, not a working one; turso is pinned =0.7.2 and workshop-workspace already has the open_database, SCHEMA_V1 with PRAGMA user_version, one-actor-per-file pattern the run log copies; WorkshopObserver in crates/workshop/gateway/src/observer.rs is an in-tree Observer implementor that Step 35 converts; and the Run window's requirement that events appear in one deterministic order never decided by wall clock is met by the log's loop-assigned seq and per-task Provenance, so a client can order by (seq) for arrival or by (task_id, task_seq) for per-task causality without a clock. Not carried over: the observer-based recorder, the Coordinates { chain_id } change to Observer::observe, and the workshop-runs crate and Run-button UI, which are a later client of harness-api outside this plan.
• Rejected alternatives:
  • A Rust state machine for models.loop stored on the chain. Reason: an explicit phase enum re-entering every exit rule across two resume points per round, and the compactor framework stays hard. Revisit: never, unless Lua instruction cost per round proves measurable.
  • A single blocking join instead of when_any(set). Reason: fanout's refill and fail-fast react to any arm ending. Revisit: none.
  • A when_all that raises on the first member failure. Reason: forces cancel-or-leak on the remaining members. Revisit: none.
  • step(now) threading a timestamp through every step. Reason: feeds a sys.now field nobody reads; effects are for rare explicit reads. Revisit: none; now() as an effect is the deferred design.
  • now() as an effect, or sys.now() as a function, shipped now. Reason: no consumer in the tree. Revisit: the first prompt that needs a live clock (the known candidate is the Mentographist, an interviewer prompt kept outside this repository in the workspace's tools-public/agents/mentograph.md, which stamps each transcript turn with the time it was asked).
  • An author-visible completes_after(seconds) timer task. Reason: an internal timer on the wait shims covers timeouts with no leak question. Revisit: a prompt needing a bare sleep or a timer composed with something that is not a task.
  • Zero-argument function calls inside {{ }}. Reason: opens prose to tool invocation; the user chose data-only. Revisit: none.
  • when_any/when_all as model tools. Reason: a model has no idle loop; results arrive as messages; await_tasks covers the blocking case. Revisit: none.
  • A bounded ring buffer of events inside the engine for tasks.events. Reason: a bound on retained history is a memory policy with no correct value, and the harness log already holds all of it. Revisit: none.
  • Scheduler-level pause/resume of a parked chain (freeze, later re-dispatch its pending request). Reason: re-issues the stuck operation and re-runs non-idempotent tool calls; resume-by-re-execution with a substituted answer is the correct mechanism. Revisit: none.
  • The engine consuming replay history (resume_task(id, history)). Reason: puts the log's shape and a replay mode inside the pure core; the harness can match exactly because the engine is deterministic. Revisit: none.
  • A separate engine-* product for the sans-IO core. Reason: an engine- prefix classifies as no family in product.rs (fewer rules, not more); the core has no consumer independent of PromptForge. Revisit: a second consumer wanting the engine without the product (another workspace product, a WASM build, a separate release).
  • Keeping a tokio driver and execute::run inside promptforge-api-runtime behind a feature. Reason: superseded once the harness exists as the only production host. Revisit: none.
  • A cargo tree CI gate on the feature-off closure. Reason: collides with workspace-hack unification. Revisit: none; the manifest test replaces it.
  • Doing nothing. Reason: leaves the fanout-with-loop serialization, the tokio coupling, and the join machinery; forecloses replay and the compactor work. Revisit: none.
• Assumptions, risks, and notes:
  • Lua randomizes its string hash seed per state, so author code iterating with pairs is not reproducible across runs. The plan's determinism claims exclude that case; bit-exact replay needs the deferred seed control.
  • The sys.id encoding is chosen during implementation after surveying readers; the guide (guide/promptforge-language-guide.md) documents it as an id, and no prompt in the tree assumes consecutive integers.
  • The engine tests total about 14,000 lines; they migrate gradually behind the test-support drivers, not in one change.
  • Between the engine change landing and the harness sessions crate landing, workshop-sessions runs on the test-support tokio driver, which means a production crate enables the runtime's test-support feature for that interval. This bends the repository's rule that features gate constraints rather than product shape; it is accepted as a temporary state, called out in the commit that introduces it, and removed by the commit that deletes workshop-sessions. The interim state must pass the Workshop suites.
  • Local Lua tool handlers run inside step synchronously; they are sandboxed Lua whose only effect is on VM state, so they are compatible with re-execution.
  • Turso's footprint (59 exclusive crates per vibe/dependency-surface.md) is accepted; the run log is its justifying workload.
  • promptforge-tool-picker is being removed by a separate plan (vibe/2026-09-18-2-remove-tool-picker.md); this plan assumes it is gone or ignores it. Found at Step 8 (2026-09-18): that removal also deleted the models_loop criterion bench that Steps 8, 14, and 40 and the Testing Plan's exit criterion rely on. Decision: Step 8 restores the bench without its picker usage (a dev-dependency on criterion only) rather than dropping the regression gate. Falsifier: the restored bench cannot be adapted to today's Environment::run API without measuring something other than round overhead.
  • models_loop bench baseline (pre-Step-8): recorded 2026-09-18 on the restored bench (cargo bench -p promptforge-api-runtime --bench models_loop, criterion 0.5, 100 samples, two consecutive runs) before either file split, on x86_64-pc-windows-msvc. models_loop: run 1 [1.9309 ms 1.9529 ms 1.9786 ms], run 2 [2.0687 ms 2.0896 ms 2.1136 ms]. compactors_fail: run 1 [770.94 µs 773.35 µs 775.57 µs], run 2 [756.91 µs 761.78 µs 766.26 µs]. Run-to-run spread on models_loop is about 7%, so Step 14 should treat anything inside 10% as noise and compare against the mean of these two runs (about 2.02 ms and 768 µs).
  • Papergate's code change lands in its own repository; this plan produces only the migration note.
  • The field study these additions come from ("What to steal for PromptForge: sans-IO engine, harness, effect and event streams, run log, replay, subtasks", 2026-09-18, six references at pinned commits) found the subject already matches or beats the references on effects-as-data at the script boundary, the single-owner scheduler, structural cancellation, the pure supervisor reducer, and test-enforced tiers; those are preserved, not redesigned.
  • Gateway supervision stays in Workshop (crates/workshop/gateway/); the harness receives the binding as data through Harness::set_gateway and never depends on workshop-gateway. Papergate supplies its own binding the same way.

Deferred and Out of Scope

• Deferred, replay: a run is bit-exact reproducible from its log. Contract: construct RunContext with the logged seed and started_at, feed answer rows through resume in seq order, expect identical effects and events including Provenance and sys.id; no timer sleeps. Prerequisites: Lua string-hash seed control (a build-level define of luai_makeseed calling a function the engine sets before creating the state), a replay driver in harness-runner, a divergence report. Revisit: when resume or audit needs it.
• Deferred, resume a cancelled task by re-execution: spawn the task again from its TaskStarted seeds under its original id (a revive variant of Spawn emitting TaskResumed), feed its own recorded answers until exhausted, then go live; the resumer may supply the answer for the effect the task was cancelled inside (task_resume { id, answer? }, tasks.resume(t, answer?)). The harness owns history, matching, divergence, and the substituted answer; the engine owns only the revive request. Revisit: with replay.
• Deferred, now(): a bare global yield shim producing a Now effect answered with a Timestamp. Revisit: first consumer.
• Deferred, author-visible completes_after: one exposed line plus a tasks_live exemption for effect-backed tasks. Revisit: first consumer.
• Deferred, a task mailbox (tasks.send, inbox()) for redirecting a running task without cancelling it. Revisit: a case resume-by-re-execution does not cover.
• Deferred, the behavior-flag gate (try_use_flag(flag, should_record)): run new behavior live and record the flag; on replay honor it only if the original run recorded it. The Flags field and column land in this plan; the gate has no job until replay exists. Revisit: the first engine change that would alter a recorded run's effects or events.
• Deferred, at-most-once claim/settle for tool effects across a restart (Claimed | AlreadySettled | AlreadyRunning | DeterminismViolation). The harness performs each effect exactly once within a run today. Revisit: with resume-by-re-execution, where a re-issued tool effect must not run twice.
• Deferred, child budget auto-cancel: the harness cancels a spawned task whose token usage or context crosses a threshold, a policy computed from the log. Revisit: when a prompt's subtasks are observed to run away.
• Deferred, the field-study idioms that map to no listed deficit or do not apply: Temporal's per-effect fsm! state-machine macro (we have one scheduler, not dozens of machines), its non-SDK wake detection (the engine holds no futures), its lookahead pre-resolution and transition coverage; zed's two-projection tool output; str0m's borrowed &mut handles and per-subsystem output queues. Revisit: none scheduled.
• Deferred, the compactor framework (replacement-returning callbacks, in-place history rewrite). Revisit: after the Lua loop lands.
• Deferred, publishing the events-to-observer adapter (which this plan ships under the runtime's test-support feature for its own suites) as a supported API for out-of-tree consumers. Revisit: an out-of-tree consumer asks.
• Out of scope: parallel Lua within one run; gateway product changes; Papergate's own code changes; the workshop-runs crate and the Run-button flat event table from the superseded "Harness API crate" plan (a later harness-api client).