Skip to content

fix(examples): end every offline example on a terminal frame - #957

Merged
drewstone merged 1 commit into
mainfrom
fix/researcher-loop-example
Aug 21, 2026
Merged

fix(examples): end every offline example on a terminal frame#957
drewstone merged 1 commit into
mainfrom
fix/researcher-loop-example

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Problem

Every offline example that scripts a worker has been settling as a failed run. Sandbox 0.31 decides a run's outcome from done alone — result carries the payload and is explicitly not terminal:

// @tangle-network/sandbox 0.31.0, runtime-api, observeEvent
if (type === "result") observeOutcomeFrame(state, type, data, false);   // terminal: false
if (type === "done")   observeOutcomeFrame(state, type, data, true);    // terminal: true
...
if (!state.terminalReached) return failed("Agent stream ended without a terminal event", ...)

Four fixtures emit result and stop. The output still parses, so the example prints numbers and looks alive — and every iteration carries sandboxOutcome: { success: false, error: 'Agent stream ended without a terminal event' }, so no verdict is produced and the loop reports the wrong decision.

Measured on origin/main before this change, against what each README documents:

example README says actually printed
quickstart/quickstart decision: pick-winner — winner: shot 1 decision: fail — winner: shot undefined, all three shots rejected
driver-loop decision: pick-winner / winner: shot 1 decision: fail, all three shots rejected
researcher-loop the clean candidate wins, two example-tenant items no winner — every iteration failed validation
quickstart/minimal decision: done — 1 iteration(s) same line, on a failed outcome

Nothing caught it: examples are typechecked, never run.

researcher-loop was broken a second way, and hidden a second way. It imported researcherProfile, ResearchTask and ResearchOutput from @tangle-network/agent-knowledge/profiles — a subpath that package publishes in neither 8.0.10 nor 10.7.0 (its exports are ., ./viz, ./cli, ./memory, ./sources, ./benchmarks). Those symbols live in this repo, at src/profiles/researcher.ts, published as @tangle-network/agent-runtime/profiles and documented at docs/api/profiles.md:1192. The example typechecked green only because tsconfig.examples.json excluded the directory.

Change

  • The four fixtures append { type: 'done', data: { outcome: { type: 'completed' } } } — the frame the repo's own kernel tests use, and the one examples/agentic-data-creation already had.
  • researcher-loop imports the preset from @tangle-network/agent-runtime/profiles, and supplies the caller-owned AgentProfile that researcherProfile({ profile, task }) requires. researcherProfile is not reintroduced upstream; the dependency stays pointing down.
  • tsconfig.examples.json no longer excludes examples/researcher-loop, so pnpm run typecheck covers it like every other example.
  • The two documentation claims that pointed at the wrong package are corrected: the example's README no longer tells a reader to pnpm add -D @tangle-network/agent-knowledge, and examples/README.md's row no longer describes it as "uses the optional agent-knowledge peer".

Proof

Every offline example, run after the change:

quickstart/minimal      decision: done — 1 iteration(s)
quickstart/quickstart   shot 0: reject … shot 1: PASS … decision: pick-winner — winner: shot 1
driver-loop             SHOT 0 [reject] → SHOT 1 [PASS] … decision: pick-winner, winner: shot 1
researcher-loop         decision: pick-winner, winner: iteration #0, score 0.700, valid true,
                        items (2) both [example-tenant], citations 2, proposedWrites 1
chat-handler            unchanged (it streams finalText through a different path)

Each now matches its own README line for line, including researcher-loop's documented proposedWrites: 1 — insert into example-tenant.

pnpm run typecheck            clean, and now covers examples/researcher-loop
pnpm run lint                 615 files, no fixes
pnpm run build                clean
pnpm run check:api-surface    2120 exports / 17 entry points, record current
pnpm run check:version-bump   consumer surface unchanged at 0.155.0 (examples are not published)
pnpm run docs:check           exit 0
pnpm test                     2803 passed / 173 failed across 22 files
  clean origin/main, same machine: 2797 passed / 171 failed across 21 files. The
  baseline's 21 files are a subset of these; the extra is tests/kernel/workspace.test.ts,
  a 20 s git-worktree timeout in the known macOS class. CI on Linux is the authority.

Simplification

Simplification: one exclusion deleted from tsconfig.examples.json, so the example set has one rule instead of one rule and an exception; the fixtures now all end the same way.
Net: +12 / -12 lines across 8 files; 1 typecheck exception removed, 1 dead package dependency removed from an example and its docs.
Not done here: examples are still not executed in CI, which is why four of them could report the wrong decision for as long as they have. Running them is a build-time cost decision and a separate change — I am not adding a gate on the strength of one incident. What this pull request does is make the failure visible the next time someone runs one.

Tests: +0, -0 deleted. The proof is the four runs above; a unit test asserting a fixture emits done would restate the fixture, and running the examples is the CI question named above.

Refs #954

Sandbox 0.31 settles a run's outcome from `done` alone: `result` carries the
payload and is explicitly not terminal (runtime-api, observeEvent). Four offline
example fixtures emit `result` and stop, so every iteration settles as "Agent
stream ended without a terminal event" and the loops report the wrong decision.

Measured before: quickstart/quickstart and driver-loop both end `decision: fail`
with every shot rejected, and researcher-loop reports no winner. Each README
documents `pick-winner`. Nothing caught it because examples are typechecked, not
run.

researcher-loop additionally imported researcherProfile, ResearchTask and
ResearchOutput from `@tangle-network/agent-knowledge/profiles`, a subpath that
package does not publish; the symbols live in this repo under
`@tangle-network/agent-runtime/profiles`. The preset also now takes a
caller-owned AgentProfile, which the example supplies. tsconfig.examples.json
excluded the directory, which is why the broken imports typechecked green;
the exclusion is gone.

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — db08622e

This PR was opened by the trusted drewstone account.

This approval is provisional and was applied by the local stand-in because the pr-reviewer webhook host is unreachable (2026-08-21). CI on this head is fully green. The full PR reviewer audit re-runs via the resweep when the service returns and will publish findings if it detects issues.

@drewstone
drewstone merged commit c175f2f into main Aug 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants