fix(examples): end every offline example on a terminal frame - #957
Merged
Conversation
Sandbox 0.31 settles a run's outcome from `done` alone: `result` carries the payload and is explicitly not terminal (runtime-api, observeEvent). Four offline example fixtures emit `result` and stop, so every iteration settles as "Agent stream ended without a terminal event" and the loops report the wrong decision. Measured before: quickstart/quickstart and driver-loop both end `decision: fail` with every shot rejected, and researcher-loop reports no winner. Each README documents `pick-winner`. Nothing caught it because examples are typechecked, not run. researcher-loop additionally imported researcherProfile, ResearchTask and ResearchOutput from `@tangle-network/agent-knowledge/profiles`, a subpath that package does not publish; the symbols live in this repo under `@tangle-network/agent-runtime/profiles`. The preset also now takes a caller-owned AgentProfile, which the example supplies. tsconfig.examples.json excluded the directory, which is why the broken imports typechecked green; the exclusion is gone.
tangletools
approved these changes
Aug 21, 2026
tangletools
left a comment
Contributor
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — db08622e
This PR was opened by the trusted drewstone account.
This approval is provisional and was applied by the local stand-in because the pr-reviewer webhook host is unreachable (2026-08-21). CI on this head is fully green. The full PR reviewer audit re-runs via the resweep when the service returns and will publish findings if it detects issues.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every offline example that scripts a worker has been settling as a failed run. Sandbox 0.31 decides a run's outcome from
donealone —resultcarries the payload and is explicitly not terminal:Four fixtures emit
resultand stop. The output still parses, so the example prints numbers and looks alive — and every iteration carriessandboxOutcome: { success: false, error: 'Agent stream ended without a terminal event' }, so no verdict is produced and the loop reports the wrong decision.Measured on
origin/mainbefore this change, against what each README documents:quickstart/quickstartdecision: pick-winner — winner: shot 1decision: fail — winner: shot undefined, all three shots rejecteddriver-loopdecision: pick-winner/winner: shot 1decision: fail, all three shots rejectedresearcher-loopexample-tenantitemsno winner — every iteration failed validationquickstart/minimaldecision: done — 1 iteration(s)Nothing caught it: examples are typechecked, never run.
researcher-loopwas broken a second way, and hidden a second way. It importedresearcherProfile,ResearchTaskandResearchOutputfrom@tangle-network/agent-knowledge/profiles— a subpath that package publishes in neither 8.0.10 nor 10.7.0 (itsexportsare.,./viz,./cli,./memory,./sources,./benchmarks). Those symbols live in this repo, atsrc/profiles/researcher.ts, published as@tangle-network/agent-runtime/profilesand documented atdocs/api/profiles.md:1192. The example typechecked green only becausetsconfig.examples.jsonexcluded the directory.Change
{ type: 'done', data: { outcome: { type: 'completed' } } }— the frame the repo's own kernel tests use, and the oneexamples/agentic-data-creationalready had.researcher-loopimports the preset from@tangle-network/agent-runtime/profiles, and supplies the caller-ownedAgentProfilethatresearcherProfile({ profile, task })requires.researcherProfileis not reintroduced upstream; the dependency stays pointing down.tsconfig.examples.jsonno longer excludesexamples/researcher-loop, sopnpm run typecheckcovers it like every other example.pnpm add -D @tangle-network/agent-knowledge, andexamples/README.md's row no longer describes it as "uses the optionalagent-knowledgepeer".Proof
Every offline example, run after the change:
Each now matches its own README line for line, including
researcher-loop's documentedproposedWrites: 1 — insert into example-tenant.Simplification
Simplification: one exclusion deleted from
tsconfig.examples.json, so the example set has one rule instead of one rule and an exception; the fixtures now all end the same way.Net: +12 / -12 lines across 8 files; 1 typecheck exception removed, 1 dead package dependency removed from an example and its docs.
Not done here: examples are still not executed in CI, which is why four of them could report the wrong decision for as long as they have. Running them is a build-time cost decision and a separate change — I am not adding a gate on the strength of one incident. What this pull request does is make the failure visible the next time someone runs one.
Tests: +0, -0 deleted. The proof is the four runs above; a unit test asserting a fixture emits
donewould restate the fixture, and running the examples is the CI question named above.Refs #954