Skip to content

feat(brief): build a retrieval brief a receipt can prove - #166

Merged
drewstone merged 1 commit into
mainfrom
feat/knowledge-brief
Aug 21, 2026
Merged

feat(brief): build a retrieval brief a receipt can prove#166
drewstone merged 1 commit into
mainfrom
feat/knowledge-brief

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Why

discovery-lab measured, on 2026-08-19: re-derivation stayed flat at 5.6–11.2% before and after citations shipped, and citing pages re-derived MORE than non-citing ones — 9.6% vs 5.6% (#157, capability 4).

"Search the store first" was a prompt instruction, and an instruction is not infrastructure. The economics of an amnesiac agent favour re-deriving a fact over fishing for it in a 2,000-page pile, so the store loses to the model every time the choice is left to the model. The fix is to hand the run the knowledge before its first token, with the ids ready to cite, and to record what was handed over.

What

buildKnowledgeBrief(visiblePages, question, { limit, excludeInvalidated, tags, kinds, maxChars }) returns { question, retrieverId, retrieverConfigDigest, hits, citationIds, results, text }.

  • text is deterministic Markdown: one - [id] title — snippet line per page, in rank order. Nothing carries a timestamp, so the same page set produces the same brief on every run.
  • results is exactly the shape createKnowledgeRetrievalReceipt takes, so the caller mints a receipt with no adapter in between. That is what turns "retrieved" from a claim into a record.
  • retrieverId and retrieverConfigDigest complete the receipt's retriever identity. The package cannot know which published version is running it once bundled, so the caller declares only version.
  • excludeInvalidated defaults to true here, the opposite of searchKnowledge. A brief offers every page it names with an id ready to cite; a refuted page in it invites a run to build on a dead claim. searchKnowledge keeps false, because a caller reading history needs those pages.
  • maxChars bounds the brief, and a page whose line does not fit is left out of text, hits, citationIds, and results alike. A receipt that recorded 7 results while the agent was shown 6 would be worse than no receipt.

Pure, Worker-safe, zero new dependencies. It composes the existing ranking, the existing receipt constructor, and the existing origin labels; it adds no second retriever and no second receipt.

Simplification

Simplification: searchKnowledge(index, query, options) is now searchKnowledgePages(index.pages, ...). One ranking implementation with two entry shapes, the index form delegating, so a chain read no longer has to fabricate a KnowledgeIndex to be searched and the two paths cannot drift. The lexical-index guard moved to the same page-set terms.
Net: +328 / -14 lines, 9 files, 1 ranking implementation kept (a second, page-set copy avoided), 0 new retrievers or receipt shapes.
Not done here: the brief does not mint the receipt itself, because minting needs a clock, a run id, and an actor, none of which belong to a pure ranking function. The tool adapter is where those meet.
Tests: +3 (a refuted page stays out of a brief unless the caller asks for it, which is the default that protects a run from citing a dead claim; the rendered text, ids, and results describe one identical set under a maxChars bound, which is the honesty property of the receipt; the results round-trip into a retrieval receipt that verifies and joins the visibility snapshot, with the config digest matching — the cross-boundary contract that makes retrieval provable), -0 deleted. No shape assertions.

Proof

pnpm run typecheck   pass (src + contracts)
pnpm run lint        pass, 237 files
pnpm run build       pass
pnpm api:surface     967 exports across 6 entry points, regenerated and committed
check:version-bump   5 export changes needing a minor bump, paid for by 10.3.0 -> 10.4.0
vitest src/knowledge-brief.test.ts src/search.test.ts   12 passed
pnpm test (full)     22 failed files | 58 passed | 3 skipped (83)

The failing files and their count are unchanged from clean origin/main on this macOS host, and every failure raises knowledge transaction directory escaped its root — the /var -> /private/var symlink under tmpdir(). tests/filesystem-search-provider.test.ts, the one search-path file in that set, fails with that same error and fails identically on main. Linux CI has no such symlink.

Refs #157

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 759d2c95

This PR was opened by the trusted drewstone account.

This approval is provisional and was applied by the local stand-in because the pr-reviewer webhook host is unreachable (2026-08-21). CI on this head is fully green. The full PR reviewer audit re-runs via the resweep when the service returns and will publish findings if it detects issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants