Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Otherwise, it stays in this package.
- Generated pages live under `knowledge/` unless the caller names another root-relative directory with `pagesDirectory` (CLI `--pages-dir`).
- Raw evidence lives under `raw/sources/` and should not be edited.
- Pass `intake` to `applyKnowledgeWriteBlocks` (CLI `--intake`) so a write that duplicates a visible page or cites a page id that exists nowhere is refused before any byte lands.
- Build the retrieval brief with `buildKnowledgeBrief` before a run starts, and mint a receipt from `brief.results`, so retrieval is recorded rather than instructed.
- Run `agent-knowledge index` after page changes.
- Run `planInvalidationPropagation` + `formatKnowledgeInvalidationProposal` after grading, so every citer of a refuted page records `citesInvalidated`.
- Run `agent-knowledge lint` before trusting or promoting knowledge.
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## 10.4.0 — 2026-08-21

### Added

- Add `buildKnowledgeBrief(visiblePages, question, options)`. It ranks the knowledge one question can see, renders a deterministic `- [id] title — snippet` line per page, and returns `results` in exactly the shape `createKnowledgeRetrievalReceipt` takes, so a retrieval is recorded rather than claimed. It is pure: no clock, no filesystem, no network. `excludeInvalidated` defaults to `true`, the opposite of `searchKnowledge`, because a brief offers every page it names with an id ready to cite.
- The brief also returns `retrieverId` and `retrieverConfigDigest`, the retriever identity a receipt needs, so a caller declares only the running package version.
- Add `searchKnowledgePages(pages, query, options)`, the ranking over a page set that is not a built index, such as the chain a run can see. `searchKnowledge(index, ...)` is now this function over `index.pages`, so the two entry points cannot drift.

## 10.3.0 — 2026-08-21

### Added
Expand Down
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,26 @@ support-kb/
index.json # generated search index
```

## Brief a run before its first token

"Search the store first" is an instruction an agent may or may not follow. A brief is infrastructure: it retrieves the settled knowledge a question can reach and hands it over with the ids a later write must cite.

```ts
const brief = buildKnowledgeBrief(originatedPages(await loadKnowledgePages(root)), question)
const receipt = createKnowledgeRetrievalReceipt({
runId,
query: brief.question,
retriever: { id: brief.retrieverId, version, configDigest: brief.retrieverConfigDigest },
visibility: createKnowledgeVisibilitySnapshot(visiblePages),
results: brief.results,
})
```

`brief.text` is deterministic Markdown, one `- [id] title — snippet` line per page in rank order.
`brief.results` is the exact shape `createKnowledgeRetrievalReceipt` takes, so what an actor was given is recorded rather than asserted.
`excludeInvalidated` defaults to **true** here, the opposite of `searchKnowledge`: a brief offers every page it names with an id ready to cite, so a refuted page in it invites a run to build on a dead claim.
`maxChars` bounds the brief, and a page whose line does not fit is left out of `text`, `hits`, `citationIds`, and `results` alike, so all four always describe one identical set.

## Propagate an invalidation

A page whose own evidence refuted it carries an `invalidation`. A reader who arrives through a citation never meets that verdict, so run the propagation pass after grading:
Expand Down
5 changes: 5 additions & 0 deletions api-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@
"CreateKnowledgeRetrievalReceiptInput": "value",
"CreateKnowledgeUseReceiptInput": "value",
"D1Adapter": "value",
"DEFAULT_KNOWLEDGE_BRIEF_LIMIT": "value",
"DEFAULT_MEMORY_CLEANUP_TIMEOUT_MS": "value",
"DEFAULT_PAGES_DIRECTORY": "value",
"DedupReason": "value",
Expand Down Expand Up @@ -171,6 +172,8 @@
"KnowledgeBenchmarkSpec": "type",
"KnowledgeBenchmarkSplit": "type",
"KnowledgeBenchmarkTaskKind": "type",
"KnowledgeBrief": "value",
"KnowledgeBriefOptions": "value",
"KnowledgeChange": "value",
"KnowledgeChangeKind": "value",
"KnowledgeChunk": "value",
Expand Down Expand Up @@ -524,6 +527,7 @@
"buildIndustryMemoryBenchmarkSmokeCases": "value",
"buildIndustryRagBenchmarkSmokeCases": "value",
"buildKnowledgeBenchmarkScenarios": "value",
"buildKnowledgeBrief": "value",
"buildKnowledgeGraph": "value",
"buildKnowledgeIndex": "value",
"buildKnowledgeLexicalIndex": "value",
Expand Down Expand Up @@ -712,6 +716,7 @@
"scoreRagAnswerArtifact": "value",
"scoreRetrievalArtifact": "value",
"searchKnowledge": "value",
"searchKnowledgePages": "value",
"sha256": "value",
"sleepForMemoryRecovery": "value",
"slugify": "value",
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@tangle-network/agent-knowledge",
"version": "10.3.0",
"version": "10.4.0",
"description": "Build, search, evaluate, and improve source-backed knowledge bases.",
"homepage": "https://github.com/tangle-network/agent-knowledge#readme",
"repository": {
Expand Down
1 change: 1 addition & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ export * from './investment-thesis-set'
export * from './investment-thesis-task'
export * from './kb-improvement'
export * from './kb-store'
export * from './knowledge-brief'
export * from './knowledge-use-receipts'
export * from './lexical-index'
export * from './lint'
Expand Down
110 changes: 110 additions & 0 deletions src/knowledge-brief.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
import { canonicalCandidateDigest } from '@tangle-network/agent-interface'
import { describe, expect, it } from 'vitest'
import { buildKnowledgeBrief } from './knowledge-brief'
import {
assertKnowledgeRetrievalMatchesVisibility,
createKnowledgeRetrievalReceipt,
createKnowledgeVisibilitySnapshot,
verifyKnowledgeRetrievalReceipt,
} from './knowledge-use-receipts'
import { originatedPages } from './run-scoped'
import type { KnowledgePage } from './types'

function page(id: string, text: string, extra: Partial<KnowledgePage> = {}): KnowledgePage {
return {
id,
path: `knowledge/${id}.md`,
title: id.replace(/-/g, ' '),
text,
frontmatter: { id },
sourceIds: [],
tags: [],
outLinks: [],
...extra,
}
}

const live = page(
'retry-budget',
'A retry budget caps the retries a run may spend and refuses the run when the budget is exhausted.',
)
const refuted = page(
'retry-forever',
'Unbounded retry recovers a run more often than a retry budget does, measured over a hundred runs.',
{
invalidation: {
verdict: 'contradicted',
observedAt: '2026-08-18T00:00:00.000Z',
reason: 'The replication measured the opposite direction.',
},
},
)

describe('buildKnowledgeBrief', () => {
it('keeps a refuted page out of the brief unless the caller asks for it', () => {
const visible = originatedPages([live, refuted])

expect(buildKnowledgeBrief(visible, 'retry budget').citationIds).toEqual(['retry-budget'])
expect(
[
...buildKnowledgeBrief(visible, 'retry budget', { excludeInvalidated: false }).citationIds,
].sort(),
).toEqual(['retry-budget', 'retry-forever'])
})

it('renders one line per page and keeps text, ids, and results describing one set', () => {
const brief = buildKnowledgeBrief(originatedPages([live, refuted]), 'retry budget')

expect(brief.text).toBe(
`- [retry-budget] retry budget — ${brief.hits[0]!.snippet.replace(/\s+/g, ' ').trim()}`,
)
expect(brief.results.map((result) => result.origin)).toEqual(['here'])

const bounded = buildKnowledgeBrief(originatedPages([live, refuted]), 'retry budget', {
excludeInvalidated: false,
maxChars: 1,
})
expect(bounded.text).toBe('')
expect(bounded.citationIds).toEqual([])
expect(bounded.results).toEqual([])
})

it('produces results a retrieval receipt accepts and a verifier joins to the snapshot', () => {
const visible = [
...originatedPages([live]),
...originatedPages(
[page('retry-storm', 'A retry storm is what a missing retry budget produces.')],
'shared',
),
]
const brief = buildKnowledgeBrief(visible, 'retry budget', { limit: 2 })
const snapshot = createKnowledgeVisibilitySnapshot(visible)

const receipt = createKnowledgeRetrievalReceipt({
runId: 'run-a',
query: brief.question,
retriever: {
id: brief.retrieverId,
version: '10.4.0',
configDigest: brief.retrieverConfigDigest,
},
visibility: snapshot,
results: brief.results,
createdAt: '2026-08-21T00:00:00.000Z',
})

assertKnowledgeRetrievalMatchesVisibility(receipt, snapshot)
expect(verifyKnowledgeRetrievalReceipt(receipt).results.map((result) => result.pageId)).toEqual(
brief.citationIds,
)
expect(receipt.retriever.configDigest).toBe(
canonicalCandidateDigest({
limit: 2,
excludeInvalidated: true,
tags: null,
kinds: null,
maxChars: null,
}),
)
})
})
151 changes: 151 additions & 0 deletions src/knowledge-brief.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
/**
* Retrieval briefing.
*
* A brief turns "retrieved" from something a run claims into something the
* infrastructure can prove. It ranks the knowledge visible to a question,
* renders it with the ids a later write must cite, and returns the ranked
* results in the shape a retrieval receipt takes, so what an actor was given
* is recorded rather than asserted.
*
* Pure: no clock, no filesystem, no network.
*/
import { canonicalCandidateDigest, type Sha256Digest } from '@tangle-network/agent-interface'
import type { OriginatedKnowledgeSearchResult } from './knowledge-use-receipts'
import type { OriginatedPage, PageOrigin } from './run-scoped'
import {
KNOWLEDGE_SEARCH_RETRIEVER_ID,
type KnowledgeSearchHit,
searchKnowledgePages,
} from './search'
import type { KnowledgeId } from './types'

/** Pages in a brief when the caller names no limit. */
export const DEFAULT_KNOWLEDGE_BRIEF_LIMIT = 5

export interface KnowledgeBriefOptions {
/** Maximum pages in the brief. Defaults to 5. */
limit?: number
/**
* Drop pages whose own evidence refuted them. Defaults to true, the opposite
* of `searchKnowledge`: a brief is injected before a run's first token and
* offers every page it names with an id ready to cite, so a refuted page in
* it is an invitation to build on a dead claim.
*/
excludeInvalidated?: boolean
/** Match pages carrying at least one of these tags. */
tags?: readonly string[]
/** Match the exact string stored in `frontmatter.kind`. */
kinds?: readonly string[]
/**
* Bound on the rendered brief. A page whose line does not fit is left out of
* the whole brief, so `text`, `hits`, `citationIds`, and `results` always
* describe one identical set of pages.
*/
maxChars?: number
}

export interface KnowledgeBrief {
readonly question: string
/** Retriever id to declare in a receipt minted from `results`. */
readonly retrieverId: typeof KNOWLEDGE_SEARCH_RETRIEVER_ID
/**
* Digest of the retrieval settings this brief used. Pass it as the
* `configDigest` of the receipt's retriever identity. The running package
* version is the caller's to declare, because a bundled build cannot read it.
*/
readonly retrieverConfigDigest: Sha256Digest
readonly hits: readonly KnowledgeSearchHit[]
/** The ids a write should persist in `cites`, in rank order. */
readonly citationIds: readonly KnowledgeId[]
/** Ranked results in the shape `createKnowledgeRetrievalReceipt` takes. */
readonly results: readonly OriginatedKnowledgeSearchResult[]
/** Deterministic Markdown: one `- [id] title — snippet` line per page, in rank order. */
readonly text: string
}

/**
* Rank the knowledge one question can see and render it for injection.
*
* The same page set produces the same brief on every run: ranking, ordering,
* and rendering are deterministic, and no field carries a timestamp.
*/
export function buildKnowledgeBrief(
visiblePages: readonly OriginatedPage[],
question: string,
options: KnowledgeBriefOptions = {},
): KnowledgeBrief {
if (!Array.isArray(visiblePages)) {
throw new TypeError('knowledge brief requires the visible pages')
}
if (typeof question !== 'string' || question.trim().length === 0) {
throw new TypeError('knowledge brief question must be a non-empty string')
}
const limit = options.limit ?? DEFAULT_KNOWLEDGE_BRIEF_LIMIT
const excludeInvalidated = options.excludeInvalidated ?? true
const maxChars = options.maxChars
if (maxChars !== undefined && (!Number.isInteger(maxChars) || maxChars < 0)) {
throw new Error(`knowledge brief maxChars must be a non-negative integer, got ${maxChars}`)
}

const originByPage = new Map<object, PageOrigin>()
for (const entry of visiblePages) originByPage.set(entry.page, entry.origin)

const ranked = searchKnowledgePages(
visiblePages.map((entry) => entry.page),
question,
{
limit,
excludeInvalidated,
...(options.tags === undefined ? {} : { tags: options.tags }),
...(options.kinds === undefined ? {} : { kinds: options.kinds }),
},
)

const hits: KnowledgeSearchHit[] = []
const lines: string[] = []
let length = 0
for (const hit of ranked) {
const line = briefLine(hit)
const next = length === 0 ? line.length : length + 1 + line.length
if (maxChars !== undefined && next > maxChars) break
hits.push(hit)
lines.push(line)
length = next
}

return Object.freeze({
question: question.trim(),
retrieverId: KNOWLEDGE_SEARCH_RETRIEVER_ID,
retrieverConfigDigest: canonicalCandidateDigest({
limit,
excludeInvalidated,
tags: options.tags === undefined ? null : [...options.tags],
kinds: options.kinds === undefined ? null : [...options.kinds],
maxChars: maxChars ?? null,
}),
hits: Object.freeze(hits),
citationIds: Object.freeze(hits.map((hit) => hit.citationId)),
results: Object.freeze(
hits.map((hit) => Object.freeze({ ...hit, origin: originOf(originByPage, hit) })),
),
text: lines.join('\n'),
})
}

function briefLine(hit: KnowledgeSearchHit): string {
const snippet = hit.snippet.replace(/\s+/g, ' ').trim()
return snippet === ''
? `- [${hit.citationId}] ${hit.page.title}`
: `- [${hit.citationId}] ${hit.page.title} — ${snippet}`
}

function originOf(
originByPage: ReadonlyMap<object, PageOrigin>,
hit: KnowledgeSearchHit,
): PageOrigin {
const origin = originByPage.get(hit.page)
if (origin === undefined) {
throw new Error(`knowledge brief ranked a page outside the visible chain: ${hit.page.path}`)
}
return origin
}
Loading