Program: finish the evidence-addressable run-scoped knowledge substrate

[drewstone]

Goal

Make agent-knowledge the only operational knowledge substrate for Discovery Lab and other Runtime applications, without absorbing provider execution or research policy.

The first implementation slice is on branch feat/external-lineage-qualified-citations: external lineage authority, scope-qualified page references, and explicit resolution outcomes.

Remaining owner work

Checked items name the PR that landed them. An unchecked item states exactly what is missing, not what is present.

Retrieval evidence

• Canonical retrieval receipt containing run, actor, query, visible roots, algorithm/version, returned qualified references, content digests, scores, ranks, and time. — feat(evidence): prove retrieval-to-outcome knowledge use #153, reshaped by feat(receipts): reference the visibility snapshot from a retrieval receipt #163 so the receipt references the visibility snapshot instead of embedding it.
• Downstream disposition linking a receipt to a claim, experiment, decision, rejection, contradiction, irrelevance judgment, or explicit no-use. — use receipts in feat(evidence): prove retrieval-to-outcome knowledge use #153; KnowledgeRetrievalDisposition with relation: 'irrelevant' | 'no-use' and no rank in feat(tools): export provider-neutral knowledge tools and record a no-use retrieval #168.
• Deterministic export/import and digest. — rfc8785-sha256 throughout, plus encodeKnowledgeVisibilitySnapshot / decodeKnowledgeVisibilitySnapshot / knowledgeVisibilityArtifactRef (feat(receipts): reference the visibility snapshot from a retrieval receipt #163).

Claim graph

• First-class support, contradiction, supersession, and retraction relationships. — support and contradiction are relation-graph predicates (feat(citations): resolve run-scoped knowledge without silent shadowing #152, feat(graph): add a labeled relation graph with a query surface #161); retraction is KnowledgePageInvalidation plus the propagation pass (feat(pages): make evidence and invalidation integrity first-class #147, feat(invalidation): propagate a page invalidation to every page that cites it #165). Missing: a page-level supersedes field and its predicate.
• Preserve all sides of a dispute; no last-write-wins truth overwrite. — a chain read retains a repeated id at every origin, and contradicts leaves both pages live.
• Referential-integrity and cycle checks with explicit unresolved states. — post-hoc audit in feat(citations): resolve run-scoped knowledge without silent shadowing #152, write-time refusal in feat(write): gate a knowledge write on duplication and citation resolution #164, promotion-time refusal in feat(promotion): promote run-scoped pages with their cited support #167, all reporting resolved / missing / ambiguous. Missing: a cites cycle check in auditKnowledgeCitations.
• Freshness/validity that keeps stale history visible but prevents it satisfying readiness. — source validUntil and the stale metric; citesInvalidated stamping and the cites-invalidated lint (feat(invalidation): propagate a page invalidation to every page that cites it #165); excludeInvalidated, default off in searchKnowledge and on in buildKnowledgeBrief (feat(invalidation): propagate a page invalidation to every page that cites it #165, feat(brief): build a retrieval brief a receipt can prove #166), so refuted pages stay readable but never reach a run as something to cite.

Promotion

• Explicit run-scope → shared-scope promotion record. — feat(promotion): promote run-scoped pages with their cited support #167, at <shared>/.agent-knowledge/promotions/<digest>.json.
• Retain source run/reference/digest, reproductions, decision evidence, actor, reason, and later retraction/supersession. — the record carries the source run, every page digest, which pages were requested and which were carried support, the actor, the reason, and the time (feat(promotion): promote run-scoped pages with their cited support #167). Later retraction reaches a promoted page through invalidation and feat(invalidation): propagate a page invalidation to every page that cites it #165. Missing: reproductions and decision evidence in the record; they live in the application's evidence refs today.
• Runs cannot write shared scope directly. — promoteRunScopedPages (feat(promotion): promote run-scoped pages with their cited support #167) is the one path into shared scope, and every write through it leaves a record.
• Promotion mutation tests and rollback. — mutation tests landed with feat(promotion): promote run-scoped pages with their cited support #167. Missing: rollback.

Tool adapter

• Provider-neutral search/read/record/resolve tools exported by Knowledge. — createKnowledgeTools (feat(tools): export provider-neutral knowledge tools and record a no-use retrieval #168) returning ToolDefinition[] from @tangle-network/agent-interface.
• Runtime only transports authorized tool definitions and calls; no Knowledge execution loop. — every handler lives in this package (feat(tools): export provider-neutral knowledge tools and record a no-use retrieval #168).
• Application supplies roots, lineage resolver, sensitivity policy, and effect authorization. — roots and lineage authority come from createRunScopedStores; effect authorization is the host's, since it decides which definitions to register. Missing: a sensitivity-policy seam; filtering today is tags, kinds, and the caller's own predicate.

Migration support

• Lossless import of Discovery Lab's retained page/frontmatter fields. — the raw frontmatter record is retained on every page (fix(frontmatter): make scalar serialization lossless #142).
• Frozen-corpus parity suite for visibility, search, evidence verdicts, citation resolution, concurrency, path safety, and round trips. — blocked: no Discovery Lab corpus fixture exists in this repo.
• Historical bare-id compatibility reports ambiguity and never silently chooses a scope. — the ambiguous resolution status (feat(citations): resolve run-scoped knowledge without silent shadowing #152), surfaced by knowledge_read and knowledge_resolve (feat(tools): export provider-neutral knowledge tools and record a no-use retrieval #168).

Acceptance

A process can restart in another checkout, load a retained run bundle, resolve every consequential citation to the same scoped page bytes, identify which retrieval influenced which decision, preserve contradictions and stale evidence, and promote nothing without an explicit record.

This issue is complete only after Discovery Lab switches new writes to this package and deletes its local page, lineage, search, citation, evidence, and promotion implementation.

[drewstone]

Checklist updated in place. This wave landed the Promotion section and the Tool adapter section, and completed the Retrieval-evidence section.

Landed since the last update

• pagesDirectory threaded through write, index, search, and chain — fix(write-protocol): thread pagesDirectory through the write, index, search, and chain paths #160 (closes Write protocol hardcodes knowledge/ while the read side is parameterized #159)
• Labeled relation graph with knowledgePageRelations as the single source of page relations — feat(graph): add a labeled relation graph with a query surface #161 (closes Provenance graph: no builder or query surface for labeled multi-edge relations (KnowledgeRelation) #124)
• BM25 lexical index behind searchKnowledge, retriever id bm25-rrf-v1 — feat(search): rank with BM25 over a real lexical index #162 (closes feat(search): adopt a real lexical index (BM25/FTS5) behind the search provider port #132)
• Retrieval receipts v2, referencing the visibility snapshot, schema 2.0.0 — feat(receipts): reference the visibility snapshot from a retrieval receipt #163 (closes Make retrieval receipts reference, not embed, the complete visibility snapshot #154)
• Write-time intake gate: duplicate refusal with three structural outs, and dangling-cite refusal — feat(write): gate a knowledge write on duplication and citation resolution #164
• Invalidation propagation, cites-invalidated lint, excludeInvalidated search option — feat(invalidation): propagate a page invalidation to every page that cites it #165
• buildKnowledgeBrief, whose results feed createKnowledgeRetrievalReceipt directly — feat(brief): build a retrieval brief a receipt can prove #166
• promoteRunScopedPages with carried cited support and a content-addressed promotion record — feat(promotion): promote run-scoped pages with their cited support #167
• createKnowledgeTools (ToolDefinition[]) and KnowledgeRetrievalDisposition with irrelevant / no-use — feat(tools): export provider-neutral knowledge tools and record a no-use retrieval #168

Published: 10.2.0 through 10.6.0.

What remains, and why

Blocked on one missing artifact — a frozen Discovery Lab corpus fixture, which does not exist in this repository:

1. Frozen-corpus parity suite for visibility, search, evidence verdicts, citation resolution, concurrency, path safety, and round trips. There is nothing to run it against. tests/contracts holds cohort and claim-compat suites only.
2. Discovery Lab switch-over — pointing tools/kb.mjs, promote-claims.mjs, retraction-stamp.mjs, and question-brief.mjs at these APIs and deleting the local page, lineage, search, citation, evidence, and promotion implementations. The acceptance clause of this issue depends on it, so this issue stays open as the tracker.

Not blocked, just not built, each an unchecked box above with the gap named:

3. Page-level supersedes and its relation-graph predicate.
4. A cites cycle check in auditKnowledgeCitations.
5. Promotion rollback. Mutation tests landed with feat(promotion): promote run-scoped pages with their cited support #167.
6. Reproductions and decision evidence inside the promotion record; they live in the application's evidence refs today.
7. A sensitivity-policy seam on the tool adapter; filtering today is tags, kinds, and the caller's predicate.

Items 3-7 are small and independent. Item 1 and item 2 need the corpus first.