Skip to content

feat(briefs): store briefs after each completed turn - #1806

Merged
HazAT merged 19 commits into
bb/fable-5-1-briefs-evidence-and-searchable-indexin-thr_4riaw5kyekfrom
briefs/m3-storage
Sep 10, 2026
Merged

HazAT merged 19 commits into
bb/fable-5-1-briefs-evidence-and-searchable-indexin-thr_4riaw5kyekfrom
briefs/m3-storage

Conversation

@HazAT

@HazAT HazAT commented Sep 9, 2026

Copy link
Copy Markdown
Member

Stacked on #1805. Milestone 3 of conversation briefs: production wiring of storage and generation. No user-visible surface yet; search, API, and dashboard follow in #1808 and #1810.

What

  • Brief generation is an app-level opt-in: createApp({ briefs: { enabled: true } }), mirroring experimental features. It is off by default because it costs one default-model call per completed Turn; the example app, the junior init scaffold, and junior chat enable it. Integration tests stay off unless they opt in, so no test turn ever reaches the model gateway.
  • junior_conversation_briefs: append-only versions, one per completed Turn (conversation_id, version PK; unique conversation_id, turn_id; through_seq; GIN index on search_text for feat(briefs): search briefs from the agent, API, and feed #1808).
  • A core briefs.updateBrief task delivered through the existing plugin task queue after Slack, web, and local Turns with a user instruction. It builds its input from durable SQL events through the same event-to-entry mapping the CLI snapshot adapter uses, runs the tuned generator with the configured default model, and stores the next version under a per-conversation lock.
  • Robustness rules, each covered by a test:
    • a retried Turn never calls the model again; it re-emits its event idempotently so a stored Brief cannot lose its cost record;
    • a Turn already covered by a newer Brief (out-of-order delivery) is skipped before any model call;
    • child conversations and purged private conversations are skipped;
    • the storage transaction refuses to write a Brief for a purged non-public conversation, so a purge racing an in-flight task cannot resurrect private content, and leftover private Brief rows keep a tree eligible for purge.
  • briefs/brief_updated structured conversation event with version, model id, item counts, and model cost. Cost rolls into the conversation auxiliary-cost breakdown under the briefs namespace and renders in conversation history.
  • Purge rule: public briefs survive transcript expiry; a non-public root loses its briefs in the same step that scrubs its title. Private stays private.
  • TERMINOLOGY.md gains Brief; the module README documents storage, versioning, purge, and the privacy gate later readers must apply.

Verification

  • tests/component/briefs/conversation-brief-task.test.ts (new), the extended purge scenario in tests/component/conversations/retention.test.ts, the plugin task scheduling and event rendering tests: 84 tests pass against Postgres.
  • tests/unit/cli/init-cli.test.ts keeps the scaffold aligned with the example app.
  • pnpm typecheck, pnpm --filter @sentry/junior lint, pnpm migration-metadata:check, pnpm file-length:check, pnpm test-architecture:check.

Add the provider-neutral Brief schema, snapshot projection, bounded structured generator, deterministic evidence checks, and Markdown renderer.

Add briefs pull and run commands for complete dashboard snapshots and reproducible local generation, with prompt and model overrides and turn-by-turn output. Document the workflow and cover projection, caps, search text, and URL filtering with unit tests.
Resolve the default Brief model from the app's fast-model configuration when a run starts, while keeping explicit CLI model overrides independent.

Accept model citations that match deterministic code change or resource evidence, count them as kept, and document and test both rules.
Cap user and assistant entries, reserve the prompt budget for them first, and use remaining space for the newest tool results while preserving transcript order.

Report omitted entry counts in the model input and cover message retention, tool priority, truncation, ordering, and the total input cap.
Build the durable record from bounded conversation input and scale model output to the record size. Add attribution, runtime-marker, URL, and merge-claim guards while preserving deterministic evidence links.\n\nRevise the prompt and Markdown report so local tuning exposes the record and every guard result.
Use the configured default model for Brief generation and add stated, confirmed, and assumed decision kinds with deterministic consistency checks.\n\nResolve snapshot authors across the full event history, separate resource events from human messages, and expose the event count in the deterministic record. Update rendering, prompt guidance, docs, and focused coverage.
Only flag unsupported merge claims when the conversation has linked evidence to validate. Compare cited URLs against HTML-unescaped transcript text and keep decoded normalized URLs. Leave unresolved snapshot actors unnamed so they cannot enter the participant or attribution record.
Add append-only Brief storage, SQL input projection, and a core post-Turn task that uses the configured default structured model. Record generation cost in a namespaced conversation event and keep task retries idempotent by Turn.

Preserve public Briefs after transcript expiry and delete non-public Briefs when purge scrubs private metadata. Share one reporting-event entry mapper between snapshot and SQL inputs, and document the Brief term and lifecycle.
@vercel

vercel Bot commented Sep 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
junior-docs Ready Ready Preview Sep 9, 2026 9:23pm UTC

Request Review

Use schema-valid local Conversation ids for the root and child task cases. Commit each Turn's complete Pi history before appending reporting events so the second Turn follows the durable agent-history branch while retaining a real tool result.
Reject late private Brief writes in the locked storage transaction and keep leftover private Brief rows eligible for purge.

Skip covered out-of-order Turns before model generation, restore idempotent update events on retries, require complete normalized URL evidence tokens, and render core Brief events outside the plugin catalog.
Treat an already-purged non-public Conversation as an expected task skip before Brief input construction or model generation. Keep the storage transaction check as the race-safe final invariant.
@HazAT HazAT changed the title feat(briefs): store post-turn conversation briefs feat(briefs): store briefs after each completed turn Sep 9, 2026
Keep core Brief task registration disabled unless createApp explicitly enables it. Enable the task for the example app and local chat, while leaving stored event rendering unconditional.

Restore plugin task tests to plugin-only expectations and cover Brief scheduling from the opted-in Brief task scenario.
Keep the Nitro scaffold aligned with the example app, which now opts into Brief generation.
Merge the Brief and generator input schemas into schema.ts, drop the
config indirection, and let callers bind the model into the completion
function so the CLI and the runtime send the same request. Keep a
closing bracket that a cited URL opened, so a Wikipedia-style link is
stored intact instead of truncated.
# Conflicts:
#	packages/junior/src/chat/briefs/README.md
#	packages/junior/src/chat/briefs/generate.ts
#	packages/junior/src/chat/briefs/snapshot.ts
#	packages/junior/tests/unit/briefs/generate.test.ts
Drop the unused store readers, flatten the SQL input adapter into
input.ts, and let the task call the model client with the same request
as `junior briefs run` so the tuned temperature applies in production.
Share the Brief test fixture instead of an inline copy.
@HazAT
HazAT marked this pull request as ready for review September 9, 2026 19:34
@github-actions github-actions Bot added the risk: high PR risk score: high label Sep 9, 2026
Never upgrade a decision kind: a human named on an assumed decision loses
the name instead of becoming a confirmation, and an unknown name keeps the
kind. Match Slack <url|label> links by their URL, try a stray /https suffix
only as an alternate match, and store the form that matched. Bound replay
evidence to what existed at the replayed event. Load app config only when
a run needs the default model so `briefs pull` works with a token alone,
and route `briefs` through the shared CLI dispatcher.
…3-storage

# Conflicts:
#	packages/junior/src/chat/briefs/snapshot.ts
The event read and the sequence boundary were independent queries, so a
Turn completing between them produced a Brief stamped with a sequence it
never read, and the next task skipped that Turn as already covered. The
task now passes its terminal sequence into the input adapter.
Co-Authored-By: Daniel Griesser <dgriesser@sentry.io>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6bd3f18. Configure here.

db,
run.conversationId,
run.runId,
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Public purge races fail the task

Medium Severity

The task lets a purged public Conversation continue so a stored Brief can re-emit its event, but when no Brief exists yet it still requires the turn_completed row. Purge has already deleted that event, so readTurnCompletedSeq throws and retries keep failing.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6bd3f18. Configure here.

@HazAT
HazAT merged commit 2103872 into main Sep 10, 2026
39 checks passed
@HazAT
HazAT deleted the briefs/m3-storage branch September 10, 2026 09:05
HazAT added a commit that referenced this pull request Sep 10, 2026
First of a four-PR stack that gives Junior **conversation briefs**: one
durable, versioned, evidence-backed record per Conversation that
survives transcript expiry and can be indexed and searched. Background:
#1784 and the "work items" discussion; a Brief is the
structured intent + outcome + evidence unit those ideas build on.

Stack: **#1805** generator and tuning loop → #1806 storage and post-turn
generation → #1808 search, tool, and API → #1810 dashboard and cost
report.

## Why

When a transcript expires, nothing durable says what was asked, what was
decided, which PR came out of it, and which decisions are still open. A
Brief keeps exactly that: a model-generated summary with source-checked
links.

## What

- `packages/junior/src/chat/briefs/`: the Brief shape and a pure
`generateBrief` over a provider-neutral `BriefInput`.
- A deterministic **record** (participants, message and turn counts,
duration, location, code changes) built from the input, not the model.
- Summary, intent, outcome with status, **decisions classified as
stated, confirmed, or assumed by Junior**, open decisions with an owner,
durable facts, keywords.
- **Evidence links are deterministic**: code changes and `resource_link`
annotations come from trusted input; a model-cited URL is kept only when
it matches a complete normalized URL token in the transcript, the
previous Brief, or that evidence. Dropped URLs, dropped attributions,
runtime-marker items, and unsupported "merged" claims are reported in an
evidence-check footer.
- Bounded prompt input: messages first, newest tool results fill the
rest, sentence-boundary truncation, depth scaled to the conversation.
- `junior briefs pull <id…> --base-url … --token … --out …` saves a
self-contained snapshot of any public conversation through the dashboard
API with a personal token.
- `junior briefs run <snapshot…> [--model] [--prompt] [--turn-by-turn]
[--out]` replays generation locally and writes `<id>.brief.json` and a
readable `<id>.brief.md`.
- The default model is the app's configured default model. It was chosen
after hand-tuning on 50 public production conversations: it separates
decisions from findings far better than the fast model at roughly two
cents more per large brief. The prompt lives in `chat/briefs/prompt.ts`.

## Verification

- `pnpm --filter @sentry/junior exec vitest run tests/unit/briefs`
- `pnpm typecheck`, `pnpm --filter @sentry/junior lint`, `pnpm
file-length:check`, `pnpm test-architecture:check`, `pnpm docs:check`
- Four rounds of real-model runs over the 50-conversation sample set,
read by hand; sample briefs are not committed (public repo).
HazAT added a commit that referenced this pull request Sep 10, 2026
Stacked on #1806. Milestone 4 of conversation briefs: make briefs
findable.

## What

- `ConversationBriefSearchStore`: full-text search over the latest brief
version per conversation, with `query`, `status`, time range, channel,
and annotation-key filters. Scope is either the current Slack
workspace's public conversations or all public conversations (web).
Purged public conversations are included on purpose; that is what a
brief is for. Private, child, and other-tenant conversations are
excluded.
- `searchConversationBriefs`: a deferred, read-only agent tool
registered for every public conversation. Results carry status, summary,
highlighted excerpt, up to five evidence links, and a dashboard link.
Its description steers the model to use it before message search for
"what did we decide" and "which PR" questions.
- The tool is provider-neutral. Slack channel resolution, the channel
filter parameter, and permalink enrichment live in a small port under
`chat/slack/tools/` that is injected only when Slack context exists, per
`policies/provider-boundaries.md`. The dashboard link builder, which
only builds core dashboard URLs, moves from `chat/slack/` to
`chat/dashboard-link.ts`.
- Detail API: `brief` (latest version, content, updated time) on `GET
/api/conversations/:id`, behind the same privacy gate as annotations.
- Feed search: `q` now matches brief text via full-text search in
addition to the title substring.

## Verification

- New `tests/component/briefs/conversation-brief-search.test.ts` and
`tests/integration/search-conversation-briefs-tool.test.ts`; extended
detail, list, and tool-registration tests.
- `pnpm typecheck`, `pnpm --filter @sentry/junior lint`,
`tool-annotations:check`, `tool-error-classification:check`, `pnpm
file-length:check`, `pnpm test-architecture:check`.

---------

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: high PR risk score: high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants