Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/ecosystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,10 @@ A subagent launched through the Agent tool runs under the same
which mirrors each `engine/hooks/<name>/claude*.hook.json` `Stop` entry, and a
hook opts out only in its own manifest with
`"subagent_stop": {"inherit": false, "reason": "..."}` (today:
`frustration-watchdog`, which reads the human's last message, and `auto-pr`,
whose PR instruction is for the session owner). Under `SubagentStop`,
`frustration-watchdog`, which reads the human's last message, `auto-pr`,
whose PR instruction is for the session owner, and `unverified-tag-ledger`,
whose ledger is keyed by session id and whose reminder needs a next user
prompt). Under `SubagentStop`,
`transcript_path` is the parent's transcript and `agent_transcript_path` is
the subagent's own, so transcript-reading hooks prefer the latter.
`UserPromptSubmit` hooks never reach a subagent, because its prompt arrives
Expand Down
81 changes: 81 additions & 0 deletions engine/hooks/unverified-tag-ledger/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# unverified-tag-ledger

A well-formed `{{CAT-UNVERIFIED: <claim> -- cannot verify: <reason>}}` tag is a
**deferral**, not a discharge. This hook makes that true mechanically.

## The gap it closes

`cat-mode/SKILL.md:269` says:

> Any hedge auto-runs prove-it in the same turn — a hedge is a trigger to
> verify, never a place to stop.

Every evidence hook implemented the opposite. `_markers/markers.py` defines
`excuses_paragraph()`, and consumers treat a well-formed tag as equivalent to
evidence — for example `prove-it-ship-gate/detect.py`:

```python
if markers.well_formed_tags(message) or has_evidence(message):
```

Only the *broken* tag shapes ever fired: `malformed_tags` (names no blocker)
and `has_legacy_marker` (the retired bare `UNVERIFIED:` form), both in
`diu-stop/claude_stop_check.py`. A correctly-formed tag produced silence from
the entire stack, was counted nowhere, and was revisited never. The written
rule said "never a place to stop" while the tooling rewarded stopping.

Observed 2026-09-11 (NiceSpeak streaming session): two well-formed tags were
emitted, each ended its turn, and neither left a trace. Both are the positive
fixtures in `tests/test_hooks.py`.

## What it does

- **Stop** (`claude_stop_check.py`) — parses well-formed tags out of the reply
and appends them to a per-session ledger, then **refuses the turn** (exit 2)
if it tagged a claim without running any verification tool. A tag earns its
place only after an attempt: requiring an attempt is not requiring success, so
run the check and tag it only when the check cannot settle the claim.
`stop_hook_active` releases the refusal, or the rewrite turn — which has no
tool call of its own — would loop forever.
- **UserPromptSubmit** (`claude_prompt_reminder.py`) — lists outstanding claims
on the next prompt, quoting the rule and naming each claim plus what it is
blocked on. The next prompt is the earliest point a reminder can change
behaviour without preventing the turn from ending at all.
- **Discharge** — a claim is resolved when a later turn runs a verification tool
(`Bash`, `Read`, `Grep`, `Glob`, `NotebookRead`) and stops re-emitting it.
- **Escalation** — a claim outstanding `ESCALATE_AFTER_TURNS` (3) turns or more
is reported as a reflect trigger rather than accumulating quietly.

Malformed tags are deliberately ignored here; `diu-stop` already rejects those.

## Ledger

`~/.cache/catstack-unverified-ledger/<session_id>.jsonl`, one JSON row per
claim (`claim`, `reason`, `first_seen`, `turns`, `resolved`). Override the
directory with `CATSTACK_TAG_LEDGER_DIR` (the tests use a tempdir). A row that
is not JSON is reported on stderr and skipped, never silently dropped.

## Tests

```
cd engine/hooks/unverified-tag-ledger && python3 -m unittest discover -s tests
```

17 tests: both real tags as positive fixtures, the refusal on an untried tag, the `stop_hook_active` release that prevents a refusal loop, a no-tag negative control, the
malformed-tag negative, discharge-on-verify, stays-outstanding-without-verify,
no duplicate on re-emit, stale escalation, per-session isolation, and the
corrupt-row report.

## Prior art

The shape is a **defect-tracking rule**: a known-unresolved item is recorded
and re-surfaced rather than left to memory. Nancy G. Leveson, *CAST Handbook:
How to Learn More from Incidents and Accidents*, 2019
(https://psas.scripts.mit.edu/home/get_file4.php?name=CAST_Handbook.pdf) names
the failure this prevents — "fixing the symptoms of problems but not tackling
the systemic causes" — by requiring the count be published before any single
item is called fixed. Saltzer and Schroeder, "Basic Principles of Information
Protection", 1975
(https://web.mit.edu/Saltzer/www/publications/protection/Basic.html) supplies
the default: base the decision on explicit permission, so absence of a check is
never read as a pass.
20 changes: 20 additions & 0 deletions engine/hooks/unverified-tag-ledger/claude.hook.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"hooks": {
"Stop": [
{
"matcher": "*",
"hooks": [
{
"type": "command",
"command": "python3 $HOME/.claude/hooks/unverified-tag-ledger/claude_stop_check.py",
"timeout": 10
}
]
}
]
},
"subagent_stop": {
"inherit": false,
"reason": "the ledger and its next-prompt reminder belong to the session owner; a subagent shares the parent's session id and would log its own tags against the parent's ledger, and it has no next user prompt to be reminded at"
}
}
15 changes: 15 additions & 0 deletions engine/hooks/unverified-tag-ledger/claude.prompt.hook.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "python3 $HOME/.claude/hooks/unverified-tag-ledger/claude_prompt_reminder.py",
"timeout": 10
}
]
}
]
}
}
31 changes: 31 additions & 0 deletions engine/hooks/unverified-tag-ledger/claude_prompt_reminder.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env python3
"""Claude Code UserPromptSubmit hook: surface CAT-UNVERIFIED claims that
earlier turns deferred and never settled. This is where cat-mode/SKILL.md:269
gets teeth -- the Stop hook cannot block the turn that emits a tag without
deadlocking, so the reminder lands on the next prompt instead.
"""
from __future__ import annotations

import json
import sys

from detect import reminder


def main() -> None:
try:
payload = json.load(sys.stdin)
except (json.JSONDecodeError, OSError) as exc:
sys.stderr.write(f"unverified-tag-ledger: unreadable payload, no reminder: {exc!r}\n")
return
try:
text = reminder(str((payload or {}).get("session_id") or ""))
except Exception as exc:
sys.stderr.write(f"unverified-tag-ledger: reminder error, continuing: {exc!r}\n")
return
if text:
print(text)


if __name__ == "__main__":
main()
35 changes: 35 additions & 0 deletions engine/hooks/unverified-tag-ledger/claude_stop_check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/env python3
"""Claude Code Stop hook: record well-formed CAT-UNVERIFIED tags against the
session, and refuse a turn that tags a claim without having run any
verification tool (cat-mode/SKILL.md:269 -- a hedge is a trigger to verify).
`stop_hook_active` releases the refusal so the rewrite turn can finish. Fails
open on read or parse errors.
"""
from __future__ import annotations

import json
import sys

from detect import evaluate


def main() -> None:
try:
payload = json.load(sys.stdin)
except (json.JSONDecodeError, OSError) as exc:
sys.stderr.write(f"unverified-tag-ledger: unreadable payload, allowing: {exc!r}\n")
return
try:
verdict = evaluate(payload if isinstance(payload, dict) else {})
except Exception as exc:
sys.stderr.write(f"unverified-tag-ledger: detector error, allowing this reply: {exc!r}\n")
return
if verdict["block"]:
sys.stderr.write(verdict["block"] + "\n")
sys.exit(2)
if verdict["note"]:
sys.stderr.write(verdict["note"] + "\n")


if __name__ == "__main__":
main()
Loading
Loading