Skip to content

feat(hooks): record CAT-UNVERIFIED tags and refuse one from a turn that ran no check - #477

Merged
mergify[bot] merged 1 commit into
mainfrom
stack/EdbertChan/reflect/unverified-tag-ledger-20260911/record-cat-unverified-tags-refuse-one-turn-ran--3466baee
Sep 12, 2026
Merged

feat(hooks): record CAT-UNVERIFIED tags and refuse one from a turn that ran no check#477
mergify[bot] merged 1 commit into
mainfrom
stack/EdbertChan/reflect/unverified-tag-ledger-20260911/record-cat-unverified-tags-refuse-one-turn-ran--3466baee

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Sep 12, 2026

Copy link
Copy Markdown
Owner

cat-mode/SKILL.md:269 says a hedge is "a trigger to verify, never a place to
stop". Every evidence hook implemented the opposite: markers.well_formed_tags()
is consumed as equivalent to evidence (prove-it-ship-gate/detect.py), so a
correctly-formed tag silenced the whole stack, was counted nowhere, and was
revisited never. Only malformed and legacy shapes ever fired. The prose forbade
stopping while the tooling rewarded it.

unverified-tag-ledger records each well-formed tag against the session, refuses
a turn that tagged a claim without running any verification tool, and
re-surfaces anything still open on the next prompt. Requiring an ATTEMPT is not
requiring success: run the check, and if it cannot settle the claim the tag is
then honest. stop_hook_active releases the refusal, or the rewrite turn -- which
has no tool call of its own -- would loop forever. A claim is discharged once a
later turn verifies and stops repeating it; one outstanding three turns or more
is reported as a reflect trigger.

Observed 2026-09-11 in a NiceSpeak streaming session: two well-formed tags were
emitted, each ended its turn, neither left a trace. Both are the positive
fixtures in tests/test_hooks.py. Replaying that session's own turn:

A) tag, zero tools -> exit=2, "ran no verification tool. Untried
claim(s): that it widened scope past the one
session I gave it"
B) same tag after Bash -> exit=0
C) stop_hook_active -> exit=0, no loop
D) no tag, no tools -> exit=0

Three of this repo's own gates caught real defects while landing it, and each
fix is in this commit:

  • test_every_claude_hook_entrypoint_is_in_settings / TestSubagentStopInheritance:
    the hook shipped with no install_claude_hook.py, so neither entry point
    reached settings.json. Added on diu-stop's pattern, called from install.sh.
  • the same two gates again, after the installer was copied from diu-stop and
    kept its bare basename markers. Because this hook uses the same two
    basenames, the idempotent "replace our own entry" step deleted diu-stop's
    entries instead. Markers are now directory-qualified, so an installer only
    ever touches its own entries. Without this, install.sh would have silenced
    diu-stop's word count, evidence and plain-words checks.
  • test_no_instructional_file_names_the_retired_marker: the README wrote the
    retired bare marker without naming it as retired.

Lands as one commit because each earlier slice failed those gates on its own,
and a slice that cannot go green alone is not a reviewable slice. The
pre-squash heads are on backup/tagledger-0fc2a13 and backup/tagledger2-81157a3.

subagent_stop opts out with a reason, registered in
tests/test_mirror_subagent_stop.py and docs/ecosystem.md: the ledger is keyed by
session id, so a subagent would log its tags against the parent's ledger, and it
has no next user prompt to be reminded at.

17 hook tests, 10 mirror tests, 6 vocabulary tests, and the full local
tests/test_install.py run all pass.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_018u8S5ct3kFhosinSbybc7W


Note

Medium Risk
Changes Claude Stop/UserPromptSubmit behavior for every session after install, including hard turn refusals and persistent local ledger state; installer merge logic was fixed to avoid touching diu-stop hooks.

Overview
Adds unverified-tag-ledger, a Claude hook pair that treats well-formed {{CAT-UNVERIFIED: …}} tags as deferred claims, not evidence that lets a turn end quietly.

On Stop, it logs tags to a per-session JSONL ledger under ~/.cache/catstack-unverified-ledger/, blocks (exit 2) when the assistant emits a tag without having used any verification tool (Bash/Read/Grep/etc.), and honors stop_hook_active so rewrite turns do not loop. On UserPromptSubmit, it injects a reminder listing outstanding claims; claims discharge after a verify turn stops repeating them, and items open 3+ turns escalate as a reflect trigger.

Install wires the hook via install.sh (symlink + install_claude_hook.py merging Stop and UserPromptSubmit into settings.json). The installer uses directory-qualified command markers so it does not remove diu-stop entries that share the same script basenames. subagent_stop.inherit: false is documented in docs/ecosystem.md and reflected in tests/test_mirror_subagent_stop.py (session-scoped ledger, no next user prompt for subagents).

Ships 17 unit tests covering real-session tag fixtures, block/allow paths, ledger lifecycle, and corrupt-row handling.

Reviewed by Cursor Bugbot for commit 30011b8. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_c37abe08-47d4-4bea-b939-870c7e831570)

@mergify

mergify Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/unverified-tag-ledger-20260911/record-cat-unverified-tags-refuse-one-turn-ran--3466baee branch from c5f0bff to c19cce7 Compare September 12, 2026 06:33
@EdbertChan

Copy link
Copy Markdown
Owner Author

This pull request is part of a Mergify stack:

# Pull Request Link
1 feat(hooks): record CAT-UNVERIFIED tags and refuse one from a turn that ran no check #477 👈
2 fix(hooks): stop the installer deleting diu-stop's settings entries #481

@EdbertChan

EdbertChan commented Sep 12, 2026

Copy link
Copy Markdown
Owner Author

Revision history

# Type Changes Reason Date
1 initial c5f0bff 2026-09-12 06:33 UTC
2 rebase c5f0bff → c19cce7 (rebase only) 2026-09-12 06:33 UTC
3 content c19cce7 → b47e685 (raw) 2026-09-12 06:40 UTC
4 content b47e685 → 30011b8 (raw) 2026-09-12 06:51 UTC

@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_52df7d7e-dfc0-407b-9602-f62365cde857)

@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/unverified-tag-ledger-20260911/record-cat-unverified-tags-refuse-one-turn-ran--3466baee branch from c19cce7 to b47e685 Compare September 12, 2026 06:40
@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_89ecab73-8345-46ac-bfae-74110749c01c)

…at ran no check

cat-mode/SKILL.md:269 says a hedge is "a trigger to verify, never a place to
stop". Every evidence hook implemented the opposite: markers.well_formed_tags()
is consumed as equivalent to evidence (prove-it-ship-gate/detect.py), so a
correctly-formed tag silenced the whole stack, was counted nowhere, and was
revisited never. Only malformed and legacy shapes ever fired. The prose forbade
stopping while the tooling rewarded it.

unverified-tag-ledger records each well-formed tag against the session, refuses
a turn that tagged a claim without running any verification tool, and
re-surfaces anything still open on the next prompt. Requiring an ATTEMPT is not
requiring success: run the check, and if it cannot settle the claim the tag is
then honest. stop_hook_active releases the refusal, or the rewrite turn -- which
has no tool call of its own -- would loop forever. A claim is discharged once a
later turn verifies and stops repeating it; one outstanding three turns or more
is reported as a reflect trigger.

Observed 2026-09-11 in a NiceSpeak streaming session: two well-formed tags were
emitted, each ended its turn, neither left a trace. Both are the positive
fixtures in tests/test_hooks.py. Replaying that session's own turn:

  A) tag, zero tools      -> exit=2, "ran no verification tool. Untried
                             claim(s): that it widened scope past the one
                             session I gave it"
  B) same tag after Bash  -> exit=0
  C) stop_hook_active     -> exit=0, no loop
  D) no tag, no tools     -> exit=0

Three of this repo's own gates caught real defects while landing it, and each
fix is in this commit:

- test_every_claude_hook_entrypoint_is_in_settings / TestSubagentStopInheritance:
  the hook shipped with no install_claude_hook.py, so neither entry point
  reached settings.json. Added on diu-stop's pattern, called from install.sh.
- the same two gates again, after the installer was copied from diu-stop and
  kept its bare basename markers. Because this hook uses the same two
  basenames, the idempotent "replace our own entry" step deleted diu-stop's
  entries instead. Markers are now directory-qualified, so an installer only
  ever touches its own entries. Without this, install.sh would have silenced
  diu-stop's word count, evidence and plain-words checks.
- test_no_instructional_file_names_the_retired_marker: the README wrote the
  retired bare marker without naming it as retired.

Lands as one commit because each earlier slice failed those gates on its own,
and a slice that cannot go green alone is not a reviewable slice. The
pre-squash heads are on backup/tagledger-0fc2a13 and backup/tagledger2-81157a3.

subagent_stop opts out with a reason, registered in
tests/test_mirror_subagent_stop.py and docs/ecosystem.md: the ledger is keyed by
session id, so a subagent would log its tags against the parent's ledger, and it
has no next user prompt to be reminded at.

17 hook tests, 10 mirror tests, 6 vocabulary tests, and the full local
tests/test_install.py run all pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018u8S5ct3kFhosinSbybc7W
Change-Id: I3466baee81db9fd240f1a527df4610ad5507919a
@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/unverified-tag-ledger-20260911/record-cat-unverified-tags-refuse-one-turn-ran--3466baee branch from b47e685 to 30011b8 Compare September 12, 2026 06:51
@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_d3c15ad3-a908-4dd1-9d1e-145c4aae130a)

@EdbertChan

Copy link
Copy Markdown
Owner Author

@Mergifyio queue

@mergify

mergify Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

This pull request spent 4 minutes 57 seconds in the queue, including 4 minutes 40 seconds running CI.

Required conditions to merge
  • check-success = lint
  • check-success = test

@mergify mergify Bot added the queued label Sep 12, 2026
@mergify
mergify Bot merged commit f2f46f1 into main Sep 12, 2026
6 checks passed
@mergify mergify Bot removed the queued label Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant