Skip to content

feat(hooks): log CAT-UNVERIFIED tags instead of treating them as evidence - #464

Closed
EdbertChan wants to merge 1 commit into
mainfrom
stack/EdbertChan/reflect/unverified-tag-ledger-20260911/log-cat-unverified-tags-instead-treating-them--1846b813
Closed

feat(hooks): log CAT-UNVERIFIED tags instead of treating them as evidence#464
EdbertChan wants to merge 1 commit into
mainfrom
stack/EdbertChan/reflect/unverified-tag-ledger-20260911/log-cat-unverified-tags-instead-treating-them--1846b813

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Summary

Claude can say "I could not check this" in a reply. Until now that was free.

Nothing wrote the claim down. It vanished the moment the reply was sent.

This logs those claims per session. Any still open are listed again the next
time the user types.

A claim closes once a later reply runs a real check and stops repeating it.

The written rule already said a hedge means go and check. The tools said the
opposite: the tag silenced every evidence gate.

Review Claim

Well-formed unverified-claim tags are recorded per session and re-surfaced on
the next prompt until a later turn verifies them.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

The new hook writes only its own ledger file under
~/.cache/catstack-unverified-ledger/ and prints text. It blocks no turn, edits
no repo file, and no other hook reads its output, so a bug here cannot change
what any existing gate decides.

Slice Rationale

Recording is separable from enforcing. This slice makes the claims visible
without changing whether a turn may end; the stacked follow-up adds the
refusal. Landing both together would mix a new data path and a new block into
one review.

Non-goals

  • Does not change diu-stop, hedge-runs-prove-it, or prove-it-ship-gate.
  • Does not change what counts as a well-formed tag; _markers/markers.py is untouched.
  • Does not refuse or block any turn. That is the stacked follow-up.
  • Does not report malformed tags, which diu-stop already rejects.

Test Plan

Test Plan
  • cd engine/hooks/unverified-tag-ledger && python3 -m unittest discover -s tests
  • python3 engine/skills/make-pr/scripts/preflight.py --base origin/main
  • python3 scripts/check_hook_test_coverage.py engine/hooks/unverified-tag-ledger
  • python3 scripts/check_no_new_comments.py
  • python3 scripts/check_ecosystem_boundaries.py

Observed on this branch:

Ran 17 tests in 0.021s
OK

unit    engine-runtime: 8 file(s)
gate    python3 scripts/check_hook_test_coverage.py engine/hooks/unverified-tag-ledger
        check_hook_test_coverage: OK (1 hook(s) checked)
ok      preflight passed

Fixture vs live: unit and fixture only. The hook is not installed on this
machine, so the refusal has never fired inside a real session:

$ ls -la ~/.claude/hooks/unverified-tag-ledger
ls: /Users/edbertchan/.claude/hooks/unverified-tag-ledger: No such file or directory
$ ls -d ~/.cache/catstack-unverified-ledger
ls: /Users/edbertchan/.cache/catstack-unverified-ledger: No such file or directory

Revert Plan

Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert <sha>, then bash install.sh to drop the symlink
  • Post-revert steps: optionally rm -rf ~/.cache/catstack-unverified-ledger
  • Data migration? No

@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_fa800cfd-771d-4e04-9bfc-6c1dcf3f2767)

…ence

cat-mode/SKILL.md:269 says a hedge is "a trigger to verify, never a place to
stop". Every evidence hook implemented the opposite: markers.well_formed_tags()
is consumed as equivalent to evidence (prove-it-ship-gate/detect.py), so a
correctly-formed tag silenced the whole stack, was counted nowhere, and was
revisited never. Only malformed and legacy tag shapes ever fired. The prose
forbade stopping while the tooling rewarded it.

unverified-tag-ledger records each well-formed tag against the session and
re-surfaces outstanding claims on the next prompt. It deliberately does not
block the emitting turn: the tag exists for checks that cannot run in that
turn, so blocking there deadlocks. A claim is discharged when a later turn runs
a verification tool and stops re-emitting it; one outstanding three turns or
more is reported as a reflect trigger.

Observed 2026-09-11 in a NiceSpeak streaming session: two well-formed tags were
emitted, each ended its turn, neither left a trace. Both are the positive
fixtures in tests/test_hooks.py.

Fail-before/pass-after, same payload: diu-stop, hedge-runs-prove-it and
prove-it-ship-gate all exit 0 silent with zero ledger rows; the new Stop hook
logs the claim and the UserPromptSubmit hook names it on the next prompt.
12 unit tests, check_hook_test_coverage.py and check_no_comments pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018u8S5ct3kFhosinSbybc7W
Change-Id: I1846b813ee3eeb6a1984eda8cbb8d39399c16d29
@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/unverified-tag-ledger-20260911/log-cat-unverified-tags-instead-treating-them--1846b813 branch from 9270245 to 44d7c23 Compare September 12, 2026 03:11
@EdbertChan

EdbertChan commented Sep 12, 2026

Copy link
Copy Markdown
Owner Author

This pull request is part of a Mergify stack:

# Pull Request Link
1 feat(hooks): log CAT-UNVERIFIED tags instead of treating them as evidence #464 👈
2 feat(hooks): refuse a CAT-UNVERIFIED tag from a turn that ran no check #468
3 fix(hooks): wire unverified-tag-ledger into settings and declare its subagent opt-out #474

@EdbertChan

Copy link
Copy Markdown
Owner Author

Revision history

# Type Changes Reason Date
1 initial 9270245 2026-09-12 03:11 UTC
2 rebase 9270245 → 44d7c23 (rebase only) 2026-09-12 03:11 UTC

@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_d6d57674-f31f-45b0-acf9-31fbd4bf623f)

@mergify

mergify Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@EdbertChan EdbertChan closed this Sep 12, 2026
@EdbertChan
EdbertChan deleted the stack/EdbertChan/reflect/unverified-tag-ledger-20260911/log-cat-unverified-tags-instead-treating-them--1846b813 branch September 12, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant