feat(hooks): log CAT-UNVERIFIED tags instead of treating them as evidence - #464
Closed
EdbertChan wants to merge 1 commit into
Conversation
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_fa800cfd-771d-4e04-9bfc-6c1dcf3f2767) |
…ence cat-mode/SKILL.md:269 says a hedge is "a trigger to verify, never a place to stop". Every evidence hook implemented the opposite: markers.well_formed_tags() is consumed as equivalent to evidence (prove-it-ship-gate/detect.py), so a correctly-formed tag silenced the whole stack, was counted nowhere, and was revisited never. Only malformed and legacy tag shapes ever fired. The prose forbade stopping while the tooling rewarded it. unverified-tag-ledger records each well-formed tag against the session and re-surfaces outstanding claims on the next prompt. It deliberately does not block the emitting turn: the tag exists for checks that cannot run in that turn, so blocking there deadlocks. A claim is discharged when a later turn runs a verification tool and stops re-emitting it; one outstanding three turns or more is reported as a reflect trigger. Observed 2026-09-11 in a NiceSpeak streaming session: two well-formed tags were emitted, each ended its turn, neither left a trace. Both are the positive fixtures in tests/test_hooks.py. Fail-before/pass-after, same payload: diu-stop, hedge-runs-prove-it and prove-it-ship-gate all exit 0 silent with zero ledger rows; the new Stop hook logs the claim and the UserPromptSubmit hook names it on the next prompt. 12 unit tests, check_hook_test_coverage.py and check_no_comments pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018u8S5ct3kFhosinSbybc7W Change-Id: I1846b813ee3eeb6a1984eda8cbb8d39399c16d29
EdbertChan
force-pushed
the
stack/EdbertChan/reflect/unverified-tag-ledger-20260911/log-cat-unverified-tags-instead-treating-them--1846b813
branch
from
September 12, 2026 03:11
9270245 to
44d7c23
Compare
Owner
Author
|
This pull request is part of a Mergify stack:
|
3 tasks
Owner
Author
Revision history
|
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_d6d57674-f31f-45b0-acf9-31fbd4bf623f) |
Contributor
|
Tick the box to add this pull request to the merge queue (same as
|
EdbertChan
deleted the
stack/EdbertChan/reflect/unverified-tag-ledger-20260911/log-cat-unverified-tags-instead-treating-them--1846b813
branch
September 12, 2026 06:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Claude can say "I could not check this" in a reply. Until now that was free.
Nothing wrote the claim down. It vanished the moment the reply was sent.
This logs those claims per session. Any still open are listed again the next
time the user types.
A claim closes once a later reply runs a real check and stops repeating it.
The written rule already said a hedge means go and check. The tools said the
opposite: the tag silenced every evidence gate.
Review Claim
Well-formed unverified-claim tags are recorded per session and re-surfaced on
the next prompt until a later turn verifies them.
Review Lane
behavior
Review Unit
engine-runtime
Safety Invariant
The new hook writes only its own ledger file under
~/.cache/catstack-unverified-ledger/and prints text. It blocks no turn, editsno repo file, and no other hook reads its output, so a bug here cannot change
what any existing gate decides.
Slice Rationale
Recording is separable from enforcing. This slice makes the claims visible
without changing whether a turn may end; the stacked follow-up adds the
refusal. Landing both together would mix a new data path and a new block into
one review.
Non-goals
diu-stop,hedge-runs-prove-it, orprove-it-ship-gate._markers/markers.pyis untouched.diu-stopalready rejects.Test Plan
Test Plan
cd engine/hooks/unverified-tag-ledger && python3 -m unittest discover -s testspython3 engine/skills/make-pr/scripts/preflight.py --base origin/mainpython3 scripts/check_hook_test_coverage.py engine/hooks/unverified-tag-ledgerpython3 scripts/check_no_new_comments.pypython3 scripts/check_ecosystem_boundaries.pyObserved on this branch:
Fixture vs live: unit and fixture only. The hook is not installed on this
machine, so the refusal has never fired inside a real session:
Revert Plan
Revert Plan
git revert <sha>, thenbash install.shto drop the symlinkrm -rf ~/.cache/catstack-unverified-ledger