From 529e75c3d16edad641a527b2862ab24fe0b4e05e Mon Sep 17 00:00:00 2001 From: Edbert Chan Date: Fri, 11 Sep 2026 02:24:05 -0700 Subject: [PATCH] skills(corpus): cat-mode and the prove-it skills name the tag cat-mode, principle-prove-it and prove-it-ship-gate still told the author to write bare `UNVERIFIED:` (or `UNVERIFIED: live path`). cat-mode is always loaded, so that line reached every session. Each now names {{CAT-UNVERIFIED: -- cannot verify: }}. Their test examples are updated to match, and test_cat_mode.py checks the skill and its verify reference with tests/escape_hatch_vocab.py. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01VKsvxJk65w6q7KnPSRYvNg --- corpus/skills/cat-mode/SKILL.md | 4 ++-- corpus/skills/cat-mode/references/verify.md | 6 +++--- corpus/skills/principle-prove-it/SKILL.md | 4 ++-- .../principle-prove-it/tests/fires_example.md | 2 +- .../tests/fires_subagent_contradiction.md | 2 +- corpus/skills/prove-it-ship-gate/SKILL.md | 4 ++-- .../prove-it-ship-gate/tests/fires_example.md | 2 +- tests/test_cat_mode.py | 16 ++++++++++++++++ 8 files changed, 28 insertions(+), 12 deletions(-) diff --git a/corpus/skills/cat-mode/SKILL.md b/corpus/skills/cat-mode/SKILL.md index 959733d6..7af3811e 100644 --- a/corpus/skills/cat-mode/SKILL.md +++ b/corpus/skills/cat-mode/SKILL.md @@ -190,7 +190,7 @@ bug: invoke `automate-me`, do not wait. workers and other work whose success is a side effect outside the repo (Linear filing, deploy, live mine) are not "done" on fixture, unit, or UI proof alone. Show live-path evidence in the same turn (ticket URL, - deployed host, observed mine hit) or write `UNVERIFIED: live path` in + deployed host, observed mine hit) or tag the claim and name the blocker in the same breath as any done/ship claim. Follow `prove-it-ship-gate` (and installed `prove-it`) on every such claim, not only when the user says "prove" or asks to investigate — a done/ship/it-works claim for @@ -256,7 +256,7 @@ agent switch, or resubmit is a fix, and none comes before the repro. **A factual or technical claim gets a real repro script, not a history search.** Judging an old comment or a "probably confabulated" suspicion needs an actual attempt under the claimed conditions, not a `git log` sweep. No citation means "never verified," not "false." -**Unhedged root-cause or fix claims about live system behavior need instrument-level proof in the same message, or `UNVERIFIED:`.** The gate is the claim type, not a hedge word. Invoking `/prove-it` once does not arm it for later claims. Any hedge auto-runs prove-it in the same turn — a hedge is a trigger to verify, never a place to stop. +**Unhedged root-cause or fix claims about live system behavior need instrument-level proof in the same message, or a `{{CAT-UNVERIFIED}}` tag naming the blocker.** The gate is the claim type, not a hedge word. Invoking `/prove-it` once does not arm it for later claims. Any hedge auto-runs prove-it in the same turn — a hedge is a trigger to verify, never a place to stop. Outputs carry failures explicitly (a status column, an error row), never dropped — [[principle-explicit-errors]]. diff --git a/corpus/skills/cat-mode/references/verify.md b/corpus/skills/cat-mode/references/verify.md index f26d937a..8f384542 100644 --- a/corpus/skills/cat-mode/references/verify.md +++ b/corpus/skills/cat-mode/references/verify.md @@ -30,7 +30,7 @@ place immediately, not left in chat until asked again. How long something was broken, how many passes found it, who wrote it, whether it ever ran — each is one `git log` and none is answerable from memory or from a file's mtime. State the command's output beside the claim, - or write `UNVERIFIED:` before it. These are the cheapest facts available and + or tag it `{{CAT-UNVERIFIED: -- cannot verify: }}`. These are the cheapest facts available and the easiest to be confidently wrong about, which is why they reach PR bodies. ## Confirming a write, and keeping its output @@ -67,11 +67,11 @@ place immediately, not left in chat until asked again. ## Unhedged causal claims about live system behavior Unhedged root-cause or fix claims about live system behavior need -instrument-level proof in the same message, or `UNVERIFIED:`. The gate is the +instrument-level proof in the same message, or a `{{CAT-UNVERIFIED}}` tag naming the blocker. The gate is the claim type ("this is why it's slow," "this is the bug"), not a hedge word. Log-reading and code-reading aren't enough: attach with `strace`/a debugger, or query live state (raw SQLite `PRAGMA`). Take a second sample before calling a hang. Invoking `/prove-it` once does not arm it for later claims — each new causal claim needs its own same-message evidence. Any hedge — "I think," -"probably," `UNVERIFIED:` — auto-runs prove-it in the same turn; a hedge is a +"probably," a retired bare `UNVERIFIED:` — auto-runs prove-it in the same turn; a hedge is a trigger to verify, never a place to stop. diff --git a/corpus/skills/principle-prove-it/SKILL.md b/corpus/skills/principle-prove-it/SKILL.md index d5b937aa..237048ff 100644 --- a/corpus/skills/principle-prove-it/SKILL.md +++ b/corpus/skills/principle-prove-it/SKILL.md @@ -24,7 +24,7 @@ same message, one of: 2. A `file:line` read this turn, named with the ref it was read at. 3. A test name plus its real pass/fail line from the runner. -Otherwise write `UNVERIFIED:` immediately before the claim. There is no +Otherwise tag the claim `{{CAT-UNVERIFIED: -- cannot verify: }}`, naming what blocks the check. There is no softer wording. The full evidence rules live in `engine/CLAUDE.core.md` and are always loaded; this file is the judgment half plus the routing below. @@ -37,7 +37,7 @@ nothing needs its exit code shown. **Blaming a gate is a causal claim.** "The hook is wrong," "the check misfired," "the classifier blocked it for no reason" — each one needs the gate's rule read this turn and quoted, with its `file:line`, next to the -input it judged. Without that, write `UNVERIFIED:` like any other cause. +input it judged. Without that, tag it like any other unchecked cause. A reversal ("actually it doesn't block that") is a new claim needing the same evidence as the first. This is Chesterton's fence: do not take a fence down until you know why it was put up (G.K. Chesterton, *The Thing*, 1929, diff --git a/corpus/skills/principle-prove-it/tests/fires_example.md b/corpus/skills/principle-prove-it/tests/fires_example.md index 591e60be..415ca886 100644 --- a/corpus/skills/principle-prove-it/tests/fires_example.md +++ b/corpus/skills/principle-prove-it/tests/fires_example.md @@ -19,7 +19,7 @@ hook and the reply says "the hook is wrong, it misfired on a read-only command." Nothing was read this turn: not the hook's detector, not the condition it tests, not the input it judged. This skill fires on that claim shape too, because blaming a gate is a causal claim like any other — it needs -the rule quoted with its `file:line` next to the input, or `UNVERIFIED:`. The +the rule quoted with its `file:line` next to the input, or a `{{CAT-UNVERIFIED}}` tag naming the blocker. The reversal that usually follows ("actually it doesn't block that") is a second claim needing its own evidence, not a correction that inherits the first one's. Chesterton's fence names the failure: the fence came down before diff --git a/corpus/skills/principle-prove-it/tests/fires_subagent_contradiction.md b/corpus/skills/principle-prove-it/tests/fires_subagent_contradiction.md index 62bb145a..5cefef0c 100644 --- a/corpus/skills/principle-prove-it/tests/fires_subagent_contradiction.md +++ b/corpus/skills/principle-prove-it/tests/fires_subagent_contradiction.md @@ -13,5 +13,5 @@ carries `grounding: read-confirmed` and a real command output or a disagreement the parent cannot act on. Correct handling is to verify before re-scoping, or to prefix the re-scope -with `UNVERIFIED:` — not to treat one delegate's unevidenced assertion as +with `{{CAT-UNVERIFIED: ... -- cannot verify: }}` — not to treat one delegate's unevidenced assertion as having moved the investigation. diff --git a/corpus/skills/prove-it-ship-gate/SKILL.md b/corpus/skills/prove-it-ship-gate/SKILL.md index 195c3674..3a8ed872 100644 --- a/corpus/skills/prove-it-ship-gate/SKILL.md +++ b/corpus/skills/prove-it-ship-gate/SKILL.md @@ -22,7 +22,7 @@ When the work under claim has live side effects: - **Fixture ≠ live.** Unit tests, mocked fixtures, and UI registration Visual Proof do not prove the live path ran. - Before stating done / shipped / working: show **live evidence in the same - turn**, or prefix the claim with `UNVERIFIED: live path`. + turn**, or tag the claim `{{CAT-UNVERIFIED: -- cannot verify: }}`. - Do not frame UI Visual Proof (for example worker registration in a settings panel) as product e2e of the live side effect. @@ -31,7 +31,7 @@ When the work under claim has live side effects: The same-turn check is a Stop hook, `engine/hooks/prove-it-ship-gate/` (installed by `install.sh`). It blocks the turn when a done/shipped/live claim sits near a live-side-effect noun with no chaseable evidence, no live command this turn, -and no `UNVERIFIED: live path` prefix. This file keeps the judgment half: deciding +and no well-formed `{{CAT-UNVERIFIED}}` tag. This file keeps the judgment half: deciding whether the work really has live side effects. ## Incident diff --git a/corpus/skills/prove-it-ship-gate/tests/fires_example.md b/corpus/skills/prove-it-ship-gate/tests/fires_example.md index 905a497b..48844647 100644 --- a/corpus/skills/prove-it-ship-gate/tests/fires_example.md +++ b/corpus/skills/prove-it-ship-gate/tests/fires_example.md @@ -5,7 +5,7 @@ This should fire: the claim is "done/shipped" and the work has a live side effect (a real Linear ticket write). Unit tests and UI registration are exactly the "fixture ≠ live" case this skill exists to block — the agent must show live evidence in the same turn or say -`UNVERIFIED: live path`. +`{{CAT-UNVERIFIED: the live path -- cannot verify: }}`. This case is also caught mechanically: `engine/hooks/prove-it-ship-gate/` blocks the turn (exit 2) on this exact message shape. See that hook's tests for the diff --git a/tests/test_cat_mode.py b/tests/test_cat_mode.py index 4cf8d895..72bc17d1 100644 --- a/tests/test_cat_mode.py +++ b/tests/test_cat_mode.py @@ -376,6 +376,22 @@ def test_routing_covers_unavailable_small_and_durable_cases(self): self.assertIn("database reads", text.lower()) +class TestEscapeHatchVocabulary(unittest.TestCase): + """cat-mode is always loaded, so an instruction here to write the retired + bare `UNVERIFIED:` would send every session into a block.""" + + def test_skill_and_verify_reference_name_the_tag_not_the_retired_marker(self): + import sys + sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + import escape_hatch_vocab as vocab + verify_ref = os.path.join(os.path.dirname(SKILL_PATH), "references", "verify.md") + for path in (SKILL_PATH, verify_ref): + with open(path, encoding="utf-8") as handle: + text = handle.read() + self.assertEqual(vocab.instructs_retired_marker(text), [], path) + self.assertIn("CAT-UNVERIFIED", text, path) + + if __name__ == "__main__": unittest.main()