Phrase judge (4) wrong-check-reflect judges meaning instead of matching phrasings

[EdbertChan]

Summary

Replies that admit an earlier mistake now reach a meaning-based judge instead of matching hand-written word patterns.

The hook remains fail-open: replies continue immediately, while verdicts arrive through the existing judge inbox.

The dictionary and README document the judged decision, and a live check covers the three recorded misfires.

Review Claim

wrong-check-reflect decides admission by asking the background judge with a phrase dictionary, with the old prose-pattern machinery removed.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

The hook never blocks a reply. Unchecked judge results say “could not judge,” and judge or state errors leave the reply untouched.

Slice Rationale

This slice contains one behavior claim across the hook, its judge dictionary, runner adapters, tests, and effectiveness check; the README documents that same decision.

Non-goals

• No other hook changes.
• No installer, settings, or install.sh changes.
• The live effectiveness check is not wired into CI because CI has no model access.

Architecture

Before

graph TD
    A["Stop reply"] --> B["hand-written phrasing patterns"]
    B --> C["local admission decision"]

Loading

After

graph TD
    A["Stop reply"] --> B["phrase dictionary + background judge"]
    B --> C["llm-judge inbox"]
    C --> D["later runner turn"]

Loading

Test Plan

Test Plan

• python3 -m unittest discover -s engine/hooks/wrong-check-reflect/tests -v && python3 -m unittest discover -s engine/hooks/llm-judge/tests -v — 16 and 59 tests passed.
• test -z "$(git grep -n -e ADMISSION_RES -e NEGATIVE_RES -e WRONGNESS_RE -e structural_admission -e find_admission -- engine/hooks/wrong-check-reflect)" — exit 0.
• python3 engine/hooks/wrong-check-reflect/eval_dictionary.py — the correction matched; both harmless replies did not.
• python3 scripts/check_no_new_comments.py --base origin/main — exit 0.
• python3 scripts/check_hook_test_coverage.py engine/hooks/wrong-check-reflect — passed.
• python3 engine/skills/make-pr/scripts/preflight.py --base origin/main — passed; review unit is engine-runtime.
• bash scripts/scrub-handoff-artifacts.sh — exit 0.

Revert Plan

Revert Plan

• Safe to revert? Yes.
• Revert command: git revert <merge-commit-sha>.
• Post-revert steps: Rerun the focused hook and llm-judge test suites.
• Data migration? No.

---

Note

Medium Risk
Detection is fully asynchronous and model-dependent, so reflect follow-ups may be missed or delayed compared to the old synchronous regex path, though the hook still fail-opens and never blocks replies.

Overview
wrong-check-reflect no longer decides admissions with local regex or structural matchers. On every Stop it always enqueues a background llm-judge job built from the new wrong-check-reflect.json phrase dictionary (meaning, match / not_match examples, static on_hit text). Harness adapters (Claude, Cursor, Codex) only call try_enqueue_judge — they no longer block, print advisories, or inject reflect text on the same turn; hits and “could not judge” arrive later through the shared judge inbox.

The removed pattern logic is preserved offline as self_retraction_scan.py for the reflect token_audit miner so historical transcripts can still be scanned without model calls. Scenarios and run_skill_scenarios.py gain expect_enqueue / expect_no_enqueue; eval_dictionary.py exercises the dictionary against a real judge runner.

^{Reviewed by Cursor Bugbot for commit 3ca40a7. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_44d7f04a-b81c-4ff7-ada5-4aa6e37c4401)

[mergify]

Queued — the merge queue status continues in this comment ↓.

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_f0fc783e-be18-4c0f-91e7-f68d2b33f9f3)

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_f8c1d816-f9db-41c9-9df7-17b6d7344924)

[EdbertChan]

@Mergifyio queue

[mergify]

Merge Queue Status

• ✅ Entered queue — 2026-09-12 01:48 UTC · Rule: admin-bypass · triggered by @EdbertChan with the @mergifyio queue command
• ✅ Checks passed · on draft merge queue: checking #462 on main (316ffb5) #463
• ✅ Merged — 2026-09-12 01:53 UTC · at e9a9750689d398db88b1f62a1bc9e9089587a058 · squash

This pull request spent 5 minutes 11 seconds in the queue, including 4 minutes 22 seconds running CI.

Required conditions to merge

• check-success = lint
• check-success = test