Bugfix: a fenced block hides every other sentence in its paragraph - #479
Merged
EdbertChan merged 4 commits intoSep 12, 2026
Conversation
… — Examine the prose surviving after fenced blocks are removed, rather than passing over the whole paragraph. Review claim: A paragraph holding a fenced block still has its non-fenced prose examined, and a sentence that earned its silence honestly keeps it. Review lane: behavior Safety invariant: Every message the detector is silent on today for a reason other than the fenced-paragraph branch stays silent; the existing inline-code and file:line clearing keeps working unchanged. Effectiveness measurement: `python3 engine/hooks/diu-stop/tests/test_hooks.py` exits 0, with a new positive case that exits non-zero before this change, plus the negative case described below that must stay silent both before and after. Slice rationale: One conceptual unit: which text inside a paragraph the detector looks at. Architectural effect: None; no new module boundaries. The paragraph loop gains a normalization ahead of the existing per-paragraph branches. Goal: Stop a fenced block from buying silence for unrelated sentences sharing its paragraph. Motivation: A false remedy claim reached an operator because it sat in the same paragraph as an unrelated fence. The operator acted on it and went looking for a package upgrade that does not exist. Alternative considerations: Removing the fenced-paragraph branch outright was set aside -- it would surface the fence body itself as prose and fire on code. Splitting paragraphs on fence boundaries was set aside as more disruptive to the surrounding offsets than removing fenced spans in place. Implementation details: Before the existing per-paragraph branches, remove fenced spans from the paragraph text and carry on with what remains. When nothing but whitespace remains, behave exactly as the current branch does. Non-goals: No vocabulary changes to the remedy or causal patterns -- that is a separate slice depending on PR 371. No change to first-hit-versus-every-hit behavior -- that is PR 433. No change to the stop_hook_active early return -- that is PR 416. Layer: domain Feature state: active Files: engine/hooks/diu-stop/claude_stop_check.py, engine/hooks/diu-stop/tests/test_hooks.py Change types: - engine/hooks/diu-stop/claude_stop_check.py: modify - engine/hooks/diu-stop/tests/test_hooks.py: modify Acceptance criteria: - `python3 engine/hooks/diu-stop/tests/test_hooks.py` exits 0 after the change, with all 73 existing cases still passing. - A paragraph of the shape "prose, fenced block, unproven remedy sentence" is flagged. - A paragraph whose claim sits beside a genuine file:line citation stays silent. Solution: Examine the prose surviving after fenced blocks are removed, rather than passing over the whole paragraph. Review claim: A paragraph holding a fenced block still has its non-fenced prose examined, and a sentence that earned its silence honestly keeps it. Review lane: behavior Safety invariant: Every message the detector is silent on today for a reason other than the fenced-paragraph branch stays silent; the existing inline-code and file:line clearing keeps working unchanged. Effectiveness measurement: `python3 engine/hooks/diu-stop/tests/test_hooks.py` exits 0, with a new positive case that exits non-zero before this change, plus the negative case described below that must stay silent both before and after. Slice rationale: One conceptual unit: which text inside a paragraph the detector looks at. Architectural effect: None; no new module boundaries. The paragraph loop gains a normalization ahead of the existing per-paragraph branches. Goal: Stop a fenced block from buying silence for unrelated sentences sharing its paragraph. Motivation: A false remedy claim reached an operator because it sat in the same paragraph as an unrelated fence. The operator acted on it and went looking for a package upgrade that does not exist. Alternative considerations: Removing the fenced-paragraph branch outright was set aside -- it would surface the fence body itself as prose and fire on code. Splitting paragraphs on fence boundaries was set aside as more disruptive to the surrounding offsets than removing fenced spans in place. Implementation details: Before the existing per-paragraph branches, remove fenced spans from the paragraph text and carry on with what remains. When nothing but whitespace remains, behave exactly as the current branch does. Non-goals: No vocabulary changes to the remedy or causal patterns -- that is a separate slice depending on PR 371. No change to first-hit-versus-every-hit behavior -- that is PR 433. No change to the stop_hook_active early return -- that is PR 416. Layer: domain Feature state: active Files: engine/hooks/diu-stop/claude_stop_check.py, engine/hooks/diu-stop/tests/test_hooks.py Change types: - engine/hooks/diu-stop/claude_stop_check.py: modify - engine/hooks/diu-stop/tests/test_hooks.py: modify Acceptance criteria: - `python3 engine/hooks/diu-stop/tests/test_hooks.py` exits 0 after the change, with all 73 existing cases still passing. - A paragraph of the shape "prose, fenced block, unproven remedy sentence" is flagged. - A paragraph whose claim sits beside a genuine file:line citation stays silent. Invoker-Finalize-Id: 7fdeca09-72d7-4d0a-8d68-4cd443e73cac
…urs — Run the hook's own suite as the shared repro for this slice. Review claim: The suite exits 0 only when fenced paragraphs have their surviving prose examined and honest citations still buy silence. Review lane: proof Safety invariant: The repro is identical before and after the change; the new positive case exits non-zero before and 0 after. Effectiveness measurement: The exit status of the suite is the effectiveness signal for this slice. Slice rationale: One conceptual unit: the deterministic repro only. Architectural effect: None. Goal: Deterministically prove the change. Motivation: This branch shipped with no case covering a mixed prose-and-fence paragraph, which is why it survived long enough to mislead an operator. Alternative considerations: Adding only the positive case was set aside -- without the citation case, a change that floods honest messages would still exit 0. Implementation details: Execute the suite as the terminal gate. Non-goals: No product edits here. Layer: e2e_regression Feature state: active Exit code: 0 Invoker-Finalize-Id: 88db3385-a0a6-46b0-8d2d-8091c0cac657
…nly gate confirming no ephemeral handoff files were left behind. Review claim: The workflow leaves no ephemeral handoff files in the tree. Review lane: proof Safety invariant: Read-only -- never deletes files, alters the index, or commits caller work. Effectiveness measurement: A non-zero exit when ephemeral handoff files remain is the signal. Slice rationale: One conceptual unit: the hygiene gate. Architectural effect: None. Goal: Confirm no ephemeral handoff files remain after every other task finishes. Motivation: Ephemeral inter-task files leak into the diff and read as part of the change. Alternative considerations: Manual inspection was set aside as non-deterministic. Implementation details: Run scripts/scrub-handoff-artifacts.sh without --apply. Non-goals: No deletion, no index changes, no commits. Layer: e2e_regression Feature state: active Exit code: 0 Invoker-Finalize-Id: 192f2887-89ae-4cfe-aa67-267352f8286b
…69673175-4f5586e3 — Terminal read-only gate confirming no ephemeral handoff files were left behind. Review claim: The workflow leaves no ephemeral handoff files in the tree. Review lane: proof Safety invariant: Read-only -- never deletes files, alters the index, or commits caller work. Effectiveness measurement: A non-zero exit when ephemeral handoff files remain is the signal. Slice rationale: One conceptual unit: the hygiene gate. Architectural effect: None. Goal: Confirm no ephemeral handoff files remain after every other task finishes. Motivation: Ephemeral inter-task files leak into the diff and read as part of the change. Alternative considerations: Manual inspection was set aside as non-deterministic. Implementation details: Run scripts/scrub-handoff-artifacts.sh without --apply. Non-goals: No deletion, no index changes, no commits. Layer: e2e_regression Feature state: active
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_471a617d-b948-4bc2-b186-e211d098aa0b) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The reply checker reviews assistant messages for unsupported claims before they are sent.
A fenced code block can share a paragraph with ordinary prose.
Previously, that block caused the whole paragraph to be skipped, allowing unrelated false claims through.
Now fenced spans are removed before the remaining prose is checked, while empty blocks and honest citations retain their existing behavior.
Review Claim
A fenced block no longer silences unrelated prose in the same paragraph, while genuine file-and-line citations still silence the claim they support.
Review Lane
behavior
Review Unit
engine-runtime
Safety Invariant
Every message the detector is silent on today for a reason other than the fenced-paragraph branch stays silent; the existing inline-code and file:line clearing keeps working unchanged.
Slice Rationale
This slice changes only which text inside a paragraph is examined. Vocabulary expansion and reporting every matching claim remain separate work.
Non-goals
Test Plan
Test Plan
python3 engine/skills/make-pr/scripts/preflight.py --base origin/main— passed; hook coverage passed.python3 engine/hooks/diu-stop/tests/test_hooks.py—Ran 75 tests;OK.python3 scripts/check_no_new_comments.py --base origin/main—ok no new comments.bash scripts/scrub-handoff-artifacts.sh—scrub-handoff-artifacts-ok.Revert Plan
Revert Plan
git revert <sha>.Note
Low Risk
Narrow change to paragraph parsing in the Stop hook’s claim heuristic; tests lock in stricter blocking and preserved file-line citation behavior.
Overview
Fixes a bypass in the Claude Stop hook’s unverified-claim detector: a fenced code block in the same paragraph as normal prose no longer causes the entire paragraph to be skipped.
find_unverified_claimnow strips closed fenced blocks from each paragraph before scanning the remaining text, so phrases like "this fixes it" after unrelated fenced content are still flagged. Paragraphs that are only a fence (or empty after stripping) are skipped as before; unclosed fence remnants still short-circuit via the existing```check.Adds
FILE_LINE_REsopath/file.ext:line(or#Lstyle) citations in the prose left after fence removal still count as evidence-shaped and silence the paragraph—matching prior behavior when a citation followed a fence in one block.New unit tests cover the regression (fence + bogus claim → blocked) and the citation case (fence +
file.ts:1276→ allowed).Reviewed by Cursor Bugbot for commit 23fd5a5. Bugbot is set up for automated code reviews on this repo. Configure here.