pr-schema-gate: an exit-0 validator that printed UNCHECKED is not a clean run

[EdbertChan]

Summary

The PR-text gate now speaks up when a validator exits 0 without actually checking anything.

Tonight a PR description went to GitHub after the validator printed UNCHECKED: PR body rules not checked (drafter-core not installed) and exited 0. The gate saw exit 0 and said nothing, so an unchecked body read as a checked one.

The gate already had an unchecked outcome for this exact situation; the exit-0 branch just never looked at the output it had already collected. It does now, and a matching rule says the same thing for any tool that can answer from a degraded source.

Review Claim

An exit-0 validator run that prints an unchecked or skipped marker is reported as unchecked, not clean.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

The hook stays advisory: this path routes into the existing unchecked message and blocks nothing. A genuine exit-0 pass with no marker still reports clean, pinned by the negative test.

Slice Rationale

One claim, one hook, plus the always-loaded rule it enforces. The prose and the code ship together so the rule is not documented without a check behind it.

Non-goals

• No change to the failure path or to any blocking behavior.
• No new hook; the degraded-source detector for other tools stays on the backlog.
• No change to which validator the gate runs.

Test Plan

Test Plan

• python3 -m unittest test_advisory -k exit_zero_with_an_unchecked against origin/main's detect.py — AssertionError: 'could not check' not found in '', FAILED (failures=1), exit 1
• python3 -m unittest discover -s engine/hooks/pr-schema-gate/tests -v with the fix — Ran 99 tests … OK, exit 0
• python3 engine/skills/make-pr/scripts/preflight.py --base origin/main — unit engine-runtime: 3 file(s), ok preflight passed, exit 0

Revert Plan

Revert Plan

• Safe to revert? Yes
• Revert command: git revert <sha>
• Post-revert steps: None; the gate returns to reporting clean on exit 0
• Data migration? No

🤖 Generated with Claude Code

https://claude.ai/code/session_01DtqAxtTmdoRE7CD5BXdGbp

---

Note

Low Risk
Advisory-only hook behavior fix with unit tests; no blocking paths or validator selection changes.

Overview
pr-schema-gate no longer treats a validator that exits 0 as a clean pass when its stdout/stderr says the PR body was never checked (e.g. UNCHECKED: … not installed). Those runs now use the existing unchecked advisory path with a message that the validator exited 0 without checking.

The hook scans collected validator output with VACUOUS_PASS_RE (UNCHECKED, SKIPPED, not installed, no rules loaded) before returning clean. A genuine pass with no such markers stays silent; tests cover the false pass and the negative case.

engine/CLAUDE.core.md adds a matching evidence rule: exit 0 and printed data are not enough when a tool can answer from a cache, replica, or offline mode—show stderr/exit code or name the source, and re-read after restarts.

^{Reviewed by Cursor Bugbot for commit c4e55c0. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_a8f9e190-f368-4b38-a49b-254f93bf27ad)

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_5989d203-b0d8-4155-981a-01119d777863)

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request