Skip to content

feat(health): add a severity tier so a check can report without failing validation - #58

Open
dev-noaman wants to merge 1 commit into
AlmanacCode:mainfrom
dev-noaman:feat/validation-issue-severity
Open

feat(health): add a severity tier so a check can report without failing validation#58
dev-noaman wants to merge 1 commit into
AlmanacCode:mainfrom
dev-noaman:feat/validation-issue-severity

Conversation

@dev-noaman

Copy link
Copy Markdown

Adds a severity tier to ValidationIssue so a check can report a condition without failing validation. Follow-up to #56/#57, but independent of them — this branch is cut from main and the two can merge in either order.

Why

ValidationResult.ok is currently:

ok=len(issues) == 0

so every check is necessarily fatal. There is no way to surface something a maintainer should see without also failing validate, ensure_valid, and everything gated on them — sync included.

That came up concretely in #57. I offered to add a health notice for pages skipped because they sit under a reserved directory, then withdrew it: under the current contract, an informational issue firing on the bundled manual/ — present in every wiki — would flip validate non-zero and fail sync everywhere. The diagnostic would have caused a worse bug than the one it explained.

The missing piece is not that particular check. It is the tier.

What changes

class IssueSeverity(StrEnum):
    ERROR = "error"
    WARNING = "warning"

class ValidationIssue(CodeAlmanacModel):
    ...
    severity: IssueSeverity = IssueSeverity.ERROR
  • ValidationResult.ok becomes "no error-severity issues".
  • ValidationResult.errors / .warnings expose the two views.
  • validation_failure_message reports the first error, so a preceding warning cannot become the headline of a failure it did not cause.
  • render_validate prints warnings in their own section, including when validation passes. A warning nobody sees is not a warning.

Backward compatibility

This is the part I most want reviewed.

All 13 existing ValidationIssue construction sites omit severity, so they default to ERROR and behave exactly as before. No current check changes classification, no exit code moves, and the JSON gains a field rather than changing one.

No existing category is reclassified. Whether unused_sources, empty_topics or orphans are really warnings is a policy decision that belongs to maintainers, so this PR adds the capability without exercising it. Happy to follow up with a reclassification if you have views.

Behaviour

warning only  -> ok=True   exit=0  warnings=1 errors=0
warning+error -> ok=False  exit=1  msg=validation failed: sources: source is missing a target
clean         -> ok=True
json carries severity: True

Note the middle line: the message names the error, not the warning that precedes it.

Tests

Four tests in tests/test_validate.py covering the default severity, a warning leaving ok true, an error still failing, and warnings not masking errors in the failure message.

Checks

Per CONTRIBUTING.md:

  • uv run pytest — failing set identical before and after (14 both ways on this machine; all pre-existing and unrelated — 5 are macOS launchd tests on Windows, one is a sqlite repositories.name isolation issue). tests/test_validate.py goes 6 → 10 passed.
  • uv run ruff check . — All checks passed
  • uv build — sdist and wheel built

If you'd rather not

This is a change to a public contract, so it is reasonable to want it shaped differently — a separate notices collection instead of a field, say, or severity confined to the health report rather than ValidationIssue. I'm happy to rework or drop it; the motivating case is only that a diagnostic should be able to inform without blocking.

…ng validation

ValidationResult.ok was `len(issues) == 0`, so every check was necessarily
fatal. That left no way to surface a condition a maintainer should see without
also failing `validate`, `ensure_valid`, and the workflows gated on them --
`sync` among them. Any informational check added under the old contract would
break every wiki it fired on.

ValidationIssue gains `severity: IssueSeverity`, defaulting to ERROR, and
ValidationResult gains `errors`/`warnings` views. `ok` now means "no
error-severity issues".

Behaviour is unchanged for everything that exists today: all 13 construction
sites omit severity and therefore stay ERROR, so no current check changes
classification and no exit code moves. This adds the capability without
exercising it -- deciding whether any existing category (unused_sources,
empty_topics, orphans) is really a warning is a policy call for maintainers,
deliberately left alone here.

Two details worth flagging:

- `validation_failure_message` now describes the first ERROR rather than the
  first issue, so a preceding warning cannot become the headline of a failure
  it did not cause.
- `render_validate` prints warnings in their own section and still prints them
  when validation passes. A warning nobody sees is not a warning.

Tests cover the default, that a warning leaves ok true, that an error still
fails, and that warnings do not mask errors in the failure message.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants