Skip to content

docs: add frontmatter-check tool and wire it into CI - #734

Open
Iheanacho-ai wants to merge 1 commit into
siderolabs:mainfrom
Iheanacho-ai:frontmatter-check
Open

docs: add frontmatter-check tool and wire it into CI#734
Iheanacho-ai wants to merge 1 commit into
siderolabs:mainfrom
Iheanacho-ai:frontmatter-check

Conversation

@Iheanacho-ai

@Iheanacho-ai Iheanacho-ai commented Sep 2, 2026

Copy link
Copy Markdown
Member

What

Adds a frontmatter-check tool that verifies pages carry the frontmatter fields their section requires, and wires a changed-files variant into CI.

Why

Talos pages need a canonical URL and all pages need title/description, but nothing enforced this — pages could ship missing them silently.

Change

  • New Go tool tools/frontmatter-check: checks title+description+canonical on Talos pages, title+description on Omni and Kubernetes guides pages.
  • Two Makefile targets: check-frontmatter (full scan, for local audits) and check-frontmatter-changed (diffs against a base ref, mirrors style-check-changed).
  • docs-ci.yaml now runs check-frontmatter-changed as a blocking gate

Testing

  • Ran: make check-frontmatter (full scan, confirms 159 pre-existing issues, none new), make check-frontmatter-changed (confirmed it flags a scratch file with missing fields and passes with none changed), gofmt -l / go vet on the new tool
  • Where: local checkout
  • By: me and Claude
  • Result: all as expected

@github-project-automation github-project-automation Bot moved this to To Do in Planning Sep 2, 2026
@talos-bot talos-bot moved this from To Do to In Review in Planning Sep 2, 2026
@smira smira removed this from Planning Sep 3, 2026
@sterlingkoch

Copy link
Copy Markdown
Member

Review pass. The tool itself is good. The CI wiring is the problem, and it's the specific failure mode this repo has been bitten by before: a check made blocking while the tree is still dirty.

1. Blocking: this gate will fail PRs that did nothing wrong, and I reproduced it. .github/workflows/docs-ci.yaml:36-37

The changed-files variant narrows which files get checked, but it checks each file's entire frontmatter, not the lines the PR touched. So any PR that edits one of the already-broken pages for an unrelated reason fails on a field the author never went near. Appending one comment line to public/talos/v1.14/learn-more/enterprise-image-factory.mdx on top of current main and running the exact CI command:

Checking changed files: public/talos/v1.14/learn-more/enterprise-image-factory.mdx
../../public/talos/v1.14/learn-more/enterprise-image-factory.mdx: missing "canonical"

Checked 1 file(s): 1 issue(s)
make: *** [check-frontmatter-changed] Error 1

How often that bites: 15 of the last 57 commits on main that touched any .mdx touched at least one of the 158 broken pages, so roughly one in four recent changes. Add continue-on-error: true until the tree is clean, or land the backfill first.

2. The step immediately below yours is non-blocking for the same reason. .github/workflows/docs-ci.yaml:44-46 is explicitly continue-on-error with the stated reason "large legacy backlog." Frontmatter has the same backlog and gets the opposite treatment. Whichever way we go, these two should agree.

3. The README claim doesn't match the behavior, and it's the load-bearing one. tools/frontmatter-check/README.md:41-45 says "without punishing untouched legacy content." True for untouched pages, not true for any page a PR does touch. That sentence is the argument for making it blocking, so it's worth correcting.

4. The regeneration loop will keep reopening this. 91 of the 158 broken pages are under public/talos/*/reference/configuration/, generated by make generate-talos-reference. Backfilling once doesn't hold, because the generator doesn't emit canonical, so the next regeneration re-emits them bare and the gate blocks that PR. #733's canonical-links target is the durable fix, but it needs wiring into the generate flow rather than existing only as a manual target.

5. Paths filter is missing the new tool. .github/workflows/docs-ci.yaml:5-13 lists tools/docs-validate/** but not tools/frontmatter-check/**, so a PR that only changes this tool won't run docs CI.

6. Unclassified paths are skipped with no output at all. tools/frontmatter-check/main.go:78-81. On #741 the target listed 13 files and printed "Checked 11" with nothing naming the two dropped (public/changelog.mdx and public/snippets/custom-variables.mdx). When hypervisor docs land in a new tree they'll be silently green. A one-line "skipped N unclassified file(s)" would cover it, or error on an unknown public/<dir>/.

7. public/omni/_ci-test.mdx is the one file no backfill covers. Zero bytes and not in the nav, so a full scan can never reach 0 while it exists. Delete it or give it frontmatter.

Minor. main.go:179-183: only string values land in the map, so title: 2026 reports as missing "description"-style wrong. Same at :170-172, where an unterminated frontmatter block reports all three fields missing rather than "unterminated frontmatter." And go-version: "1.22" at docs-ci.yaml:28 against go 1.25.1 in the new go.mod means every run downloads a second toolchain; sync-omni-config-reference.yaml already uses go-version-file:.

Numbers, in case they're useful. Current main is 159 issues over 158 files (1706 files scanned: 114 missing canonical, 44 description, 1 title). #735's head brings that to 2. #733's head brings it to 45. Their file sets together cover 157 of the 158.

Good work worth noting. The parser is solid: I threw ten edge cases at it (no frontmatter, description: "", empty block scalar, bare canonical:, CRLF, a --- rule later in the body, a --- inside a code fence, unterminated block, numeric value, list value) and every pass/fail decision was correct. Only two produce a misleading message. Makefile:394-410 mirrors style-check-changed line for line, and using go run with no container variant correctly matches validate-docs-nav, which is the closer sibling. Fork PRs are fine too, which I checked rather than assumed: fetch-depth: 0 is already set at docs-ci.yaml:23, and the identical base-ref mechanism in style-check-changed correctly enumerated changed files on #736, #738, and #741, all of which are fork PRs.

@sterlingkoch sterlingkoch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See inline comments (most are agent-to-agent comments).

Adds a Go tool that verifies required frontmatter fields per section
(title/description/canonical for Talos, title/description for Omni and
Kubernetes guides), plus a changed-files variant used as the CI gate so
the large legacy backlog doesn't block unrelated PRs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Amarachi Iheanacho <amarachi.iheanacho@siderolabs.com>
@Iheanacho-ai

Copy link
Copy Markdown
Member Author

@sterlingkoch 1,2, and 4 are fixed by these prs: https://github.com/siderolabs/docs/pull/735/changes and #733

but i fixed the others

@sterlingkoch sterlingkoch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the code, but don't merge this until #733 and #735 are both in. The step at .github/workflows/docs-ci.yaml:43-44 is still blocking with no continue-on-error, so landing it before the backfills would fail PRs that did nothing wrong. Measured with this PR's own tool:

  • main today: 159 issues over 1706 files
  • main + #733: 45
  • main + #733 + #735: 2, and both are public/omni/_ci-test.mdx, which this PR deletes

So the tree hits 0 exactly when this one goes in last. Order is #733, then #735, then this.

The code changes are all good: README wording at tools/frontmatter-check/README.md:71 is accurate now, tools/frontmatter-check/** is in the paths filter, classUnknown plus the "Skipped N exempt file(s)" line closes the silent-skip gap, _ci-test.mdx is gone, and go-version-file kills the second toolchain download.

Two things about #735 that gate it, both new since yesterday. Eleven files have raw conflict markers committed at lines 3, 5 and 8 (the nine learn-more pages and four opennebula.mdx). And #733 and #735 now conflict with each other on 62 files, which wasn't true when I looked yesterday: #733 writes the v1.14 canonical, #735 writes v1.13, same line. Every one resolves by taking #733's side, which is another argument for dropping the canonical half of #735 and keeping just the descriptions.

Worth knowing as evidence for this PR: docs-checks is currently green on #735 with those 11 broken-YAML files in it. Nothing in CI parses frontmatter today and the style check is continue-on-error, so invalid frontmatter merges silently. That's the gap this closes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants