chore: replace pa11y-ci with axe incomplete-result warnings in Playwright - #858
lukasmatta wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The warning command currently masks failed Playwright accessibility tests and exits successfully.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Replaces redundant pa11y-ci checks with Playwright axe incomplete-result annotations and warning reporting.
Changes:
- Removes pa11y-ci configuration, dependency, CI job, and documentation.
- Annotates axe incomplete results, while enforcing none for composition tests.
- Adds a CLI utility and npm script for listing warnings.
File summaries
| File | Description |
|---|---|
README.md |
Documents the consolidated accessibility workflow. |
playwright/README.md |
Removes pa11y-ci references. |
playwright/print-a11y-warnings.js |
Prints warning annotations from JSON reports. |
playwright/fixtures/axe-helpers.ts |
Adds incomplete-result annotations. |
playwright/cps-ui-kit/accessibility-cps-ui-kit.spec.ts |
Annotates component scan incompletes. |
playwright/composition/accessibility-composition.spec.ts |
Annotates and rejects composition incompletes. |
package.json |
Removes pa11y and adds the warning command. |
package-lock.json |
Removes pa11y dependency tree. |
.pa11yci |
Deletes obsolete scan configuration. |
.gitignore |
Ignores the new temporary report. |
.github/workflows/cps-shared-ui-checkers.yml |
Removes the pa11y CI job. |
Review details
- Files reviewed: 9/11 changed files
- Comments generated: 3
- Review effort level: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
🟡 Changes recommended
The advertised command name and CI artifact documentation remain inconsistent with the implementation.
Get a fresh assessment by requesting another Copilot review.
Review details
- Files reviewed: 9/11 changed files
- Comments generated: 2
- Review effort level: Balanced
Playwright test results
Details
|
Coverage report for library
Test suite run success3233 tests passing in 76 suites. Report generated by 🧪jest coverage report action from d0f501e |
pa11y-ci and our Playwright+axe suite were redundant as both use axe-core. The only difference was how we used them: pa11y-ci failed on axe's "incomplete" results (things it couldn't fully confirm, e.g. combobox aria-controls, occluded color-contrast), while Playwright's
expectNoViolationssilently dropped them.Changes:
annotateIncomplete()toaxe-helpers.ts, wired into both accessibility specs