Agent Skills for the work after an agent writes code: requirement defects, real code review, PRD gaps, manual test cases, and the test data to run them.
English | 简体中文
Quick Start · What it looks like · How It Works · Blind Evaluation · Limitations · Getting Started · FAQ · Support Matrix
Coding agents write the change. These skills check the requirement, the review, the cases, and the testdata.
(Above is a sample page: same renderer as a local run, canned findings.)
For Cursor, Claude Code, Codex, and OpenClaw. Install with npx skills add (Agent Skills). Local-first; no codexqa account.
Coding agents make a green pull request cheap. The expensive part is now requirement bugs, rubber-stamp reviews, and test work the model cannot finish alone:
| What goes wrong after the agent writes code | Skill |
|---|---|
The PR looks right and CI is green; the spec said “discount at 10 items” and the code used > |
defect-detection — business-logic / requirement defects, not only Semgrep shape |
| You need AI code review of a local branch or PR: file:line, runtime impact, a fix — not “LGTM” | code-reviewer |
| The PRD contradicts the API note, or a P0 has no way to fail | requirements-analyzer |
| QA still writes the manual test-case library from the PRD by hand | testcase-generation |
Cases are full of {placeholder} and nobody created the real IDs on a backend |
testdata-generation |
We have not measured and published a score per host — what was actually checked is in the support matrix. Findings are candidates for a human.
codexqa is a public, local-first Agent Skills pack. Five skills; they do not share one input:
| Skill | You bring | It does |
|---|---|---|
defect-detection |
Git URL + branch (plus requirements or cases) | Clone, analyze changed methods, write gated findings |
code-reviewer |
A local Git checkout + branch / PR / commit | Playbook CR with a P0 / P1 / P2 report; does not clone |
requirements-analyzer |
PRD / stories / API notes (documents) | One gap/conflict register with P0 / P1 verification |
testcase-generation |
PRD / design / specs under prd/ |
Write and update a manual case library. code/ is update-only |
testdata-generation |
A construct request, cases, and/or OpenAPI | Call a backend (or local mock) and return real IDs; not a git clone |
defect-detection workflow:
- Task creation, repository cloning, branch and diff context collection
- Changed-method analysis with AST rules and optional Java call-graph analysis
- Local JSON providers plus optional HTTP, GitHub, test-case, document, issue, and trace adapters
- Structured finding validation, write-back, ranking, tagging, and HTML reports
- A deterministic known-good/seeded-defect fixture and an automated CLI test suite
Method write-ups live in each skill: defect-detection, code-reviewer, requirements-analyzer, testcase-generation, testdata-generation. Index: docs/HOW_IT_WORKS.md.
Static rules catch bugs you can recognise by shape: a swallowed exception, a hardcoded secret, a missing null check. The bugs that survive review are usually different: the code looks fine, the tests are green, and it still does not match the requirement.
- Spec says “discount at 10 items”; the code uses
>, so 10 items get no discount. - A refund pays the requested amount and never caps it at the remaining balance.
- One function updates a counter; the cache that reads the same data is never invalidated.
Those bugs are not in the syntax tree. They sit in the gap between code and intent, and the intent lives in requirements and test cases. A model can compare the two. Left unconstrained, it also invents methods it never read, stamps “looks good” on hundreds of methods, and files so much noise that people stop reading.
So the split is: the model judges meaning; the infrastructure makes that judgement checkable.
Repo + branch + requirements or cases
│
▼
Collect context, analyse the change
│
▼
AST rules + optional call graph
│
▼
Agent review, findings validated
│
▼
Structured findings + HTML report
│
▼
Human review
codexqa runs the workflow. The host Agent / model does the semantic review. Local providers need no private backend; adapters can attach the same flow to an external platform.
Three controls do the real work:
- Tiers. Methods tied to a stated requirement or case get deeper analysis; the rest are not treated equally.
- 23 write-back rules. Rejected if the code was never read, the method name is not in the source, or the batch looks like autopilot output.
- A close gate. Coverage, report consistency, and evidence depth are checked again before the task can finish.
On the inventory-service blind fixture, seven business-logic defects sit inside ordinary feature work, plus four decoys that look wrong but are correct. One recorded agent run found 7/7 with 0 false positives, and none of the seven came from the 102 Semgrep seed rules. That is one model, one run, one in-house fixture — not a benchmark. Methodology, design, known limitations.
These are all sample pages (same renderer as a local run, canned findings). Open the HTML if an image is stale.
testcase-generation writes Markdown cases. This is that file rendered: steps, expected results, empty Construction column. Open the page.
code-reviewer report: each finding has location, rule, runtime impact, and a fix. Open the page.
requirements-analyzer: one gap / conflict register with an executable check per row. Open the page.
testdata-generation fills {placeholder} with IDs the backend actually returned. Open the page.
The defect-detection report is the page at the top; you can also open it directly.
The skill instructions target Cursor, Claude Code, Codex, and OpenClaw: install, start a new session, hand over the material. Installing is not the same as a complete run on that host; per-component status is in the support matrix.
defect-detection does language-aware method extraction for the languages below, each with its own Semgrep seed pack (102 seed rules in total). The extractors are regex / brace / indentation based, not full parsers:
| Language | Method-level extraction | Extra |
|---|---|---|
| Java | ✓ | Optional GitNexus call graph |
| Kotlin · Scala | ✓ | — |
| JavaScript · TypeScript | ✓ | Where the one recorded blind-eval sample (7/7) ran |
| Python | ✓ | 17 seed rules; method bounds from indentation |
| Go | ✓ | Optional go vet / staticcheck overlays |
| C · C++ | ✓ | Macro-heavy code is approximated |
| C# | ✓ | — |
Optional overlays run when the binary is installed and are skipped when it is not: gitleaks, trivy / grype, bandit, gosec, cppcheck, eslint, detekt.
npx skills add openqa-cn/codexqa --skill defect-detection
npx skills add openqa-cn/codexqa --skill code-reviewer
npx skills add openqa-cn/codexqa --skill requirements-analyzer
npx skills add openqa-cn/codexqa --skill testcase-generation
npx skills add openqa-cn/codexqa --skill testdata-generationChoose an Agent when prompted. For a user-level Codex installation add --agent codex --global. Each --skill copies one directory.
No codexqa or npm account is required. See Getting started for runtime requirements, installation scope, and troubleshooting.
- Install the skill you need with the command above.
- Start a new Coding Agent session.
- Give the agent the material that skill expects. They are not interchangeable.
Start with one: review a branch (defect-detection).
Review REPOSITORY_URL at BRANCH_NAME with defect-detection.
Requirement: checkout amounts must be greater than zero.
For each suspected defect, report the location, trigger, evidence, and fix.
Replace the uppercase placeholders. The workflow collects context, analyzes changed methods, validates findings, and produces a report for human review.
A model-free fixture of the contract (not a detection-accuracy claim):
node examples/checkout-boundary/verify.mjsWhat to say to the other four skills
Review a local checkout (code-reviewer) — open the repository in the agent. This skill diffs in place; it does not clone.
Review the current branch with code-reviewer against main.
For each finding give severity, file:line, the rule, the runtime impact, and a fix.
There is no public fixture yet. For method-level requirement defects with write-back gates, use defect-detection instead.
Analyze requirements (requirements-analyzer) — give documents, not a repo.
Analyze these requirement documents with requirements-analyzer.
Produce one gap/conflict register. P0 items must include verification fields.
Do not invent endpoints or SLAs that are not in the source.
This reviews the PRD. To write a case library from prd/, use testcase-generation.
Generate manual test cases from a PRD (testcase-generation) — put PRD / design / specs under prd/ first. code/ is not required for generate.
Generate a manual case library with testcase-generation from the documents under prd/.
Do not invent engineering fields that are not in the source. Mark those TBD.
When the agent stops on a PRD vs design conflict, reply Confirm follow PRD or Item N follow technical design. There is no public fixture yet.
Construct test data (testdata-generation) — not a git clone. Say what to construct, or point at written cases / OpenAPI:
Create a standard catalog product named Northwind Standard with testdata-generation.
Use the local mock if no enterprise gateway is configured.
For write-back into cases:
Prepare test data for this case file and write the IDs back as preconditions.
Default backend is the bundled mock on http://127.0.0.1:8765. A demo ID is not proof that a real system was written.
What you install is files, not a service. npx skills add … --skill <name> copies one directory, and the agent you already use reads it — no account, no gateway, no workflow to migrate.
- Semantic judgement stays with the model you already pay for. This repository ships no model and does not rank them. It organizes the context, then makes the model's conclusion checkable: 23 write-back rules, tiers, a close gate.
- Local-first is literal. Local JSON providers persist to disk and the report is an HTML file you can double-click. Configure the HTTP / GitHub adapters only when you want an external system attached.
- Install one at a time. The five skills do not share an input; installing all of them mostly helps the agent pick the wrong one. Install for the task in front of you.
- Capabilities and claims are written down separately. What ships versus what is planned: capability map and roadmap. What was actually checked per component: support matrix. What it cannot do: known limitations.
Important
The current defect-detection workflow is a usable engineering tool, not an experiment-only prototype. It produces candidates for human confirmation and does not replace tests, static analysis, security review, or maintainer judgment.
This project helps organize code context and evidence; it does not replace tests, static analysis, security review, or maintainer judgment. It cannot prove the absence of defects or infer business rules that were not provided. Findings are candidates, not automatic merge decisions.
Local providers write data to disk. Repository cloning, document fetching, remote providers, automatic Semgrep/GitNexus installation, and the host Agent/model may use the network. Review FAQ, support matrix, and security policy before using private source.
Before contributing, run the repository checks:
python3 scripts/check-docs.py
export NODE_OPTIONS=--experimental-strip-types
(cd skills/defect-detection && npm test)
node examples/checkout-boundary/verify.mjsThese checks cover documentation links, translation section parity, CLI and provider behavior, packaging, isolation, write-back validation, and the bundled boundary fixture. They do not prove that every defect will be found.
| I want to… | Start here |
|---|---|
| Run a review today | Getting Started · Quick start |
| Understand how it reaches a conclusion, and what stops autopilot output | How the skills work · defect-detection method |
| Know what it misses and when not to trust it | Known Limitations · Support Matrix |
| Check whether code or data leaves my machine | FAQ · Security Policy |
| Reproduce the 7/7 blind evaluation myself | Examples · inventory-service · Methodology |
| See what is next and what is only planned | Capability map and roadmap · Changelog |
| Understand the repository layout and why docs sit where they do | Architecture |
| Know where open source ends and commercial begins | Commercial boundary · LICENSE |
| Send a PR or cut a release | Contributing · Publishing |
- Report reproducible bugs through GitHub Issues
- Ask questions and discuss implementations in GitHub Discussions
- For security vulnerabilities, follow SECURITY.md
When asking for help, include the commit or skill version, operating system, Agent, command, expected result, and actual result. Remove credentials, private source, and proprietary logs.
Elsewhere: Website · Product · Skill Hub
Useful contributions include minimal public reproductions of false positives or missed defects, known-good controls, new analysis rules, fixtures, and documentation improvements. Remove credentials, private source, and proprietary logs before sharing. Start with Contributing, examples, and Publishing.
Apache-2.0 · GitHub