Skip to content

Add Agent Playbook section for automated updates - #142

Open
stefanvacareanu7 wants to merge 3 commits into
masterfrom
docs/agent-update-playbook
Open

Add Agent Playbook section for automated updates#142
stefanvacareanu7 wants to merge 3 commits into
masterfrom
docs/agent-update-playbook

Conversation

@stefanvacareanu7

Copy link
Copy Markdown
Contributor

Summary

  • Adds a new "Agent Playbook: Updating This Repository End-to-End" section to README.md, documenting how an AI agent (or human) should perform a version bump (SonarAnalyzer.VisualBasic and related deps), regenerate docs/patterns.json, build, and test this repo end-to-end.
  • Documentation-only change; no code, Dockerfile, or CI config was modified.

Test plan

  • README renders correctly (markdown table/formatting checked locally)
  • No other files touched

@codacy-production

codacy-production Bot commented Jul 28, 2026

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 2 medium

Alerts:
⚠ 2 issues (≤ 1 issue of at least medium severity)

Results:
2 new issues

Category Results
BestPractice 2 medium (2 false positives)

View in Codacy

AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.

Run reviewer

TIP This summary will be updated as you push new changes.

@codacy-production codacy-production Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

The addition of the Agent Playbook is a significant improvement for automation, and the PR currently meets Codacy quality standards. However, the documentation contains specific point-in-time version examples that will likely mislead automated agents in future update cycles.

There are also gaps in the verification of the playbook's technical accuracy. Specifically, the Makefile targets ('make documentation', etc.) and the file paths for configuration files mentioned in the text have not been cross-referenced with the actual project structure. Addressing these gaps is essential to ensure the playbook is a reliable source of truth for both humans and AI agents.

About this PR

  • The playbook references several Makefile targets and configuration file paths (e.g., .circleci/config.yml). Please verify that these targets and paths are correct and functional within the current repository structure to prevent the documentation from providing incorrect instructions to automated agents.

Test suggestions

  • Verify Markdown rendering for tables, lists, and formatting in the new Playbook section.
  • Verify the accuracy of the documented Makefile targets ('make documentation', 'make configure', 'make build') against the actual project structure.
  • Verify that the listed version-encoding files (e.g., .github/dependabot.yml, .circleci/config.yml) exist and match the described paths.
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Verify the accuracy of the documented Makefile targets ('make documentation', 'make configure', 'make build') against the actual project structure.
2. Verify that the listed version-encoding files (e.g., .github/dependabot.yml, .circleci/config.yml) exist and match the described paths.

TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback

Comment thread README.md

The `docs/` directory is machine-consumed configuration, not just documentation:

- `docs/patterns.json` — the list of Sonar VB.NET rules ("patterns") Codacy knows about, their categories/severities/parameters, and which are enabled by default. Generated file, do not hand-edit. **Note:** as of this writing its `"version"` field reads `8.13` while `Analyzer.csproj` pins `SonarAnalyzer.VisualBasic` at `8.15.0.24505` — regeneration has historically lagged behind dependency bumps, so don't assume the two are in sync unless you check.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM RISK

Suggestion: Avoid hardcoding specific point-in-time version numbers in the descriptive text. This documentation will quickly become stale and may mislead future automated updates.

Suggested change
- `docs/patterns.json` — the list of Sonar VB.NET rules ("patterns") Codacy knows about, their categories/severities/parameters, and which are enabled by default. Generated file, do not hand-edit. **Note:** as of this writing its `"version"` field reads `8.13` while `Analyzer.csproj` pins `SonarAnalyzer.VisualBasic` at `8.15.0.24505` — regeneration has historically lagged behind dependency bumps, so don't assume the two are in sync unless you check.
- `docs/patterns.json` — the list of Sonar VB.NET rules ("patterns") Codacy knows about, their categories/severities/parameters, and which are enabled by default. Generated file, do not hand-edit. **Note:** its "version" field often lags behind the `SonarAnalyzer.VisualBasic` version pinned in `Analyzer.csproj` — regeneration has historically lagged behind dependency bumps, so do not assume the two are in sync.

Comment thread README.md
### 3. Step-by-step update procedure

1. **Bump the version(s)** in `src/Analyzer/Analyzer.csproj` (and `.github/dependabot.yml` ignore list / `.circleci/config.yml` orbs, if applicable) as scoped by the task.
2. **Regenerate the docs.** Requires `xmllint` (`apt-get install libxml2-utils` or `apk add libxml2-utils`), Mono, and `dotnet`, plus network access to download the Sonar plugin JAR: `make documentation`. This runs `update-docs` (downloads `sonar-vbnet-plugin-<version>.jar` from the sonar-dotnet GitHub release matching the bumped version, unzips it, extracts `sonar-vbnet-rules.xml`), then `build-docs` (compiles `src/DocsGenerator`), then runs `DocsGenerator.exe` to regenerate `docs/patterns.json` and `docs/description/*`. Review the diff for new/removed/renamed patterns and stale fixture references. Confirm `docs/patterns.json`'s `"version"` field now matches the bumped version (it can otherwise silently drift, as noted above).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ LOW RISK

Suggestion: This section duplicates requirements and commands already documented in the 'Generate Docs' section. Additionally, the playbook depends on docker and gh which are not listed in the initial requirements summary.

Try running the following prompt in your coding agent:

Consolidate the documentation generation instructions by referencing the 'Generate Docs' section for core requirements, and update the 'Agent Playbook' tool requirements to include 'gh' (GitHub CLI) and 'docker' for the end-to-end workflow.

Comment thread README.md
| `src/Analyzer/Analyzer.csproj` → `Microsoft.CodeAnalysis.VisualBasic.Workspaces` | Roslyn VB workspace APIs the analyzer host depends on | Check [NuGet](https://www.nuget.org/packages/Microsoft.CodeAnalysis.VisualBasic.Workspaces) for compatibility with the target Sonar analyzer version before bumping. |
| `src/Analyzer/Analyzer.csproj` → `Codacy.Engine.Seed` | Codacy's .NET engine SDK | Check NuGet for newer versions if asked to update it; not tied to Sonar bumps. |
| `src/Analyzer/Analyzer.csproj` → `ReverseMarkdown`, `Newtonsoft.Json`, `SQLitePCLRaw.core` | Supporting libraries (used by `DocsGenerator`/`Analyzer`) | Handled routinely by Dependabot; bump manually only if asked or if blocking another change. |
| `.github/dependabot.yml` → `ignore` list | Dependabot is currently told to **skip** several newer `SonarAnalyzer.VisualBasic` and `ReverseMarkdown` releases (e.g. `8.17.0.26580` through `8.21.0.30542`) | If you are deliberately bumping past one of these ignored versions, remove/update the corresponding `ignore` entry so Dependabot doesn't immediately re-flag or conflict; if those versions were ignored due to a known incompatibility, investigate why before un-ignoring. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ LOW RISK

Suggestion: Avoid using specific version strings in the Dependabot examples to prevent the documentation from becoming immediately outdated.

Suggested change
| `.github/dependabot.yml``ignore` list | Dependabot is currently told to **skip** several newer `SonarAnalyzer.VisualBasic` and `ReverseMarkdown` releases (e.g. `8.17.0.26580` through `8.21.0.30542`) | If you are deliberately bumping past one of these ignored versions, remove/update the corresponding `ignore` entry so Dependabot doesn't immediately re-flag or conflict; if those versions were ignored due to a known incompatibility, investigate why before un-ignoring. |
`.github/dependabot.yml``ignore` list | Dependabot is currently told to **skip** several `SonarAnalyzer.VisualBasic` and `ReverseMarkdown` releases. | If you are deliberately bumping past an ignored version, update the corresponding `ignore` entry so Dependabot doesn't immediately re-flag or conflict; investigate why versions were ignored before un-ignoring. |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants