Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 20 additions & 22 deletions .agents/README.md
Original file line number Diff line number Diff line change
@@ -1,46 +1,44 @@
# `.agents/` — agent-agnostic source of truth
# `.agents/` — agent compatibility and shared config

This directory is the canonical location for assets shared by AI coding agents
working in this repository (Claude Code, Codex, Cursor, …).
This directory exposes the ModelOpt plugin skills to repository-local agents
and holds shared configuration.

## Layout

```text
.agents/
├── skills/ # SKILL.md files (canonical)
│ └── <skill-name>/SKILL.md
├── skills → ../plugins/modelopt/skills
├── plugins/
│ └── marketplace.json # Codex marketplace
├── scripts/ # shared helper scripts (sync-upstream-skills.sh, …)
└── clusters.yaml.example # remote-cluster config template
```

## Why this exists

Different agents look for skills/config in vendor-specific directories. Rather
than maintaining N copies that drift out of sync, **`.agents/` is the single
source of truth** — each agent's guidance or install mechanism points here
directly.
plugins/modelopt/
├── .claude-plugin/
├── .codex-plugin/
└── skills/ # canonical SKILL.md files
├── common/ # shared skill support files
└── <skill-name>/SKILL.md
```

## How each agent finds these

Each agent points at `.agents/` through whatever mechanism it supports — never
a copy:

- **Claude Code** only auto-discovers skills under `.claude/skills/`, so
`.claude/` holds relative in-repo symlinks back into `.agents/`:
`.claude/skills → ../.agents/skills`, `.claude/scripts → ../.agents/scripts`,
and `.claude/clusters.yaml.example → ../.agents/clusters.yaml.example`. These
follow the same committed-symlink pattern already used elsewhere in this repo
(e.g. `CLAUDE.md`, `tools/launcher/modules/Model-Optimizer`).
- **Future agents** (Codex, Cursor, …) add their own symlink or config pointing
at `.agents/`.
`.claude/skills/` holds relative symlinks into `.agents/skills/`.
- **Repository agents** use `.agents/skills`, a relative symlink into the
plugin.
- **Claude Code and Codex plugins** load `plugins/modelopt/skills` directly.

## Editing rules

- **Always edit files under `.agents/`**.
- **Always edit skills under `plugins/modelopt/skills/`**.
- Vendored-verbatim skills (`launching-evals`, `accessing-mlflow`) are managed
by `.agents/scripts/sync-upstream-skills.sh` — do not modify by hand.
- New skills go in `.agents/skills/<skill-name>/SKILL.md` following the
conventions of existing skills (e.g. `.agents/skills/monitor/SKILL.md`).
- New skills go in `plugins/modelopt/skills/<skill-name>/SKILL.md`.
- Shared support files go in `plugins/modelopt/skills/common/`.

## Project-level cluster config

Expand Down
5 changes: 5 additions & 0 deletions .agents/TOOLING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ of the always-loaded agent instructions.
Update `AGENTS.md` for repository-wide agent instructions. `CLAUDE.md` is
symlinked to `AGENTS.md`, so changes there apply to both Codex and Claude Code.

## Installable Skills

The `modelopt` plugin packages the repository skills for use from any
workspace. Installation commands are in the [README](../README.md#ai-agents).

## Local Overrides

For private local instructions, use the tool-specific override file:
Expand Down
20 changes: 20 additions & 0 deletions .agents/plugins/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"name": "modelopt",
"interface": {
"displayName": "NVIDIA Model Optimizer"
},
"plugins": [
{
"name": "modelopt",
"source": {
"source": "local",
"path": "./plugins/modelopt"
},
"policy": {
"installation": "AVAILABLE",
"authentication": "ON_INSTALL"
},
"category": "Productivity"
}
]
}
5 changes: 3 additions & 2 deletions .agents/scripts/sync-upstream-skills.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,14 @@
#
# Requires: gh, base64, awk. Run from the repo root.
#
# The script overwrites .agents/skills/<skill>/ with upstream contents and
# The script overwrites plugins/modelopt/skills/<skill>/ through the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot comment.

Although writes through .agents/skills reach the new canonical tree on a symlink-capable checkout, the script still ends with git diff $DEST_BASE/launching-evals $DEST_BASE/accessing-mlflow. Git pathspec traversal does not follow a tracked directory symlink, so this advertised review command now fails (or shows no canonical changes) after a sync. Please make DEST_BASE and the final review paths plugins/modelopt/skills, using .agents/skills only as the compatibility entry point.

# .agents/skills compatibility symlink and
# re-applies our provenance lines into each SKILL.md frontmatter. If you have
# local changes to a vendored skill, they will be lost — that is expected,
# since vendored-verbatim skills should not be modified locally.
#
# Note: .claude/skills/ (and other agent-specific skill dirs) are symlinks to
# .agents/skills/ — see .agents/README.md.
# plugins/modelopt/skills/ — see .agents/README.md.

set -euo pipefail

Expand Down
1 change: 1 addition & 0 deletions .agents/skills
21 changes: 21 additions & 0 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"$schema": "https://json.schemastore.org/claude-code-marketplace.json",
"name": "modelopt",
"version": "0.1.0",
"description": "Model Optimizer agent plugins.",
"owner": {
"name": "NVIDIA Corporation"
},
"plugins": [
{
"name": "modelopt",
"source": "./plugins/modelopt",
"description": "Skills for Model Optimizer development, quantization, deployment, and evaluation.",
"version": "0.1.0",
"author": {
"name": "NVIDIA Corporation"
},
"category": "development"
}
]
}
1 change: 1 addition & 0 deletions .claude/skills/benchmark-model-kernels
4 changes: 3 additions & 1 deletion .github/workflows/unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ on:
- "tools/mcp/**"
- "tools/resource_monitor.py"
- ".agents/skills/**"
- "plugins/modelopt/skills/**"
Comment thread
chadvoegele marked this conversation as resolved.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot comment.

The latest manifest-validation comment is still open. plugins/modelopt/.claude-plugin/**, .codex-plugin/**, .mcp.json, and the marketplace manifests are not included in either the push paths or changed-files input, so a manifest-only change skips the skills/plugin checks. Include these paths and run an actual manifest/plugin validator in the skills job.

schedule:
- cron: "0 0 * * *" # Nightly
workflow_dispatch:
Expand Down Expand Up @@ -55,6 +56,7 @@ jobs:
tools/mcp/**
tools/resource_monitor.py
.agents/skills/**
plugins/modelopt/skills/**
Comment thread
chadvoegele marked this conversation as resolved.
linux:
runs-on: ubuntu-latest
timeout-minutes: 15
Expand Down Expand Up @@ -190,7 +192,7 @@ jobs:
# Override addopts to drop the repo's coverage/instafail plugins (not installed here).
run: |
pip install pytest
python -m pytest .agents/skills/ -o addopts="" -p no:cacheprovider -v
python -m pytest plugins/modelopt/skills/ -o addopts="" -p no:cacheprovider -v
unit-pr-required-check:
# Run even if some jobs are skipped
if: ${{ github.event_name == 'pull_request' && always() }}
Expand Down
4 changes: 2 additions & 2 deletions .markdownlint-cli2.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,5 @@ config:
# Vendored upstream skills — kept byte-identical to upstream via
# .agents/scripts/sync-upstream-skills.sh; do not reformat.
ignores:
- ".agents/skills/launching-evals/**"
- ".agents/skills/accessing-mlflow/**"
- "plugins/modelopt/skills/launching-evals/**"
- "plugins/modelopt/skills/accessing-mlflow/**"
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@ repos:
exclude: ^modelopt_recipes/configs/

- id: sync-claude-skills
name: sync .claude/skills/ symlinks from .agents/skills/
name: sync .claude/skills/ symlinks from plugin skills
entry: bash tools/precommit/sync_claude_skills.sh
language: system
files: ^\.agents/skills/
files: ^plugins/modelopt/skills/
Comment thread
kevalmorabia97 marked this conversation as resolved.
pass_filenames: false

- id: check-launcher-yaml
Expand Down
10 changes: 4 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,10 @@ These instructions apply to AI-assisted work in this repository.
- Start with `README.md` for project overview and install.
- Use `modelopt/` for source, `tests/` for focused test coverage, and
`examples/` or `docs/` for usage patterns.
- **Agent skills and shared config live under `.agents/`** — the canonical,
agent-agnostic source of truth (`.agents/skills/<name>/SKILL.md`,
`.agents/scripts/`, `.agents/clusters.yaml.example`). Claude Code's
`.claude/skills`, `.claude/scripts`, and `.claude/clusters.yaml.example` are
relative symlinks into `.agents/`. Always edit files under `.agents/`, not the
symlink path. See `.agents/README.md` for the convention.
- **Agent skills live under `plugins/modelopt/skills/`**, the installable
plugin's canonical skill tree. `.agents/skills` and `.claude/skills` expose
those skills through relative symlinks. Shared agent config and scripts
remain under `.agents/`. See `.agents/README.md` for the convention.

## Coding guidelines

Expand Down
20 changes: 19 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,25 @@ Please read our [Contributing](./CONTRIBUTING.md) guidelines for details on how

## AI Agents

For AI-assisted development setup, see the [agent tooling notes](./.agents/TOOLING.md).
ModelOpt's agent skills can be installed from this repository and used in any

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot comment.

The plugin is not currently usable from an arbitrary workspace as advertised. Only plugins/modelopt is packaged, but installed skills still execute checkout-relative paths such as .agents/skills/benchmark-model-kernels/scripts/..., .agents/skills/day0-release/scripts/..., and .agents/scripts/{nel-next,gdpval-sif}.sh; the latter two helpers are not under the plugin tree at all. deployment also shows scripts/deploy.sh, which resolves against the user's workspace rather than the installed skill directory. These commands therefore fail unless the current workspace happens to be a Model-Optimizer checkout. Please package every runtime helper and resolve it through a Claude/Codex-portable plugin-root mechanism (or ensure every affected workflow explicitly obtains and enters a checkout before using checkout paths), then add a smoke test that installs/loads the plugin from a temporary non-ModelOpt directory and verifies representative helper paths.

@chadvoegele chadvoegele Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[agent comment]

Addressed in b79d701. The nel-next and GDPVal helpers now ship under plugins/modelopt/skills/evaluation/scripts/; bundled helper and cross-skill references resolve from $SKILL_DIR rather than the workspace; and a pre-commit validator rejects checkout-relative plugin paths or missing packaged helpers. I also installed the rebased plugin with both Codex and Claude from an unrelated temporary workspace and exercised representative helper entry points successfully.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Installed marketplace copies only contain plugins/modelopt, but several bundled skills still execute checkout-relative paths such as .agents/skills/..., .agents/scripts/nel-next.sh, and scripts/deploy.sh; some required helpers are outside the plugin tree entirely. From a non-ModelOpt workspace these commands fail, contradicting this claim. Please package the required runtime helpers and resolve them relative to the installed skill/plugin, then add a smoke test from an unrelated working directory.

@chadvoegele chadvoegele Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[agent comment]

Addressed in b79d701. The nel-next and GDPVal helpers now ship under plugins/modelopt/skills/evaluation/scripts/; bundled helper and cross-skill references resolve from $SKILL_DIR rather than the workspace; and a pre-commit validator rejects checkout-relative plugin paths or missing packaged helpers. I also installed the rebased plugin with both Codex and Claude from an unrelated temporary workspace and exercised representative helper entry points successfully.

workspace.

### Claude Code

```bash
claude plugin marketplace add https://github.com/NVIDIA/Model-Optimizer.git
claude plugin install modelopt@modelopt
```

### Codex

```bash
codex plugin marketplace add https://github.com/NVIDIA/Model-Optimizer.git
```

Then open `/plugins`, select the `modelopt` marketplace, and install `modelopt`.
Contributors can also use the skills directly from a checkout. See the
[agent tooling notes](./.agents/TOOLING.md).

### Top Contributors

Expand Down
30 changes: 30 additions & 0 deletions plugins/modelopt/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
"name": "modelopt",
"displayName": "ModelOpt",
"version": "0.1.0",
"description": "Skills for Model Optimizer development, quantization, deployment, and evaluation.",
"author": {
"name": "NVIDIA Corporation"
},
"homepage": "https://github.com/NVIDIA/Model-Optimizer",
"repository": "https://github.com/NVIDIA/Model-Optimizer",
"license": "Apache-2.0",
"keywords": [
"modelopt",
"quantization",
"evaluation",
"deployment",
"llm"
],
"mcpServers": {
"modelopt": {
"command": "uvx",
"args": [
"--from",
"git+https://github.com/NVIDIA/Model-Optimizer.git#subdirectory=tools/mcp",
Comment thread
chadvoegele marked this conversation as resolved.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot comment.

This existing supply-chain/reproducibility comment is unresolved in both this manifest and .mcp.json: uvx installs from the repository's mutable default branch. Pin the Git source to the same tested release tag or full commit in both configurations.

"modelopt-mcp"
]
}
}
}
38 changes: 38 additions & 0 deletions plugins/modelopt/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"name": "modelopt",
"version": "0.1.0",
"description": "Skills for Model Optimizer development, quantization, deployment, and evaluation.",
"author": {
"name": "NVIDIA Corporation",
"url": "https://github.com/NVIDIA/Model-Optimizer"
},
"homepage": "https://github.com/NVIDIA/Model-Optimizer",
"repository": "https://github.com/NVIDIA/Model-Optimizer",
"license": "Apache-2.0",
"keywords": [
"modelopt",
"quantization",
"evaluation",
"deployment",
"llm"
],
"skills": "./skills/",
"mcpServers": "./.mcp.json",
"interface": {
"displayName": "ModelOpt",
"shortDescription": "Optimize, deploy, and evaluate models.",
"longDescription": "Provides Model Optimizer workflows for post-training quantization, deployment, evaluation, result comparison, and release validation.",
"developerName": "NVIDIA",
"category": "Developer Tools",
"capabilities": [
"Interactive",
"Write"
],
"defaultPrompt": [
"Quantize this model with ModelOpt.",
"Deploy and evaluate this checkpoint.",
"Compare the baseline and quantized results."
],
"brandColor": "#76B900"
}
}
10 changes: 10 additions & 0 deletions plugins/modelopt/.mcp.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"modelopt": {
"command": "uvx",
"args": [
"--from",
"git+https://github.com/NVIDIA/Model-Optimizer.git#subdirectory=tools/mcp",
"modelopt-mcp"
]
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ order:
GPU needed:

```bash
python .agents/skills/benchmark-model-kernels/scripts/benchmark_model.py <model> \
python "$SKILL_DIR/scripts/benchmark_model.py" <model> \
--tp <tp> --ep <ep> --ms <m1> <m2> ... --print_only
```

Expand All @@ -63,7 +63,7 @@ order:

```bash
CUDA_VISIBLE_DEVICES=<gpu-index> \
python .agents/skills/benchmark-model-kernels/scripts/benchmark_model.py <model> \
python "$SKILL_DIR/scripts/benchmark_model.py" <model> \
--tp <tp> --ep <ep> --ms <m1> <m2> ... \
--flashinfer_repo <flashinfer-repo> --workdir <workdir>
```
Expand Down Expand Up @@ -112,7 +112,7 @@ missing shape:

```bash
CUDA_VISIBLE_DEVICES=<gpu-index> \
python .agents/skills/benchmark-model-kernels/scripts/benchmark_via_builtin.py \
python "$SKILL_DIR/scripts/benchmark_via_builtin.py" \
--flashinfer_repo <flashinfer-repo> --ms <m1> <m2> ... \
--nks <n>,<k>,<name> --workdir <workdir>
```
Expand Down
10 changes: 10 additions & 0 deletions plugins/modelopt/skills/common/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
name: common
description: Shared ModelOpt support files. Use only when another ModelOpt skill directs you here.
---

# Shared ModelOpt Support

This skill is a placeholder required for Codex plugin validation.

Read only the file named by the calling ModelOpt skill.
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,13 @@ If the cluster config contains multiple clusters and the user did not name the t
For remote, connect:

```bash
source .agents/skills/common/remote_exec.sh
source "$SKILL_DIR/remote_exec.sh"
remote_load_cluster <cluster_name>
remote_check_ssh
remote_detect_env # sets REMOTE_ENV_TYPE = slurm / docker / bare
```

If remote but no config, ask user for: hostname, SSH username, SSH key path, remote workdir. Create `~/.config/modelopt/clusters.yaml` (see `skills/common/remote-execution.md` for format).
If remote but no config, ask user for: hostname, SSH username, SSH key path, remote workdir. Create `~/.config/modelopt/clusters.yaml` (see `remote-execution.md` for format).

## Env-3. What compute is available?

Expand Down Expand Up @@ -79,4 +79,4 @@ Return to the skill's SKILL.md for the execution path based on these results.

## Multi-user / Slack bot

If `MODELOPT_WORKSPACE_ROOT` is set, read `skills/common/workspace-management.md` before proceeding.
If `MODELOPT_WORKSPACE_ROOT` is set, read `workspace-management.md` before proceeding.
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ See `.agents/clusters.yaml.example` for a fully annotated example with multiple
## 2. Connect and Establish Persistent Session

```bash
source .agents/skills/common/remote_exec.sh
source "$SKILL_DIR/remote_exec.sh"
remote_load_cluster <cluster_name> # or omit name to use default_cluster
remote_check_ssh # validates connectivity + starts persistent session
```
Expand Down Expand Up @@ -153,6 +153,6 @@ remote_sync_from <remote_output_subdir> /local/output/

## Reference Files

- **`skills/common/remote_exec.sh`** — Full utility library (session, run, sync, SLURM, Docker helpers)
- **`remote_exec.sh`** — Full utility library (session, run, sync, SLURM, Docker helpers)
- **`.agents/clusters.yaml`** — Active cluster configuration (canonical; `.claude/clusters.yaml` also accepted for back-compat)
- **`.agents/clusters.yaml.example`** — Annotated example config
Loading