[vectorset] Runbook support

[Jack Moffitt (metajack)]

Adds big-ann-benchmarks style runbook support to vectorset.

[Copilot]

Pull request overview

Adds Big ANN Benchmarks–style “runbook” execution to the vectorset binary by introducing a runbook parser, dataset catalog/spec loading, a driver-based runner that executes insert/delete/replace/search steps, and JSON report emission for later analysis.

Changes:

• Added runbook parsing (Runbook / Recipe / Operation) and a Runner that executes runbooks with optional dataset filtering and progress reporting.
• Added dataset catalog + dataset spec parsing/loading (including step ground-truth loading) to support runbook-driven benchmarking.
• Added report structs and JSON output; updated CLI to include a new run subcommand and added example dataset specs/docs.

Reviewed changes

Copilot reviewed 14 out of 22 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
vectorset/src/test_utils.rs	Adds a small helper for creating temporary YAML files used by unit tests.
vectorset/src/runner.rs	Implements the runbook execution engine, operation orchestration, progress, and report generation.
vectorset/src/runbook.rs	Implements runbook YAML parsing, validation, and unit tests.
vectorset/src/report.rs	Defines JSON-serializable report structures for run results.
vectorset/src/main.rs	Adds run subcommand wiring, connection config changes, and integrates catalog/runbook/runner flow.
vectorset/src/garnet.rs	Introduces a Driver implementation for Garnet-backed vector set operations.
vectorset/src/driver.rs	Adds the Driver trait abstraction used by the runner.
vectorset/src/dataset.rs	Adds dataset spec parsing, data loading, and step ground-truth loading utilities.
vectorset/src/catalog.rs	Adds dataset catalog loading from a directory of dataset spec YAML files.
vectorset/README.md	Documents the new run command and updates general usage guidance.
vectorset/datasets/wikipedia-1M.yaml	Adds an example dataset spec entry for the Wikipedia 1M dataset.
vectorset/datasets/wikipedia-100K.yaml	Adds an example dataset spec entry for the Wikipedia 100K dataset.
vectorset/Cargo.toml	Adds dependencies required for runbooks, reporting, and dataset parsing; adds tempfile for tests.
vectorset/Cargo.lock	Locks newly introduced dependencies.
vectorset/.gitignore	Ignores generated reports/ output directory.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.55%. Comparing base (1db5ee0) to head (40ef49b).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1349      +/-   ##
==========================================
- Coverage   91.57%   91.55%   -0.03%     
==========================================
  Files         521      521              
  Lines       99598   100347     +749     
==========================================
+ Hits        91211    91872     +661     
- Misses       8387     8475      +88     

Flag	Coverage Δ
miri	91.55% <ø> (-0.03%)	⬇️
unittests	91.23% <ø> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 17 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Magdalen Dobson Manohar (magdalendobson)]

Can we move these to the higher-level test data folder? Also, they are small, but since they're not human-readable it might be better to use LFS in keeping with the other binary files, even small ones.

[Magdalen Dobson Manohar (magdalendobson)]

Let's maybe not commit these as files since they won't work without set paths, and might get out of date. Instead, can we create one for a dataset in test_data, and then have a test that actually takes it in and validates it?

[Magdalen Dobson Manohar (magdalendobson)]

In general, it feels like a ton of what is set up here is very very close to what already exists in diskann-benchmark and diskann-benchmark-core, including some pretty close redefining of structs. Could this be reworked to reuse as much as possible from there? Or if not could you explain the differences?

Another question/wonder: should this just be in diskann-benchmark where all the other runbooks are run?