Skip to content

Warn when the CBMC on PATH does not match the pinned version - #4723

Open
ivmat wants to merge 1 commit into
model-checking:mainfrom
ivmat:cbmc-pin-rebase
Open

Warn when the CBMC on PATH does not match the pinned version#4723
ivmat wants to merge 1 commit into
model-checking:mainfrom
ivmat:cbmc-pin-rebase

Conversation

@ivmat

@ivmat ivmat commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

kani-dependencies pins a CBMC version, but the pin is only enforced at setup/CI time (install_deps.sh, kani-regression.sh). At runtime kani-driver uses whatever cbmc resolves from PATH with no check, so a locally installed CBMC can silently diverge from the pin while the user believes the pinned toolchain is in use. On a machine with several CBMC versions installed this is easy to hit and invisible when it happens — results get attributed to a toolchain that never ran.

This PR makes kani-driver resolve cbmc --version from PATH, compare it against the pin, and warn naming both versions when they differ. Design points:

  • The pin is embedded at compile time (include_str! of kani-dependencies) rather than read at runtime: release bundles don't ship that file, so a runtime read would silently disable the check for exactly the users least able to audit their toolchain. rustc tracks the included file, so editing the pin still rebuilds.
  • --version is handled explicitly (disable_version_flag) so kani --version / cargo kani --version actually run the check — clap's built-in flag exits during parsing, before any driver code runs. The first output line stays kani <version> / cargo-kani <version>, so scripts that parse it keep working; the CBMC lines are appended after it.
  • For cargo kani, the version flag is detected on the raw arguments before project configuration is merged, so --version cannot be broken by a malformed Cargo.toml (matching the robustness of clap's built-in, whose version action previously fired inside cargo_locate_project's early parse). The scan matches whole arguments only and stops at -- or --cbmc-args; a post-parse fallback catches spellings the scan cannot see, such as -V clustered with other short flags. Known residual: a clustered -V combined with a malformed Cargo.toml reports the TOML error instead of the version — loudly, not silently.
  • The warning is suppressed under --quiet during verification (preserving the zero-output contract in check-quiet.sh), but --version as an explicit query always reports.
  • tools/build-kani bundling whichever cbmc is on the builder's PATH is the build-time analogue of the same problem; left as a follow-up to keep this focused. With this check, a mismatched bundle at least reports itself at runtime.

Manual testing

Unit tests cover the version parse/compare and the raw --version scan. Verified live both ways — a matching CBMC 6.10.0 is silent, a mismatched 6.8.0 fires naming both versions. Both script-based version regression tests (kani-version-flag-version, cargo-kani-version-flag-version) pass. cargo kani --version verified to work from a directory whose Cargo.toml is malformed, and a --version after --cbmc-args is forwarded to CBMC, not intercepted.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

…sion

`kani-dependencies` pins the CBMC version, but that pin was only enforced at setup/CI time by
`install_deps.sh` / `kani-regression.sh`. At runtime `kani-driver` used whatever `cbmc` it resolved
from `PATH` with no check, so a locally installed, unpinned CBMC could silently diverge from the pin
while every log claimed the pinned toolchain was in use.

`kani-driver` now resolves the CBMC version actually on `PATH` via `cbmc --version`, compares it
against the pin, and warns naming both versions when they differ. The pin is embedded at compile time
with `include_str!("../../kani-dependencies")` rather than read from the install root at runtime:
release bundles do not ship `kani-dependencies`, so a runtime read would silently disable the check
for exactly the users least able to audit their own toolchain. Embedding also removes the
dev-repo/release asymmetry entirely, and rustc tracks the included file as a build dependency, so
editing the pin still triggers a rebuild.

`--version` is handled explicitly rather than by clap's generated flag (`disable_version_flag`), so
that `kani --version` and `cargo kani --version` actually perform this check. Previously clap
intercepted the flag and exited before the driver ran, which would have made "check `--version`
before a campaign" useless advice. The explicit path preserves clap's conventional first line
(`kani <version>` / `cargo-kani <version>`) so scripts that parse it -- including the two
script-based version regression tests -- keep working, and appends the CBMC lines after it.

For `cargo kani`, the flag is detected by scanning the raw arguments before any project
configuration is merged, so `--version` cannot be broken by a malformed `Cargo.toml` -- matching the
robustness of clap's built-in, whose version action previously fired inside
`cargo_locate_project`'s early parse, before any TOML was read. The scan matches whole arguments
only and stops at `--` or `--cbmc-args` (everything beyond those belongs to CBMC, not Kani); a
post-parse fallback catches spellings the scan cannot see, such as `-V` clustered with other short
flags. Known residual: a clustered `-V` combined with a malformed `Cargo.toml` reports the TOML
error instead of the version -- loudly, not silently.

The check is deliberately not emitted under `--quiet` during verification, preserving the
zero-output contract asserted by `tests/script-based-pre/check-quiet/check-quiet.sh`; `--version`
is an explicit one-shot query, not verification output, so it reports regardless.

Note for reviewers: `tools/build-kani` bundles whichever `cbmc` is on the builder's `PATH`
(`which::which("cbmc")`), which is the build-time analogue of the same problem. That is left as a
follow-up to keep this change focused; with this check in place a mismatched bundle now at least
reports itself at runtime.
@ivmat
ivmat requested a review from a team as a code owner August 7, 2026 06:39
@feliperodri feliperodri added [C] Internal Tracks some internal work. I.e.: Users should not be affected. [I] CI / Infrastructure Work done to CI, tests and infrastructure. T-CBMC Issue related to an existing CBMC issue labels Aug 7, 2026

@feliperodri feliperodri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ivmat again, the code is massively over-narrated... version.rs is roughly half comments, and main.rs has 8-line rationale essays in front of 3-line blocks. This can be a real maintenance burden: prose comments drift from code, and this volume invites that. Let's cut them ~in half (keep the why for include_str!, --quiet, and the raw-scan; drop the paragraphs restating what the code plainly does).

@feliperodri feliperodri assigned ivmat and unassigned tautschnig Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[C] Internal Tracks some internal work. I.e.: Users should not be affected. [I] CI / Infrastructure Work done to CI, tests and infrastructure. T-CBMC Issue related to an existing CBMC issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants