Skip to content

test(harness): report the CPU feature set, so a green run says what it tested - #84

Merged
StuartCameronCode merged 1 commit into
mainfrom
test/report-cpu-features
Aug 26, 2026
Merged

test(harness): report the CPU feature set, so a green run says what it tested#84
StuartCameronCode merged 1 commit into
mainfrom
test/report-cpu-features

Conversation

@StuartCameronCode

Copy link
Copy Markdown
Owner

Follow-up to #83. Diagnostic only — nothing in the pipeline reads this.

Why

GitHub's hosted Windows runners are a mixed fleet for AVX-512, not a migrated one, so which kernels a job exercises is a draw — and nothing in the output said which draw it got. Several bundled plugins select a code path from exactly those bits.

That is how the CTMF AVX-512 crash (#83) reached main and then looked spontaneous:

run x264 "using cpu capabilities" result
nightly 08-21, 08-22, 08-23 ... BMI2 AVX2 pass
nightly 08-24, 08-25 ... BMI2 AVX2 **AVX512** red
two manual re-rolls on 08-25 ... BMI2 AVX2 (both) pass — vacuously

Those last two are the point. They were dispatched specifically to prove the #83 fix end-to-end, both drew non-AVX-512 hardware, and both would have passed against the unfixed tree. Nothing in the log said so unless you knew to go grep x264's banner in an unrelated part of the output.

What

WorkerHarness's startup banner gains a cpu= line:

WorkerHarness: platform=windows-x64
  deps=...
  worker=...
  input=...
  cpu=x86_64 [sse2 sse4.1 avx avx2 fma avx512f]

It comes from the worker's own --probe-cpu ({"arch":..., "features":[...]}, alongside the existing --probe-opencl) rather than being derived Dart-side, so it reports what the actual pipeline process sees — including under emulation, where an x86_64 worker on Apple Silicon reports what Rosetta exposes rather than what the binary was compiled for.

Tests

describeCpu() never throws — a diagnostic that fails a run would be worse than one that says "unknown". The cost is that a broken probe degrades silently, exactly when it matters, so both sides are pinned:

  • app/test/worker_cpu_probe_test.dart (push gate) — the harness can read the probe, and the line is not "unknown".
  • cpu_features_are_reported_and_plausible (Rust) — the list is non-empty, contains sse2 on x86-64 (it is in the baseline, so its absence means a broken probe rather than a modest CPU), and never claims AVX-512 without AVX2.

What this does not do

It does not make CI deterministic — you still can't choose the hardware. It makes a green run self-describing, so "it passed" becomes "it passed on this hardware". The durable guard against a dispatch regression remains a script-generation assertion that runs everywhere regardless of the draw (test_152 in #83); the heavy end-to-end test can only ever confirm opportunistically.

🤖 Generated with Claude Code

…t tested

GitHub's hosted Windows runners are a MIXED fleet for AVX-512, not a
migrated one, so which kernels a job exercises is a draw — and nothing in
the output said which draw it got. Several bundled plugins pick a code
path from exactly those bits.

That is how the CTMF AVX-512 crash reached main and then looked
spontaneous: the nightly passed for three nights on non-AVX-512 runners,
went red the night it drew one that had it (against a tree unchanged for
four days), and two deliberate re-rolls afterwards both landed back on
non-AVX-512 machines and passed vacuously.

WorkerHarness's banner now carries a `cpu=` line:

  cpu=x86_64 [sse2 sse4.1 avx avx2 fma avx512f]

It comes from the worker's own `--probe-cpu` rather than being derived
Dart-side, so it reports what the actual pipeline process sees —
including under emulation, where an x86_64 worker on Apple Silicon
reports what Rosetta exposes rather than what it was compiled for.

`describeCpu()` never throws: a diagnostic that fails a run would be
worse than one saying "unknown". That means a broken probe degrades
silently exactly when it matters, so both sides are pinned —
worker_cpu_probe_test.dart in the push gate and a Rust unit test. The
feature list must be non-empty, must contain sse2 on x86-64 (it is in
the baseline, so its absence means a broken probe rather than a modest
CPU), and must not claim AVX-512 without AVX2.

Diagnostic only — nothing in the pipeline reads it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@StuartCameronCode
StuartCameronCode merged commit 5f162cf into main Aug 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant