Skip to content

Bench: add exact algorithmic-growth evidence profiles #670

Description

@chrisbbreuer

Parents: #461, #493, #494

Blocks evidence closure for: #615, #616, #618, #619, #621

Problem

The exact-parent A/B profile correctly requires wall time, retired instructions, cycles, and energy to remain within 5% RSD on a quiet reference host. Compiler classifier fixes can reduce a candidate boundary from quadratic work to a few milliseconds, below the energy and cycle meters' stable resolution. Multiplying identical jobs enough to stabilize the candidate makes the exact quadratic parent prohibitively long at the 4K growth point.

This is not permission to weaken the full-efficiency profile or publish noisy wall/energy results. We need a distinct algorithmic-growth artifact whose claim boundary is normalized compiler work across input sizes, while full efficiency remains a separate evidence class.

Observed exact #615 attempts on the quiet reference host, with exact parent e9412de and candidate 8da47d8, Zig 0.17.0-dev.1441, seven order-balanced pairs, nominal thermals, and identical checksums:

  • 10 jobs at 1K: candidate wall 4.000 ms; candidate energy RSD 105%, correctly blocked.
  • 100 jobs at 1K: candidate wall 37.237 ms; candidate wall/cycles/energy RSD 25.1%/11.4%/9.3%, correctly blocked.
  • Retired instructions were stable in both attempts (candidate RSD 0.09% and 0.14%) and exact allocation replay was invariant.

Raw rejected artifacts remain outside the repository in the active #615 evidence workspace until this contract is implemented.

Scope

  • Add a versioned algorithmic_growth artifact/profile alongside, not inside, the current full-efficiency exact-parent profile.
  • Score at least three frozen input widths using exact parent/candidate revisions, identical generated sources, admission/result checksums, structural checksums, allocation replay, and normalized retired instructions per logical job.
  • Require stable instruction samples at every width and declare wall/cycles/energy diagnostic or unavailable with explicit provenance; never encode missing or noisy counters as zero.
  • Report fitted/adjacent growth ratios with raw rows so bounded near-linear candidate growth and superlinear parent growth are auditable without a throughput claim.
  • Keep one separately sized full-efficiency row when a representative boundary can satisfy the existing wall/instruction/cycle/energy policy.
  • Preserve fail-closed competing-process, cleanliness, exact-first-parent, toolchain, thermal, power, and checksum guards.
  • Add schema/tool self-tests for short candidate boundaries, nonlinear parents, unavailable/noisy metrics, normalization, and rejected publication attempts.

No-workaround rules

  • No threshold relaxation, selective sample removal, source rewriting, asymmetric parent/candidate work, heuristic extrapolation, wall-time claim from a diagnostic row, or replacement of the existing full-efficiency profile.
  • No stacks, queues, jobs-runtime, Promise, Worker, or Map/Set iterator changes.

Acceptance

  • A versioned algorithmic-growth schema and validator reject identity, checksum, normalization, width, sample, or stability drift.
  • Compiler: scan pending TDZ references once #615's frozen 1K/2K/4K witnesses produce exact-parent growth evidence without claiming noisy wall/energy performance.
  • At least one unchanged full-efficiency fixture still blocks on unstable energy, proving the existing policy was not weakened.
  • Raw commands, exact revisions/toolchains/dependencies, samples, normalized metrics, and rendered methodology are checked in and docs/link gates pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarksBenchmark design, evidence, and regression gatesperformanceMeasured runtime, memory, scaling, or build performancetoolingBuild, test, documentation, and developer tooling

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions