Add PerfLab counter threshold and direction metadata

[DrewScoggins]

Summary

• add optional per-counter regression thresholds to the PerfLab Reporting contract
• represent explicitly unknown counter direction while preserving existing boolean callers and legacy JSON
• enforce exactly one default counter, default-is-top, unique names, known direction, and nonblank units for monitored counters
• preserve storage-only non-top counter compatibility

Validation

• 45 Reporting tests
• clean Reporting build and formatting verification

[Copilot]

Pull request overview

This PR updates the Reporting “PerfLab” JSON contract to support optional per-counter regression thresholds and to represent an explicitly unknown counter direction (while keeping legacy higherIsBetter compatibility), and adds stricter validation of test/counter invariants during serialization and reporting JSON generation.

Changes:

• Added CounterDirection plus nullable RegressionThreshold, and introduced explicit handling for “unknown direction” via higherIsBetter: null.
• Enforced counter/test validity (exactly one default counter; default must also be top; unique counter names; known direction and nonblank units for monitored counters) via validation hooks and Reporter.GetJson() validation.
• Expanded unit tests to cover legacy JSON compatibility, new threshold serialization, unknown-direction behavior, and validation failures.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
src/tools/Reporting/Reporting/Test.cs	Adds counter/test validation and runs it on serialize/deserialize; tightens default/top counter invariants.
src/tools/Reporting/Reporting/Reporter.cs	Validates tests when constructing from JSON and before emitting JSON; improves result-table width handling when metric name can be null.
src/tools/Reporting/Reporting/Counter.cs	Adds direction/threshold metadata and validation; preserves legacy JSON via higherIsBetter handling (including null for unknown).
src/tools/Reporting/Reporting.Tests/ReporterTests.cs	Adds/updates tests for legacy compatibility, thresholds, unknown direction, and new validation rules.

Suppressed comments (2)

src/tools/Reporting/Reporting/Test.cs:34

• The exception message includes an extra '$' before the counter name (it will render as a literal '$' in output). Remove the stray '$' so the message reports the actual name cleanly.

This issue also appears on line 65 of the same file.

        if (counter.DefaultCounter && Counters.Any(c => c.DefaultCounter))
        {
            throw new Exception($"Duplicate default counter, name: ${counter.Name}");
        }

src/tools/Reporting/Reporting/Test.cs:69

• The duplicate-counter-name validation message includes an extra '$' before the name, which will show up as a literal '$' in logs/errors. Remove the stray '$'.

        var duplicateCounter = Counters.GroupBy(c => c.Name).FirstOrDefault(group => group.Count() > 1);
        if (duplicateCounter is not null)
        {
            throw new InvalidOperationException($"Duplicate counter name, name: ${duplicateCounter.Key}");
        }

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.