bench(pco): scalar access baseline for CodSpeed - #9896
Conversation
Four cases of scalar reads out of a PCO array: a single read, and 1,024 clustered, nullable clustered, and scattered reads. Establishes a CodSpeed baseline so the probe work in encodings/pco can show its effect on merge. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Merging this PR will regress 2 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | random_i8[0.5] |
71.3 µs | 94.9 µs | -24.81% |
| ❌ | Simulation | new_raw_prim_test_between[i32, 2048] |
64.1 µs | 77.9 µs | -17.66% |
| ⚡ | Simulation | execute_scalar_struct_wide |
503 µs | 389.6 µs | +29.12% |
| ⚡ | Simulation | random_i16[0.8] |
96.4 µs | 78.5 µs | +22.83% |
| ⚡ | Simulation | execute_scalar_struct_simple |
106.5 µs | 89.4 µs | +19.02% |
| ⚡ | Simulation | allocate_drop_arrow[0] |
456.9 ns | 402.7 ns | +13.45% |
| 🆕 | Simulation | scalar_access[(1, false, false)] |
N/A | 208.8 µs | N/A |
| 🆕 | Simulation | scalar_access[(1024, false, false)] |
N/A | 132.3 ms | N/A |
| 🆕 | Simulation | scalar_access[(1024, false, true)] |
N/A | 133.3 ms | N/A |
| 🆕 | Simulation | scalar_access[(1024, true, false)] |
N/A | 100.7 ms | N/A |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ji/pco-scalar-bench (7403cf4) with develop (b5f43ba)2
Footnotes
-
218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
develop(d69291f) during the generation of this report, so b5f43ba was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Drops the probe bench added by this PR; the scalar bench from #9896 is the baseline, and reading it through one `RepeatedArrayProbe` shows the retained page decode on the clustered cases. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CqrLKgPqFYGZK5sjk1qe7 Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
One divan benchmark,
scalar_access, reading scalars out of a PCO array throughexecute_scalar, to give CodSpeed a baseline before the scalar-probe work lands. The probe PR will switch the repeated cases torepeated_probe()under the same bench names so the comparison shows up on merge.Fixture: 16 384
u32rows with valuesrow / 16, PCO level 8, 1024 values per page. Nullable cases null every 11th row. Indices come from a fixed-seed LCG, clustered within 256 rows starting at row 4096 (one page) or scattered over the whole array.Cases are
(access_count, nullable, scattered):(1, false, false)(1024, false, false)(1024, true, false)(1024, false, true)Today every read decodes its page, so clustered and scattered cost the same. The single-read case is the control for the one-off path, which the probe work does not change.
Overlaps with #9895, which covers the same ground with cross-chunk and in-chunk cases; only one of the two should merge.
Checks
cargo bench -p vortex-pco --bench scalar: builds and runs🤖 Generated with Claude Code