Constant canonicalisation uses similar techniques to SparseArray - #9889
robert3005 wants to merge 2 commits into
Conversation
…ization `ConstantArray::append_to_builder` already shares one copy of a constant run's values the way `SparseArray` shares its fill value, but `constant_canonicalize` still materialized per row. Bring it in line: - A constant fixed-size list now builds one row's elements and tiles that single copy over the run, as `fixed_size_list_fill_tile` does for a sparse fill. Elements that are all the same scalar - which a null list's placeholders always are - stay one `ConstantArray` covering the whole run, so the common cases stop being `list_size * len` scalar appends. - A constant string or binary run adopts the scalar's own buffer instead of copying its bytes, and only when the value is too long to inline. A value of exactly `MAX_INLINED_SIZE` bytes lives in its view, so the buffer pushed for it was never referenced; `uncompressed_size_in_bytes` carried the same off-by-one and is corrected to match. - A constant list's offsets and sizes take the narrowest width that can describe the list, via `match_smallest_list_offset_type!`, rather than always costing eight bytes a row once decoded. Signed-off-by: Robert <robert@spiraldb.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KwhzKebXYiuUjPxBmDdTF9
`execute_sparse_decimal` pushed the fill value through `DecimalBuilder` a value at a time. Build the buffer directly instead, the way `constant_canonicalize` already does for a constant decimal run. Also tightens the comments added in the previous commit. Signed-off-by: Robert <robert@spiraldb.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KwhzKebXYiuUjPxBmDdTF9
Merging this PR will regress 2 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | random_i8[0.5] |
70.4 µs | 94.3 µs | -25.33% |
| ❌ | WallTime | filtered_owned_i64_avx512[OneNullInEight] |
22.8 µs | 26.7 µs | -14.63% |
| ⚡ | Simulation | random_i16[0.95] |
97.7 µs | 80.1 µs | +21.98% |
| ⚡ | WallTime | dict_canonicalize_gt_u8_avx2[16000000] |
7.9 ms | 6.8 ms | +17.06% |
| ⚡ | Simulation | compact_sliced[(1024, 90)] |
1.5 µs | 1.4 µs | +11.92% |
| ⚡ | Simulation | compact_sliced[(2048, 90)] |
1.6 µs | 1.4 µs | +11.42% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/inspiring-maxwell-he0qon (99c8ec3) with develop (c4561ac)
Footnotes
-
218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
cc4d6f2 to
99c8ec3
Compare
1 participant
I have spent quite a bit of time looking at SparseArray canonicalisation and noticed that ConstantArray would benefit from similar improvements