perf(array): append repeated and viewed lists in bulk, not a value at a time [builders-child-stack]

[robert3005]

Four callers walked a builder one list at a time where a whole run of them was available up front.

Sparse canonicalization — the gaps. The fill value was appended once per row. A gap is a run of one value, so it goes in as a single ConstantArray instead: canonicalizing a constant list array points every view at one copy of the value, so a gap now costs the fill value's elements once however many rows it covers. Appending a 10,000-row gap of a three-element fill produced 30,000 elements; it now produces 3. The capacity estimate follows, bounding the fill by the number of gaps rather than the number of filled rows.

Sparse canonicalization — the patches. The patch loop appended one list per patch. Patches landing on consecutive rows now go in as one slice of the patch array, so the builder sees an append per gap rather than one per patch. This is the case that matters once #8964 lands: with 100 patches on consecutive rows, the elements child went from 101 chunks to 2.

To slice a run out of the patch values they have to be laid out in row order, so they are flattened once up front with MakeExact rather than a list at a time inside the loop. That has two further effects worth noting for review: a null patch now carries a zero-size view instead of the elements the old code skipped, so an appended run holds exactly the elements its patches reference; and the elements half of the capacity estimate becomes exact rather than an upper bound.

ListBuilder::append_listview_array sliced the elements array and appended the slice once per list, which is what its ListViewBuilder twin stopped doing. ListArray offsets can only describe contiguous, in-order lists, so flatten the incoming views to that layout — a no-op when they are laid out that way already — and then append the referenced elements in one go, walking only the metadata to rebase the offsets.

This is groundwork for the rest of the stack: once a nested builder keeps every appended array as a chunk, a caller appending a value at a time would produce a chunk per row.

What is still per-row

Patches that alternate with fill rows (indices = [0, 2, 4, …]) still cost one append each, and there is no coalescing to be had — those rows genuinely alternate between patch values and fill values, so the elements child genuinely interleaves. Only a copy could flatten that, which is the trade #8964 leaves to the caller.

Tests

Two new tests in encodings/sparse, each checked against a mutated implementation:

• a 10,000-row sparse list with two patches, asserting the fill's elements are stored once per gap (11 elements total, not 30,000)
• a run of consecutive patches with overlapping, out-of-order views and a null patch inside the run, separated by gaps from a second run — covering the run arithmetic and everything that has to survive it. Dropping the MakeExact flatten fails this test and nothing else; using the old per-patch next_index formula fails it along with five existing ones.

One in vortex-array: an overlapping ListViewArray and a sliced (leading-slack) one appended into the same ListBuilder.

Checks

• cargo nextest run --workspace — 7168 passed
• cargo clippy --all-targets --all-features — clean
• cargo +nightly fmt --all --check — clean
• cargo test --doc -p vortex-array -p vortex-sparse — passed

🤖 Generated with Claude Code

[codspeed-hq]

Merging this PR will degrade performance by 83.3%

❌ 4 regressed benchmarks
✅ 1837 untouched benchmarks
⏩ 55 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	canonicalize_sparse_list[(512, 7, 4)]	254.4 µs	3,582.5 µs	-92.9%
❌	Simulation	canonicalize_sparse_list[(1024, 17, 8)]	380.3 µs	3,056.2 µs	-87.56%
❌	Simulation	canonicalize_sparse_fixed_size_list[(1024, 17, 8)]	324.2 µs	1,241.2 µs	-73.88%
❌	Simulation	canonicalize_sparse_fixed_size_list[(512, 7, 4)]	224.4 µs	666.8 µs	-66.34%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing claude/builders-bulk-loop-callers-9ze0t6 (3304731) with claude/listview-builder-keep-overlaps-9ze0t6 (0779ba5)}

Footnotes

1. 55 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩