perf: specialize primitive sums for run-end arrays

[connortsui20]

Summary

Computes whole-array and grouped Sum/SumV2 directly from primitive run values and lengths, with decimals using the existing fallback. Local median timings drop from 1.770 to 1.062 µs for whole-array sums and from 12.70 to 7.603 µs for 2-element groups for nullable i32 inputs with 64-element runs.

Changes

Uses direct traversal for all-valid runs, valid-run indices for partially valid inputs, and a shared cursor for fixed-size groups. Weighted floating-point sums can round differently from repeated addition, and signed overflow can depend on how partial sums are combined. The existing DuckDB overflow issue discussed in #9829 remains separate work in the shared aggregate/integration layer.

Benchmark methodology and results

Local aarch64-apple-darwin medians with Rust 1.98.0 and 100 samples, using cargo bench -p vortex-runend --bench run_end_sum -- --sample-count 100. Both modes use 31e3f624bad64e9046c5f5adb418903e01ac64e9, with the baseline disabling the run-end kernels in its session. This comparison isolates the specialization. Each input contains 2,048 nullable i32 elements, with 64-element runs and every fifth run null. Grouped timings include finalization to a primitive array.

Workload	Specialized	Kernels disabled
Whole array	1.062 µs	1.770 µs
2-element groups	7.603 µs	12.70 µs
128-element groups	2.312 µs	2.916 µs

Two additional grouped cases use all-valid run values. With the same fixtures applied to the previous implementation at f6685e9ba706f8df162fbb2071c94018f97c7418, medians changed from 7.499 to 6.374 µs for 2-element groups and from 2.082 to 2.062 µs for 128-element groups.

The benchmark also covers run lengths 4 and 1,024, group sizes 1 and 8, and whole-array AllInvalid, AllValid, and NonNullable validity. See the benchmark workflow and CodSpeed results. The overall CodSpeed report also flags existing walltime benchmarks outside this specialization, which were not investigated here.

[codspeed-hq]

Merging this PR will degrade performance by 11.87%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 2 regressed benchmarks
✅ 2188 untouched benchmarks
🆕 35 new benchmarks
⏩ 218 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	dict_canonicalize_gt_u8_avx512[16000000]	6.8 ms	11.3 ms	-39.91%
❌	WallTime	arrow_checked_add_u32_neon[16384]	12.2 µs	20.3 µs	-39.87%
⚡	WallTime	dict_canonicalize_gt_u8_neon[1000000]	559.8 µs	487.3 µs	+14.89%
⚡	Simulation	allocate_drop_arrow[0]	456.9 ns	402.7 ns	+13.45%
⚡	WallTime	dict_canonicalize_gt_u8_neon[16000000]	9.4 ms	8.3 ms	+12.92%
🆕	Simulation	grouped_runend_fallback[1, 1024]	N/A	738.5 µs	N/A
🆕	Simulation	grouped_runend_fallback[1, 4]	N/A	830.8 µs	N/A
🆕	Simulation	grouped_runend_fallback[1, 64]	N/A	763.8 µs	N/A
🆕	Simulation	grouped_runend_fallback[128, 1024]	N/A	149.8 µs	N/A
🆕	Simulation	grouped_runend_fallback[128, 4]	N/A	217.6 µs	N/A
🆕	Simulation	grouped_runend_fallback[128, 64]	N/A	147.3 µs	N/A
🆕	Simulation	grouped_runend_fallback[2, 1024]	N/A	443.9 µs	N/A
🆕	Simulation	grouped_runend_fallback[2, 4]	N/A	518.4 µs	N/A
🆕	Simulation	grouped_runend_fallback[2, 64]	N/A	454.6 µs	N/A
🆕	Simulation	grouped_runend_fallback[8, 1024]	N/A	220.6 µs	N/A
🆕	Simulation	grouped_runend_fallback[8, 4]	N/A	287.9 µs	N/A
🆕	Simulation	grouped_runend_fallback[8, 64]	N/A	222.2 µs	N/A
🆕	Simulation	grouped_runend[1, 1024]	N/A	455.4 µs	N/A
🆕	Simulation	grouped_runend[1, 4]	N/A	482.3 µs	N/A
🆕	Simulation	grouped_runend[1, 64]	N/A	461.4 µs	N/A
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/sum-runend (31e3f62) with develop (e3b8eb2)}

Footnotes

1. 218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩