fix: support decimal-to-integer array casts - #9873
Conversation
Signed-off-by: Xiangpeng Hao <me@xiangpeng.systems>
|
@claude review |
|
LGTM. Can you also add a duckdb and a datafusion SLT please? |
Signed-off-by: Xiangpeng Hao <me@xiangpeng.systems>
Merging this PR will degrade performance by 10.22%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | mul_u64_nonnull_neon |
15.2 µs | 20.8 µs | -27.04% |
| ❌ | Simulation | random_i32[0.95] |
91.8 µs | 110.5 µs | -16.93% |
| ❌ | WallTime | mul_i64_nonnull_neon |
17.2 µs | 20.1 µs | -14.6% |
| ❌ | WallTime | multiply_shapes_neon[(16384, PerRowPerRow)] |
17.2 µs | 20 µs | -14.39% |
| ❌ | WallTime | mul_u32_nonnull_avx512 |
5.6 µs | 6.3 µs | -10.83% |
| ⚡ | Simulation | random_i8[0.5] |
94.2 µs | 71 µs | +32.55% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing XiangpengHao:codex/decimal-to-integer-cast (2f4101e) with develop (7ecae9e)
Footnotes
-
218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Summary
Decimal scalars can be cast to integers, but decimal arrays currently have no corresponding cast kernel. This makes expressions such as
CAST(row_number() OVER () - 1 AS BIGINT)fail in downstream SQL execution withNo CastKernel ... from decimal(21,0) to i64?.Add decimal-array casts to all eight primitive integer types and share the conversion implementation with scalar casts. The policy remains: reject values outside the target range before truncating fractional digits toward zero. For example,
12.9becomes12, while127.1toi8and-0.1to an unsigned integer are rejected.Changes
Use checked integer arithmetic for scaling and range checks. This also corrects scalar rounding for large integers, handles negative scales, and supports
i256values without narrowing throughi128orf64.Reuse the existing decimal buffer conversion path, preserving validity and skipping invalid lanes.
Add 64 regression cases covering scalar/array agreement, all six decimal storage widths, all integer targets, exact bounds, fractional overflow, nulls, empty arrays, and extreme scales.
Add DataFusion and DuckDB SLTs over a decimal column stored in a Vortex file. DataFusion asserts cast pushdown and checks parity with pushdown disabled. DuckDB asserts that the cast stays in its own projection, preserving rounding semantics. Both cover nulls, fractions, large exact values, and
BIGINTbounds.Validation
cargo nextest run -p vortex-array -E 'test(decimal)': 377 passed, including all 64 new cases.cargo nextest run -p vortex-sqllogictest -E 'test(decimal_to_integer)': 2 passed (DataFusion and DuckDB).vortex-arraywith the CI-pinnednightly-2026-09-10formatter.Int64values0, 1; the affected two-episode decode evaluation completed with 12 images.Workspace-wide checks and clippy were not run locally.