Skip to content

fix: support decimal-to-integer array casts - #9873

Merged
myrrc merged 3 commits into
vortex-data:developfrom
XiangpengHao:codex/decimal-to-integer-cast
Sep 14, 2026
Merged

myrrc merged 3 commits into
vortex-data:developfrom
XiangpengHao:codex/decimal-to-integer-cast

Conversation

@XiangpengHao

@XiangpengHao XiangpengHao commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Decimal scalars can be cast to integers, but decimal arrays currently have no corresponding cast kernel. This makes expressions such as CAST(row_number() OVER () - 1 AS BIGINT) fail in downstream SQL execution with No CastKernel ... from decimal(21,0) to i64?.

Add decimal-array casts to all eight primitive integer types and share the conversion implementation with scalar casts. The policy remains: reject values outside the target range before truncating fractional digits toward zero. For example, 12.9 becomes 12, while 127.1 to i8 and -0.1 to an unsigned integer are rejected.

Changes

  • Use checked integer arithmetic for scaling and range checks. This also corrects scalar rounding for large integers, handles negative scales, and supports i256 values without narrowing through i128 or f64.

  • Reuse the existing decimal buffer conversion path, preserving validity and skipping invalid lanes.

  • Add 64 regression cases covering scalar/array agreement, all six decimal storage widths, all integer targets, exact bounds, fractional overflow, nulls, empty arrays, and extreme scales.

  • Add DataFusion and DuckDB SLTs over a decimal column stored in a Vortex file. DataFusion asserts cast pushdown and checks parity with pushdown disabled. DuckDB asserts that the cast stays in its own projection, preserving rounding semantics. Both cover nulls, fractions, large exact values, and BIGINT bounds.

Validation

  • cargo nextest run -p vortex-array -E 'test(decimal)': 377 passed, including all 64 new cases.
  • cargo nextest run -p vortex-sqllogictest -E 'test(decimal_to_integer)': 2 passed (DataFusion and DuckDB).
  • Formatted vortex-array with the CI-pinned nightly-2026-09-10 formatter.
  • Downstream driver rebuild and SQL verification: the failing expression now returns Int64 values 0, 1; the affected two-episode decode evaluation completed with 12 images.

Workspace-wide checks and clippy were not run locally.

Signed-off-by: Xiangpeng Hao <me@xiangpeng.systems>
@XiangpengHao

Copy link
Copy Markdown
Contributor Author

@claude review

@myrrc myrrc added the changelog/feature A new feature label Sep 14, 2026
@myrrc
myrrc enabled auto-merge (squash) September 14, 2026 19:34
@myrrc
myrrc disabled auto-merge September 14, 2026 19:34
@myrrc

myrrc commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

LGTM. Can you also add a duckdb and a datafusion SLT please?

Signed-off-by: Xiangpeng Hao <me@xiangpeng.systems>
@codspeed-hq

codspeed-hq Bot commented Sep 14, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 10.22%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 5 regressed benchmarks
✅ 2191 untouched benchmarks
⏩ 218 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime mul_u64_nonnull_neon 15.2 µs 20.8 µs -27.04%
Simulation random_i32[0.95] 91.8 µs 110.5 µs -16.93%
WallTime mul_i64_nonnull_neon 17.2 µs 20.1 µs -14.6%
WallTime multiply_shapes_neon[(16384, PerRowPerRow)] 17.2 µs 20 µs -14.39%
WallTime mul_u32_nonnull_avx512 5.6 µs 6.3 µs -10.83%
Simulation random_i8[0.5] 94.2 µs 71 µs +32.55%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing XiangpengHao:codex/decimal-to-integer-cast (2f4101e) with develop (7ecae9e)

Open in CodSpeed

Footnotes

  1. 218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@myrrc
myrrc merged commit 0a2d04c into vortex-data:develop Sep 14, 2026
80 of 82 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants