Skip to content

Extend expression planners for Snowflake compatibility - #47

Open
osipovartem wants to merge 14 commits into
embucket-sync-df55.0.0from
expr-planner-aggregate-schema
Open

Extend expression planners for Snowflake compatibility#47
osipovartem wants to merge 14 commits into
embucket-sync-df55.0.0from
expr-planner-aggregate-schema

Conversation

@osipovartem

@osipovartem osipovartem commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add backward-compatible schema-aware hooks for aggregate and scalar expression planners
  • preserve the existing planner methods as defaults, so upstream behavior is unchanged
  • pass the function input schema from SqlToRel
  • plan qualified wildcard arguments and wildcard options such as alias.* EXCLUDE (...)
  • add an opt-in datafusion.sql_parser.trim_decimal_literal_trailing_zeros setting for Snowflake-compatible exact decimal inference
  • update h2 to 0.4.16 to address RUSTSEC-2026-0258

This is the collapsed DataFusion side of the Rustice Snowflake compatibility stack. It includes the work previously reviewed independently in #48, #49, and #50.

Compatibility

All new behavior is opt-in through custom expression planners or the new configuration setting. Default DataFusion parsing, type inference, and expression planning remain unchanged.

The hooks allow Rustice to implement schema-dependent Snowflake syntax including qualified wildcard expansion in aggregate and scalar functions. Decimal normalization reproduces Snowflake inference such as:

  • 10.00 -> DECIMAL(2, 0)
  • 10.10 -> DECIMAL(3, 1)
  • 0.00100 -> DECIMAL(4, 3)
  • 1.2300e2 -> DECIMAL(3, 0)

Validation

  • DataFusion SQL unit tests: 92 passed
  • aggregate and scalar planner integration tests
  • qualified wildcard planner tests
  • decimal literal planner integration tests
  • cargo check -p datafusion
  • Clippy for datafusion-common, datafusion-expr, datafusion-sql, and datafusion with -D warnings
  • cargo fmt --all -- --check
  • Taplo workspace formatting check
  • full GitHub Actions matrix for the stacked PRs, including SQLite sqllogictests

@osipovartem osipovartem changed the title Expose input schema to aggregate expression planners Extend expression planners for Snowflake compatibility Sep 1, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation core common sqllogictest labels Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant