Skip to content

bench: multi-conjunct shared-prefix struct row-filter pushdown - #23524

Open
SubhamSinghal wants to merge 2 commits into
apache:mainfrom
SubhamSinghal:parquet-struct-row-filter-bench
Open

bench: multi-conjunct shared-prefix struct row-filter pushdown#23524
SubhamSinghal wants to merge 2 commits into
apache:mainfrom
SubhamSinghal:parquet-struct-row-filter-bench

Conversation

@SubhamSinghal

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

The row-filter planner in datafusion-datasource-parquet walks the accessed struct-field paths for every filter, resolves the Parquet leaves, and builds a pruned Arrow schema. When multiple predicates share a common struct prefix, the planner can consolidate that traversal work — this is the case the StructAccessTree refactor in #23217 is intended to accelerate.

The existing bench suite does not cover this shape:

  • parquet_struct_query — single-field struct predicates only
    (WHERE s['id'] = 5).
  • parquet_struct_projection — projection-only, no WHERE clause.

So multi-conjunct shared-prefix planning is currently uninstrumented, and the effect of consolidating access-path traversals is invisible against the existing suite.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

No. Benchmark-only; no library or API surface is touched.

@github-actions github-actions Bot added the core Core DataFusion crate label Jul 13, 2026
@kumarUjjawal

Copy link
Copy Markdown
Contributor

Hi @SubhamSinghal I will take a look. Can you please resolve the conflict.

@SubhamSinghal

Copy link
Copy Markdown
Contributor Author

resolved merge conflict

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.87%. Comparing base (62650ef) to head (939dad2).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23524      +/-   ##
==========================================
- Coverage   80.87%   80.87%   -0.01%     
==========================================
  Files        1101     1101              
  Lines      375765   375765              
  Branches   375765   375765              
==========================================
- Hits       303915   303904      -11     
- Misses      53747    53754       +7     
- Partials    18103    18107       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants