perf: Concrete TopK array storage by MassivePizza · Pull Request #23609 · apache/datafusion

MassivePizza · 2026-07-15T15:13:27Z

Which issue does this PR close?

N/A

Rationale for this change

Eliminate the overhead of casting with dyn Any for every item.

What changes are included in this PR?

Cast once per batch and store the typed array.
PrimitiveHashTables was already generic, so this was a big and simple win.
StringHashTable was made ~~to use an enum for consistency/readability~~ generic for a miniscule perf improvement over enum.

Are these changes tested?

Should be covered by existing tests.
For benching try cargo bench -p datafusion --bench topk_aggregate -- "top k=10 aggregate".

top k=10 aggregate 10000000 time-series rows
                        time:   [10.787 ms 10.940 ms 11.093 ms]
                        change: [−38.879% −36.987% −35.111%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

top k=10 aggregate 10000000 worst-case rows
                        time:   [17.414 ms 17.797 ms 18.203 ms]
                        change: [−46.131% −44.095% −41.958%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

Benchmarking top k=10 aggregate 10000000 time-series rows [Utf8View]: Collecting 100 samples in estimated 5.9278 s (500 itera
top k=10 aggregate 10000000 time-series rows [Utf8View]
                        time:   [11.777 ms 11.954 ms 12.147 ms]
                        change: [−37.279% −35.443% −33.760%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

Benchmarking top k=10 aggregate 10000000 worst-case rows [Utf8View]: Collecting 100 samples in estimated 6.3472 s (300 iterat
top k=10 aggregate 10000000 worst-case rows [Utf8View]
                        time:   [20.337 ms 20.786 ms 21.301 ms]
                        change: [−37.920% −35.713% −33.401%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

Are there any user-facing changes?

None.

alamb · 2026-07-16T17:26:44Z

run benchmark topk_aggregate

adriangbot · 2026-07-16T17:27:53Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4994752917-1116-k6xvg 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (8a7b678) to 95de385 (merge-base) diff using: topk_aggregate
Results will be posted here when complete

File an issue against this benchmark runner

codecov-commenter · 2026-07-16T17:51:47Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.87%. Comparing base (62650ef) to head (e5861d2).

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #23609   +/-   ##
=======================================
  Coverage   80.87%   80.87%           
=======================================
  Files        1101     1101           
  Lines      375765   375733   -32     
  Branches   375765   375733   -32     
=======================================
- Hits       303915   303891   -24     
+ Misses      53747    53735   -12     
- Partials    18103    18107    +4

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

adriangbot · 2026-07-16T18:05:44Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.00     63.7±6.94ms        ? ?/sec    1.00     63.5±6.93ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.01     65.1±8.58ms        ? ?/sec    1.00     64.3±8.09ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.41      5.8±0.14ms        ? ?/sec    1.00      4.1±0.07ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.05     55.7±3.17ms        ? ?/sec    1.00     53.0±3.16ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.41      5.8±0.08ms        ? ?/sec    1.00      4.1±0.06ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.07     56.6±2.91ms        ? ?/sec    1.00     53.0±2.95ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.00     51.1±2.32ms        ? ?/sec    1.06     54.2±2.90ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.00     49.1±1.39ms        ? ?/sec    1.06     52.0±2.38ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.00   299.2±16.08ms        ? ?/sec    1.04   311.5±17.93ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.00   289.7±10.10ms        ? ?/sec    1.07   310.2±11.00ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.30     11.7±0.35ms        ? ?/sec    1.00      9.0±0.24ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.26     13.0±0.47ms        ? ?/sec    1.00     10.4±0.34ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.29     18.2±0.97ms        ? ?/sec    1.00     14.2±0.73ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.27     19.8±1.15ms        ? ?/sec    1.00     15.6±0.77ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.01     11.6±0.25ms        ? ?/sec    1.00     11.5±0.21ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.00      9.8±0.18ms        ? ?/sec    1.00      9.8±0.17ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.00     11.6±0.28ms        ? ?/sec    1.01     11.7±0.41ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.00      9.8±0.26ms        ? ?/sec    1.00      9.8±0.21ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	1140.3s
Peak memory	2.1 GiB
Avg memory	323.3 MiB
CPU user	2089.2s
CPU sys	118.7s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	1120.2s
Peak memory	2.1 GiB
Avg memory	319.2 MiB
CPU user	1979.1s
CPU sys	126.4s
Peak spill	0 B

File an issue against this benchmark runner

alamb · 2026-07-16T18:10:53Z

Benchmarks look good -- I am running them once more to see if htey are reproducable

alamb · 2026-07-16T18:10:54Z

run benchmark topk_aggregate

adriangbot · 2026-07-16T18:13:38Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4995135144-1119-ltwqj 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (8a7b678) to 95de385 (merge-base) diff using: topk_aggregate
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-07-16T18:40:05Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.00     61.5±8.15ms        ? ?/sec    1.06     65.1±6.29ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.00     59.5±6.85ms        ? ?/sec    1.03     61.5±5.50ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.40      5.8±0.08ms        ? ?/sec    1.00      4.1±0.07ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.00     31.8±0.48ms        ? ?/sec    1.65     52.4±2.91ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.39      5.8±0.06ms        ? ?/sec    1.00      4.1±0.06ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     31.8±0.30ms        ? ?/sec    1.70     53.9±2.74ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.01     52.3±2.59ms        ? ?/sec    1.00     51.9±2.41ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.01     50.2±2.42ms        ? ?/sec    1.00     49.7±1.68ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.08   323.4±16.09ms        ? ?/sec    1.00   300.1±13.64ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.00   299.5±11.02ms        ? ?/sec    1.00   299.2±10.60ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.31     11.7±0.42ms        ? ?/sec    1.00      9.0±0.30ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.26     13.0±0.46ms        ? ?/sec    1.00     10.3±0.35ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.29     18.2±1.02ms        ? ?/sec    1.00     14.1±0.71ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.27     19.7±0.94ms        ? ?/sec    1.00     15.5±0.77ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.00     11.6±0.24ms        ? ?/sec    1.02     11.8±0.75ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.00      9.8±0.13ms        ? ?/sec    1.01      9.9±0.31ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.00     11.6±0.27ms        ? ?/sec    1.01     11.7±0.45ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.00      9.8±0.15ms        ? ?/sec    1.01      9.9±0.31ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	790.2s
Peak memory	2.1 GiB
Avg memory	463.9 MiB
CPU user	2138.8s
CPU sys	106.2s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	785.2s
Peak memory	2.1 GiB
Avg memory	470.4 MiB
CPU user	1976.8s
CPU sys	116.1s
Peak spill	0 B

File an issue against this benchmark runner

alamb · 2026-07-16T19:41:29Z

🤔

These two seem to reproducibly get worse

distinct 10000000 rows asc [no TopK]                              1.00     31.8±0.48ms        ? ?/sec    1.65     52.4±2.91ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     31.8±0.30ms        ? ?/sec    1.70     53.9±2.74ms        ? ?/sec

Can you look into that @MassivePizza ?

MassivePizza · 2026-07-17T09:31:18Z

Only getting noise locally. I don't think my changes should affect "no TopK" regardless.

distinct 10000000 rows desc [no TopK]
                        time:   [29.979 ms 30.184 ms 30.388 ms]
                        change: [−0.4089% +0.4916% +1.3107%] (p = 0.27 > 0.05)
                        No change in performance detected.

distinct 10000000 rows asc [no TopK]
                        time:   [30.436 ms 30.733 ms 31.053 ms]
                        change: [+1.5647% +2.7108% +3.9667%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe

distinct 10000000 rows desc [TopK]
                        time:   [4.5273 ms 4.5704 ms 4.6145 ms]
                        change: [−23.068% −22.347% −21.532%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

distinct 10000000 rows asc [TopK]
                        time:   [4.3722 ms 4.3967 ms 4.4224 ms]
                        change: [−25.664% −24.988% −24.372%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

avoiding from(Vec) into drop.

xudong963

LGTM

xudong963 · 2026-07-30T08:13:44Z

run benchmark topk_aggregate

xudong963 · 2026-07-30T08:14:06Z

🤔

These two seem to reproducibly get worse

distinct 10000000 rows asc [no TopK]                              1.00     31.8±0.48ms        ? ?/sec    1.65     52.4±2.91ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     31.8±0.30ms        ? ?/sec    1.70     53.9±2.74ms        ? ?/sec

Can you look into that @MassivePizza ?

Retriggered the benchmark, let's see if it can be reproduced

adriangbot · 2026-07-30T08:16:50Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5128314293-1298-wrfkh 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (5e07c8d) to e8a65f2 (merge-base) diff using: topk_aggregate
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-07-30T08:50:32Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.06     62.8±6.31ms        ? ?/sec    1.00     59.1±3.44ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.00     61.9±6.07ms        ? ?/sec    1.00     62.2±6.77ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.44      5.9±0.22ms        ? ?/sec    1.00      4.1±0.05ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.00     52.5±2.83ms        ? ?/sec    1.10     57.5±2.99ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.44      5.9±0.21ms        ? ?/sec    1.00      4.1±0.07ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     52.8±2.73ms        ? ?/sec    1.11     58.4±2.82ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.02     52.3±2.30ms        ? ?/sec    1.00     51.3±2.67ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.00     50.1±2.41ms        ? ?/sec    1.00     49.9±1.91ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.00   295.0±14.49ms        ? ?/sec    1.02   300.1±16.69ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.05   296.3±10.37ms        ? ?/sec    1.00   283.3±13.48ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.33     11.6±0.38ms        ? ?/sec    1.00      8.8±0.26ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.26     12.9±0.42ms        ? ?/sec    1.00     10.2±0.35ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.32     18.2±1.07ms        ? ?/sec    1.00     13.8±0.64ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.29     19.8±1.11ms        ? ?/sec    1.00     15.3±0.77ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.03     11.9±0.72ms        ? ?/sec    1.00     11.6±0.26ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.02      9.9±0.43ms        ? ?/sec    1.00      9.8±0.16ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.01     11.8±0.55ms        ? ?/sec    1.00     11.6±0.31ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.04     10.2±0.63ms        ? ?/sec    1.00      9.8±0.22ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	1005.2s
Peak memory	2.1 GiB
Avg memory	351.3 MiB
CPU user	1952.9s
CPU sys	112.1s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	1005.2s
Peak memory	2.0 GiB
Avg memory	340.7 MiB
CPU user	1932.9s
CPU sys	132.8s
Peak spill	0 B

File an issue against this benchmark runner

MassivePizza · 2026-07-30T11:45:29Z

Still seeing a weird 10% there that I can't repro locally, but the rest looks good 😕

alamb · 2026-07-31T22:03:28Z

run benchmark topk_aggregate

adriangbot · 2026-07-31T22:06:44Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5147849531-1336-clk92 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (5e07c8d) to e8a65f2 (merge-base) diff

Run configuration

run benchmark topk_aggregate

Results will be posted here when complete

File an issue against this benchmark runner

alamb · 2026-07-31T22:06:48Z

This is very nice @MassivePizza and @xudong963 and @jayzhan211 -- I think it is close.

adriangbot · 2026-07-31T22:48:01Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing concrete-topk-storage (5e07c8d) to e8a65f2 (merge-base) diff

Run configuration

run benchmark topk_aggregate

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.00     66.0±7.15ms        ? ?/sec    1.05     69.1±7.78ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.00     63.1±6.09ms        ? ?/sec    1.09     68.8±7.93ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.41      5.9±0.18ms        ? ?/sec    1.00      4.2±0.13ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.00     53.3±3.41ms        ? ?/sec    1.12     59.5±4.63ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.46      6.1±0.29ms        ? ?/sec    1.00      4.2±0.07ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     53.8±3.17ms        ? ?/sec    1.10     59.5±3.69ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.00     53.4±2.87ms        ? ?/sec    1.00     53.2±3.27ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.00     51.0±2.74ms        ? ?/sec    1.00     51.2±2.57ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.00   294.7±17.84ms        ? ?/sec    1.04   306.9±23.15ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.00   307.3±20.70ms        ? ?/sec    1.03   317.5±17.50ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.29     11.8±0.50ms        ? ?/sec    1.00      9.1±0.29ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.29     13.4±0.55ms        ? ?/sec    1.00     10.4±0.40ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.33     18.4±1.04ms        ? ?/sec    1.00     13.8±0.68ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.32     20.2±1.23ms        ? ?/sec    1.00     15.3±0.73ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.01     11.8±0.33ms        ? ?/sec    1.00     11.6±0.30ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.02     10.1±0.32ms        ? ?/sec    1.00      9.9±0.49ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.00     11.5±0.27ms        ? ?/sec    1.08     12.5±0.58ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.00      9.9±0.30ms        ? ?/sec    1.00      9.9±0.28ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	1235.3s
Peak memory	2.2 GiB
Avg memory	291.6 MiB
CPU user	1986.3s
CPU sys	124.9s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	1230.3s
Peak memory	2.1 GiB
Avg memory	296.5 MiB
CPU user	1932.5s
CPU sys	136.4s
Peak spill	0 B

File an issue against this benchmark runner

MassivePizza · 2026-08-03T12:16:16Z

This PR feels pretty solid on its own now.

~~I'll be testing a StringHashTable that doesn't clone any Strings in a separate PR.~~ Right, already tried that and it doesn't work because batches are temporary, silly me.

kumarUjjawal · 2026-08-03T14:38:01Z

run benchmark topk_aggregate

adriangbot · 2026-08-03T14:40:59Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5167784427-1352-jst7g 6.12.85+ #1 SMP Wed Jun 17 20:31:55 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (e5861d2) to 62650ef (merge-base) diff

Run configuration

run benchmark topk_aggregate

Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-03T15:20:06Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing concrete-topk-storage (e5861d2) to 62650ef (merge-base) diff

Run configuration

run benchmark topk_aggregate

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.00     63.1±5.64ms        ? ?/sec    1.05     66.2±6.50ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.00     60.6±5.40ms        ? ?/sec    1.06     64.4±7.67ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.38      5.9±0.10ms        ? ?/sec    1.00      4.3±0.11ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.02     57.5±3.90ms        ? ?/sec    1.00     56.2±3.87ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.44      6.0±0.14ms        ? ?/sec    1.00      4.2±0.06ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     52.8±3.07ms        ? ?/sec    1.10     58.0±4.22ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.00     52.7±2.69ms        ? ?/sec    1.04     54.6±3.03ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.01     51.3±2.59ms        ? ?/sec    1.00     50.9±2.92ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.00   312.7±18.80ms        ? ?/sec    1.06   331.3±19.82ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.00   324.3±22.88ms        ? ?/sec    1.02   331.8±19.08ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.28     12.2±0.49ms        ? ?/sec    1.00      9.5±0.31ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.30     13.9±0.55ms        ? ?/sec    1.00     10.7±0.48ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.31     19.0±1.33ms        ? ?/sec    1.00     14.4±0.74ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.30     20.7±1.19ms        ? ?/sec    1.00     15.9±0.94ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.01     11.8±0.26ms        ? ?/sec    1.00     11.7±0.30ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.01      9.9±0.28ms        ? ?/sec    1.00      9.8±0.23ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.00     11.9±0.44ms        ? ?/sec    1.04     12.3±0.28ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.00      9.8±0.18ms        ? ?/sec    1.01     10.0±0.25ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	1170.3s
Peak memory	2.2 GiB
Avg memory	308.3 MiB
CPU user	1982.6s
CPU sys	118.5s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	1165.3s
Peak memory	2.1 GiB
Avg memory	308.0 MiB
CPU user	1974.9s
CPU sys	133.0s
Peak spill	0 B

File an issue against this benchmark runner

MassivePizza added 4 commits July 15, 2026 16:58

store concrete type in TopK structures

2a56ad5

store concrete string type in TopK hash-table

14a5c02

swap in-place in TopK heap

293a0e8

fix error from cherry-pick

3b28922

github-actions Bot added the physical-plan Changes to the physical-plan crate label Jul 15, 2026

jayzhan211 approved these changes Jul 16, 2026

View reviewed changes

MassivePizza added 2 commits July 16, 2026 17:40

fmt and clippy

6214dfa

Merge branch 'main' into concrete-topk-storage

8a7b678

alamb added the performance Make DataFusion faster label Jul 16, 2026

MassivePizza added 6 commits July 17, 2026 12:43

Merge branch 'main' into concrete-topk-storage

f44bbd9

Merge branch 'main' into concrete-topk-storage

4fc87b5

Merge branch 'main' into concrete-topk-storage

0069af0

remove redundant data_type in PrimitiveHashTable

a82f5fa

Undo StringArrayType convenience check,

dfbf3f9

avoiding from(Vec) into drop.

simplify set_batch in PrimitiveHashTable

5e07c8d

MassivePizza mentioned this pull request Jul 27, 2026

[X-3345] perf: Concrete Top-K storage massive-com/arrow-datafusion#69

Merged

xudong963 approved these changes Jul 30, 2026

View reviewed changes

github-actions Bot added the auto detected api change Auto detected API change label Jul 30, 2026

alamb reviewed Jul 31, 2026

View reviewed changes

Comment thread datafusion/physical-plan/src/aggregates/topk/hash_table.rs Outdated

MassivePizza added 3 commits August 3, 2026 13:47

replace StringArrayType enum with arrow trait

55a2e33

Merge remote-tracking branch 'upstream/main' into concrete-topk-storage

22747b1

use some_value helper

e5861d2

github-actions Bot removed the auto detected api change Auto detected API change label Aug 3, 2026

Uh oh!

Conversation

MassivePizza commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

alamb commented Jul 16, 2026

Uh oh!

adriangbot commented Jul 16, 2026

Uh oh!

codecov-commenter commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

adriangbot commented Jul 16, 2026

Uh oh!

alamb commented Jul 16, 2026

Uh oh!

alamb commented Jul 16, 2026

Uh oh!

adriangbot commented Jul 16, 2026

Uh oh!

adriangbot commented Jul 16, 2026

Uh oh!

alamb commented Jul 16, 2026

Uh oh!

MassivePizza commented Jul 17, 2026

Uh oh!

xudong963 left a comment

Choose a reason for hiding this comment

Uh oh!

xudong963 commented Jul 30, 2026

Uh oh!

xudong963 commented Jul 30, 2026

Uh oh!

adriangbot commented Jul 30, 2026

Uh oh!

adriangbot commented Jul 30, 2026

Uh oh!

MassivePizza commented Jul 30, 2026

Uh oh!

alamb commented Jul 31, 2026

Uh oh!

Uh oh!

adriangbot commented Jul 31, 2026

Uh oh!

alamb commented Jul 31, 2026

Uh oh!

adriangbot commented Jul 31, 2026

Uh oh!

MassivePizza commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kumarUjjawal commented Aug 3, 2026

Uh oh!

adriangbot commented Aug 3, 2026

Uh oh!

adriangbot commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

MassivePizza commented Jul 15, 2026 •

edited

Loading

codecov-commenter commented Jul 16, 2026 •

edited

Loading

MassivePizza commented Aug 3, 2026 •

edited

Loading