Skip to content

perf(sort): fast path for full batch when it sorts before all other streams in SortPreservingMergeStream - #23345

Draft
rluvaton wants to merge 11 commits into
apache:mainfrom
rluvaton:fast-batch-wise-skip
Draft

perf(sort): fast path for full batch when it sorts before all other streams in SortPreservingMergeStream#23345
rluvaton wants to merge 11 commits into
apache:mainfrom
rluvaton:fast-batch-wise-skip

Conversation

@rluvaton

@rluvatonrluvaton commented Jul 6, 2026

Copy link
Copy Markdown
Member
Comparing HEAD and fast-batch-wise-skip
--------------------
Benchmark sort_tpch10.json
--------------------
┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ fast-batch-wise-skip ┃ Change ┃
┡━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ Q2 │ 1421.45 / 1431.48 ±10.65 / 1451.01 ms │ 201.29 / 208.62 ±6.77 / 217.57 ms │ +6.86x faster │
│ Q4 │ 2741.65 / 2770.88 ±16.58 / 2789.94 ms │ 392.40 / 423.10 ±20.71 / 453.79 ms │ +6.55x faster │

Which issue does this PR close?

N/A

Rationale for this change

this is part of an effort I'm doing to make sort much much faster.

here I'm making it faster when the data is somewhat sorted

in this there are 2 things that we save on in case we get to the fast path:

  1. Not dealing with the the loser tree
  2. Skip row by row comparison
  3. interleave and data copy - as we output the same batch as input

What changes are included in this PR?

fast skip when can emit entire batch

Are these changes tested?

yes and existing tests

Are there any user-facing changes?

Not API changes but behavioral changes regarding the size of the output batches:

  • sort may not respect batch_size for already sorted input - it will return the same size as the input stream
  • sort may produce small batches between full sorted batches
    e.g. we have 3 streams and batch size is 4:
s1: 1, 10, 12, 13
s2: 3, 4, 5, 6
s3: 2, 9, 11, 11

the output will be:

batch 1: 1, 2
batch 2: 3, 4, 5, 6
batch 3: 9, 10, 11, 11
batch 4: 12, 13

TODO

  • Need to add benchmark
    • for already sorted
    • for random
    • for reverse ordering

@github-actionsgithub-actionsBot added the physical-plan Changes to the physical-plan crate label Jul 6, 2026
@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmarks

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4895826694-867-l8b6b 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (f3cffeb) to 6a0e76e (merge-base) diff using: tpcds
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4895826694-866-fn2jv 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (f3cffeb) to 6a0e76e (merge-base) diff using: clickbench_partitioned
Results will be posted here when complete


File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark sort_preserving_merge

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4895826694-868-v64xg 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (f3cffeb) to 6a0e76e (merge-base) diff using: tpch
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4895853179-869-8t7j4 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (f3cffeb) to 6a0e76e (merge-base) diff using: sort_preserving_merge
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and fast-batch-wise-skip
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ fast-batch-wise-skip ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1 │ 38.31 / 38.95 ±0.94 / 40.82 ms │ 38.31 / 39.26 ±1.15 / 41.12 ms │ no change │
│ QQuery 2 │ 19.10 / 19.59 ±0.35 / 20.14 ms │ 19.05 / 19.40 ±0.24 / 19.79 ms │ no change │
│ QQuery 3 │ 31.43 / 33.05 ±2.10 / 36.86 ms │ 30.84 / 34.16 ±2.84 / 39.23 ms │ no change │
│ QQuery 4 │ 17.58 / 17.81 ±0.14 / 17.97 ms │ 17.68 / 18.60 ±0.66 / 19.45 ms │ no change │
│ QQuery 5 │ 38.59 / 39.27 ±0.78 / 40.77 ms │ 38.96 / 40.24 ±1.06 / 41.38 ms │ no change │
│ QQuery 6 │ 16.32 / 16.46 ±0.14 / 16.66 ms │ 16.28 / 17.51 ±1.45 / 19.89 ms │ 1.06x slower │
│ QQuery 7 │ 43.46 / 45.02 ±1.13 / 46.52 ms │ 45.18 / 48.11 ±2.88 / 53.10 ms │ 1.07x slower │
│ QQuery 8 │ 43.64 / 43.95 ±0.38 / 44.61 ms │ 43.55 / 43.68 ±0.11 / 43.86 ms │ no change │
│ QQuery 9 │ 50.00 / 51.20 ±0.90 / 52.74 ms │ 49.92 / 50.72 ±0.75 / 51.80 ms │ no change │
│ QQuery 10 │ 42.44 / 42.63 ±0.16 / 42.93 ms │ 42.67 / 42.89 ±0.15 / 43.06 ms │ no change │
│ QQuery 11 │ 13.55 / 13.72 ±0.20 / 14.06 ms │ 13.55 / 13.83 ±0.17 / 14.03 ms │ no change │
│ QQuery 12 │ 24.12 / 24.72 ±0.65 / 25.90 ms │ 24.27 / 25.19 ±0.94 / 26.92 ms │ no change │
│ QQuery 13 │ 32.22 / 33.75 ±1.34 / 36.04 ms │ 32.20 / 35.27 ±2.54 / 39.37 ms │ no change │
│ QQuery 14 │ 23.77 / 24.03 ±0.16 / 24.22 ms │ 23.62 / 23.97 ±0.18 / 24.13 ms │ no change │
│ QQuery 15 │ 31.39 / 31.53 ±0.16 / 31.85 ms │ 31.16 / 32.00 ±0.69 / 32.85 ms │ no change │
│ QQuery 16 │ 13.97 / 14.16 ±0.11 / 14.29 ms │ 14.05 / 14.16 ±0.09 / 14.25 ms │ no change │
│ QQuery 17 │ 73.62 / 75.02 ±1.55 / 77.25 ms │ 74.47 / 75.30 ±1.16 / 77.60 ms │ no change │
│ QQuery 18 │ 58.95 / 59.84 ±0.88 / 61.44 ms │ 57.16 / 60.18 ±1.54 / 61.37 ms │ no change │
│ QQuery 19 │ 33.58 / 34.70 ±1.24 / 36.64 ms │ 33.06 / 33.67 ±0.92 / 35.47 ms │ no change │
│ QQuery 20 │ 32.19 / 32.48 ±0.20 / 32.78 ms │ 31.93 / 32.49 ±0.45 / 33.05 ms │ no change │
│ QQuery 21 │ 54.79 / 56.62 ±1.93 / 60.04 ms │ 56.11 / 57.62 ±2.11 / 61.82 ms │ no change │
│ QQuery 22 │ 13.98 / 14.47 ±0.55 / 15.16 ms │ 13.92 / 14.20 ±0.17 / 14.41 ms │ no change │
└───────────┴────────────────────────────────┴────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD) │ 762.96ms │
│ Total Time (fast-batch-wise-skip) │ 772.45ms │
│ Average Time (HEAD) │ 34.68ms │
│ Average Time (fast-batch-wise-skip) │ 35.11ms │
│ Queries Faster │ 0 │
│ Queries Slower │ 2 │
│ Queries with No Change │ 20 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴──────────┘

Resource Usage

tpch — base (merge-base)

MetricValue
Wall time5.0s
Peak memory1.1 GiB
Avg memory496.1 MiB
CPU user21.9s
CPU sys1.7s
Peak spill0 B

tpch — branch

MetricValue
Wall time5.0s
Peak memory1.2 GiB
Avg memory519.5 MiB
CPU user22.4s
CPU sys1.7s
Peak spill0 B

File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and fast-batch-wise-skip
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ fast-batch-wise-skip ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 5.81 / 6.25 ±0.80 / 7.86 ms │ 5.62 / 6.13 ±0.88 / 7.89 ms │ no change │
│ QQuery 2 │ 80.56 / 80.77 ±0.22 / 81.15 ms │ 81.03 / 81.82 ±1.01 / 83.73 ms │ no change │
│ QQuery 3 │ 29.92 / 30.08 ±0.15 / 30.32 ms │ 31.04 / 31.54 ±0.26 / 31.78 ms │ no change │
│ QQuery 4 │ 481.06 / 494.06 ±7.09 / 500.82 ms │ 538.73 / 569.12 ±23.25 / 595.98 ms │ 1.15x slower │
│ QQuery 5 │ 52.08 / 52.48 ±0.36 / 53.11 ms │ 53.70 / 53.82 ±0.07 / 53.90 ms │ no change │
│ QQuery 6 │ 37.14 / 37.36 ±0.14 / 37.57 ms │ 38.13 / 38.48 ±0.28 / 38.79 ms │ no change │
│ QQuery 7 │ 94.84 / 95.50 ±0.45 / 96.00 ms │ 95.48 / 100.20 ±3.49 / 105.23 ms │ no change │
│ QQuery 8 │ 37.06 / 37.55 ±0.38 / 38.08 ms │ 36.55 / 37.25 ±0.39 / 37.68 ms │ no change │
│ QQuery 9 │ 53.30 / 54.88 ±1.31 / 56.36 ms │ 54.00 / 54.99 ±0.83 / 56.26 ms │ no change │
│ QQuery 10 │ 63.99 / 64.78 ±0.72 / 66.07 ms │ 63.81 / 65.71 ±2.45 / 70.54 ms │ no change │
│ QQuery 11 │ 300.79 / 304.59 ±2.95 / 309.30 ms │ 307.19 / 311.36 ±3.50 / 316.60 ms │ no change │
│ QQuery 12 │ 28.84 / 29.30 ±0.36 / 29.73 ms │ 28.91 / 29.06 ±0.12 / 29.26 ms │ no change │
│ QQuery 13 │ 118.53 / 121.45 ±4.36 / 130.05 ms │ 119.15 / 120.51 ±1.20 / 122.65 ms │ no change │
│ QQuery 14 │ 431.47 / 435.81 ±3.99 / 442.30 ms │ 415.21 / 419.69 ±2.40 / 421.96 ms │ no change │
│ QQuery 15 │ 64.49 / 66.05 ±1.40 / 68.69 ms │ 57.44 / 58.38 ±0.57 / 59.14 ms │ +1.13x faster │
│ QQuery 16 │ 7.36 / 7.46 ±0.08 / 7.60 ms │ 6.91 / 7.08 ±0.26 / 7.59 ms │ +1.05x faster │
│ QQuery 17 │ 81.89 / 84.03 ±1.75 / 87.22 ms │ 80.99 / 83.86 ±2.42 / 88.08 ms │ no change │
│ QQuery 18 │ 123.93 / 124.96 ±0.96 / 126.68 ms │ 123.48 / 125.73 ±1.62 / 127.64 ms │ no change │
│ QQuery 19 │ 41.61 / 42.65 ±1.19 / 44.91 ms │ 42.08 / 42.72 ±0.86 / 44.39 ms │ no change │
│ QQuery 20 │ 35.69 / 36.60 ±0.70 / 37.52 ms │ 36.28 / 37.97 ±1.44 / 40.66 ms │ no change │
│ QQuery 21 │ 17.51 / 17.82 ±0.29 / 18.24 ms │ 17.91 / 18.03 ±0.10 / 18.19 ms │ no change │
│ QQuery 22 │ 61.80 / 63.29 ±1.36 / 65.68 ms │ 63.07 / 63.61 ±0.46 / 64.41 ms │ no change │
│ QQuery 23 │ 348.82 / 353.45 ±4.00 / 358.68 ms │ 346.86 / 389.18 ±25.46 / 416.99 ms │ 1.10x slower │
│ QQuery 24 │ 227.47 / 229.57 ±2.32 / 233.71 ms │ 229.68 / 241.61 ±8.90 / 253.87 ms │ 1.05x slower │
│ QQuery 25 │ 109.83 / 112.41 ±1.52 / 114.10 ms │ 113.70 / 116.78 ±2.25 / 120.07 ms │ no change │
│ QQuery 26 │ 57.75 / 58.59 ±0.59 / 59.33 ms │ 59.75 / 60.25 ±0.36 / 60.72 ms │ no change │
│ QQuery 27 │ 6.48 / 6.66 ±0.17 / 6.99 ms │ 6.54 / 6.69 ±0.16 / 6.99 ms │ no change │
│ QQuery 28 │ 61.02 / 61.96 ±0.60 / 62.59 ms │ 61.13 / 62.04 ±0.55 / 62.79 ms │ no change │
│ QQuery 29 │ 97.51 / 101.56 ±4.41 / 109.66 ms │ 98.40 / 102.52 ±5.62 / 113.39 ms │ no change │
│ QQuery 30 │ 32.78 / 33.19 ±0.45 / 33.96 ms │ 31.98 / 32.46 ±0.37 / 33.12 ms │ no change │
│ QQuery 31 │ 111.97 / 112.58 ±0.58 / 113.69 ms │ 111.90 / 114.66 ±2.61 / 118.92 ms │ no change │
│ QQuery 32 │ 20.45 / 22.25 ±3.05 / 28.33 ms │ 21.25 / 21.68 ±0.26 / 21.95 ms │ no change │
│ QQuery 33 │ 38.45 / 38.87 ±0.27 / 39.20 ms │ 38.42 / 38.67 ±0.27 / 39.19 ms │ no change │
│ QQuery 34 │ 9.78 / 10.04 ±0.40 / 10.84 ms │ 10.04 / 10.38 ±0.51 / 11.39 ms │ no change │
│ QQuery 35 │ 73.35 / 73.96 ±0.56 / 74.84 ms │ 72.92 / 73.55 ±0.39 / 74.09 ms │ no change │
│ QQuery 36 │ 5.99 / 6.13 ±0.19 / 6.52 ms │ 6.09 / 6.21 ±0.15 / 6.50 ms │ no change │
│ QQuery 37 │ 7.22 / 7.32 ±0.15 / 7.61 ms │ 7.09 / 7.26 ±0.11 / 7.42 ms │ no change │
│ QQuery 38 │ 61.75 / 62.98 ±1.21 / 65.09 ms │ 62.88 / 64.36 ±1.99 / 68.31 ms │ no change │
│ QQuery 39 │ 90.42 / 91.32 ±0.71 / 92.54 ms │ 90.75 / 91.56 ±0.65 / 92.73 ms │ no change │
│ QQuery 40 │ 24.05 / 24.21 ±0.13 / 24.41 ms │ 23.54 / 23.91 ±0.29 / 24.34 ms │ no change │
│ QQuery 41 │ 11.71 / 11.94 ±0.15 / 12.14 ms │ 11.52 / 11.76 ±0.17 / 12.02 ms │ no change │
│ QQuery 42 │ 23.86 / 24.39 ±0.36 / 24.91 ms │ 24.00 / 24.30 ±0.21 / 24.57 ms │ no change │
│ QQuery 43 │ 4.98 / 5.07 ±0.08 / 5.21 ms │ 5.12 / 5.20 ±0.12 / 5.45 ms │ no change │
│ QQuery 44 │ 9.22 / 9.34 ±0.13 / 9.57 ms │ 9.63 / 10.98 ±2.56 / 16.09 ms │ 1.18x slower │
│ QQuery 45 │ 37.67 / 40.03 ±2.89 / 45.59 ms │ 40.52 / 41.03 ±0.38 / 41.42 ms │ no change │
│ QQuery 46 │ 12.68 / 12.91 ±0.26 / 13.30 ms │ 11.99 / 12.48 ±0.32 / 12.86 ms │ no change │
│ QQuery 47 │ 228.57 / 231.67 ±2.14 / 234.84 ms │ 229.16 / 234.02 ±4.51 / 240.95 ms │ no change │
│ QQuery 48 │ 96.75 / 97.72 ±1.02 / 99.63 ms │ 96.31 / 97.29 ±0.66 / 98.27 ms │ no change │
│ QQuery 49 │ 76.79 / 77.44 ±0.50 / 77.94 ms │ 76.82 / 77.73 ±0.81 / 78.93 ms │ no change │
│ QQuery 50 │ 59.13 / 59.87 ±1.09 / 62.00 ms │ 59.14 / 59.46 ±0.24 / 59.83 ms │ no change │
│ QQuery 51 │ 90.87 / 93.20 ±1.73 / 95.49 ms │ 93.40 / 95.57 ±2.38 / 98.78 ms │ no change │
│ QQuery 52 │ 24.25 / 26.32 ±3.73 / 33.77 ms │ 24.68 / 25.06 ±0.58 / 26.20 ms │ no change │
│ QQuery 53 │ 30.19 / 30.64 ±0.34 / 31.13 ms │ 30.47 / 30.61 ±0.18 / 30.94 ms │ no change │
│ QQuery 54 │ 55.98 / 56.53 ±0.49 / 57.28 ms │ 56.27 / 56.42 ±0.12 / 56.60 ms │ no change │
│ QQuery 55 │ 23.79 / 23.93 ±0.11 / 24.12 ms │ 23.82 / 23.89 ±0.08 / 24.05 ms │ no change │
│ QQuery 56 │ 39.35 / 40.14 ±0.69 / 41.17 ms │ 38.96 / 40.87 ±3.17 / 47.20 ms │ no change │
│ QQuery 57 │ 175.55 / 178.41 ±3.33 / 184.92 ms │ 178.96 / 180.10 ±0.74 / 181.15 ms │ no change │
│ QQuery 58 │ 115.95 / 118.18 ±2.11 / 121.49 ms │ 117.11 / 119.01 ±1.30 / 120.97 ms │ no change │
│ QQuery 59 │ 117.66 / 118.25 ±0.45 / 118.97 ms │ 118.11 / 119.90 ±2.07 / 123.82 ms │ no change │
│ QQuery 60 │ 39.70 / 41.07 ±1.04 / 42.92 ms │ 39.90 / 40.87 ±0.61 / 41.82 ms │ no change │
│ QQuery 61 │ 13.54 / 13.68 ±0.09 / 13.79 ms │ 12.36 / 12.65 ±0.23 / 13.06 ms │ +1.08x faster │
│ QQuery 62 │ 49.07 / 49.55 ±0.48 / 50.36 ms │ 46.76 / 46.97 ±0.24 / 47.42 ms │ +1.05x faster │
│ QQuery 63 │ 31.83 / 32.14 ±0.38 / 32.88 ms │ 29.84 / 30.53 ±0.58 / 31.61 ms │ +1.05x faster │
│ QQuery 64 │ 425.13 / 443.76 ±11.52 / 460.80 ms │ 413.28 / 419.54 ±7.40 / 432.12 ms │ +1.06x faster │
│ QQuery 65 │ 150.06 / 155.45 ±7.30 / 169.65 ms │ 143.39 / 146.51 ±2.45 / 150.75 ms │ +1.06x faster │
│ QQuery 66 │ 79.23 / 80.61 ±1.73 / 83.85 ms │ 81.42 / 82.45 ±1.12 / 84.61 ms │ no change │
│ QQuery 67 │ 243.17 / 247.42 ±6.20 / 259.71 ms │ 249.06 / 272.61 ±17.84 / 294.32 ms │ 1.10x slower │
│ QQuery 68 │ 12.36 / 12.71 ±0.53 / 13.75 ms │ 13.47 / 13.91 ±0.62 / 15.12 ms │ 1.09x slower │
│ QQuery 69 │ 58.33 / 58.83 ±0.28 / 59.13 ms │ 61.70 / 62.59 ±0.58 / 63.44 ms │ 1.06x slower │
│ QQuery 70 │ 105.25 / 108.38 ±3.20 / 114.33 ms │ 115.05 / 121.78 ±7.84 / 134.55 ms │ 1.12x slower │
│ QQuery 71 │ 35.44 / 38.24 ±4.39 / 46.94 ms │ 39.03 / 39.49 ±0.38 / 40.04 ms │ no change │
│ QQuery 72 │ 2055.37 / 2224.86 ±192.13 / 2562.07 ms │ 2096.88 / 2166.51 ±63.71 / 2270.18 ms │ no change │
│ QQuery 73 │ 9.80 / 10.04 ±0.15 / 10.26 ms │ 10.52 / 10.76 ±0.18 / 11.04 ms │ 1.07x slower │
│ QQuery 74 │ 172.99 / 176.34 ±2.75 / 180.15 ms │ 211.23 / 213.22 ±1.89 / 216.76 ms │ 1.21x slower │
│ QQuery 75 │ 150.08 / 151.32 ±0.92 / 152.83 ms │ 161.80 / 165.13 ±5.40 / 175.87 ms │ 1.09x slower │
│ QQuery 76 │ 35.79 / 39.04 ±3.69 / 45.73 ms │ 37.25 / 38.10 ±0.56 / 38.69 ms │ no change │
│ QQuery 77 │ 61.72 / 62.32 ±0.41 / 62.72 ms │ 63.53 / 65.39 ±1.54 / 68.09 ms │ no change │
│ QQuery 78 │ 198.55 / 201.76 ±2.64 / 205.69 ms │ 210.76 / 213.11 ±2.01 / 216.58 ms │ 1.06x slower │
│ QQuery 79 │ 67.25 / 68.00 ±0.73 / 69.38 ms │ 70.54 / 73.07 ±3.61 / 80.23 ms │ 1.07x slower │
│ QQuery 80 │ 100.87 / 102.17 ±2.00 / 106.14 ms │ 101.51 / 104.93 ±2.63 / 107.91 ms │ no change │
│ QQuery 81 │ 26.05 / 26.53 ±0.33 / 26.94 ms │ 26.32 / 26.80 ±0.81 / 28.42 ms │ no change │
│ QQuery 82 │ 16.66 / 17.96 ±2.45 / 22.87 ms │ 16.80 / 16.93 ±0.08 / 17.04 ms │ +1.06x faster │
│ QQuery 83 │ 40.20 / 40.55 ±0.24 / 40.94 ms │ 41.11 / 42.96 ±2.71 / 48.31 ms │ 1.06x slower │
│ QQuery 84 │ 30.90 / 31.06 ±0.13 / 31.24 ms │ 30.56 / 31.17 ±0.42 / 31.73 ms │ no change │
│ QQuery 85 │ 107.96 / 110.63 ±2.90 / 115.54 ms │ 107.52 / 108.93 ±1.31 / 111.43 ms │ no change │
│ QQuery 86 │ 25.84 / 26.25 ±0.27 / 26.69 ms │ 25.78 / 25.99 ±0.11 / 26.08 ms │ no change │
│ QQuery 87 │ 63.22 / 64.56 ±1.22 / 66.75 ms │ 63.65 / 65.47 ±2.54 / 70.50 ms │ no change │
│ QQuery 88 │ 64.50 / 66.57 ±3.45 / 73.44 ms │ 63.75 / 64.13 ±0.31 / 64.61 ms │ no change │
│ QQuery 89 │ 36.92 / 37.61 ±0.49 / 38.19 ms │ 36.32 / 37.32 ±0.92 / 38.81 ms │ no change │
│ QQuery 90 │ 17.66 / 18.03 ±0.29 / 18.43 ms │ 17.55 / 17.83 ±0.20 / 18.15 ms │ no change │
│ QQuery 91 │ 47.49 / 47.64 ±0.16 / 47.93 ms │ 46.93 / 48.23 ±2.06 / 52.31 ms │ no change │
│ QQuery 92 │ 30.26 / 30.96 ±0.55 / 31.89 ms │ 30.37 / 31.18 ±0.80 / 32.24 ms │ no change │
│ QQuery 93 │ 50.83 / 52.10 ±1.44 / 54.69 ms │ 50.30 / 50.98 ±0.63 / 52.03 ms │ no change │
│ QQuery 94 │ 38.84 / 39.28 ±0.31 / 39.70 ms │ 38.63 / 39.51 ±1.00 / 41.42 ms │ no change │
│ QQuery 95 │ 81.05 / 82.15 ±0.62 / 82.94 ms │ 82.48 / 84.84 ±2.02 / 88.00 ms │ no change │
│ QQuery 96 │ 24.39 / 24.75 ±0.35 / 25.34 ms │ 24.62 / 24.80 ±0.28 / 25.34 ms │ no change │
│ QQuery 97 │ 46.47 / 48.44 ±2.41 / 53.13 ms │ 46.86 / 47.46 ±0.57 / 48.54 ms │ no change │
│ QQuery 98 │ 42.63 / 43.31 ±0.50 / 43.97 ms │ 41.94 / 42.85 ±0.62 / 43.87 ms │ no change │
│ QQuery 99 │ 69.91 / 70.53 ±0.69 / 71.77 ms │ 71.70 / 73.14 ±2.05 / 77.15 ms │ no change │
└───────────┴────────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 10147.32ms │
│ Total Time (fast-batch-wise-skip) │ 10307.09ms │
│ Average Time (HEAD) │ 102.50ms │
│ Average Time (fast-batch-wise-skip) │ 104.11ms │
│ Queries Faster │ 8 │
│ Queries Slower │ 14 │
│ Queries with No Change │ 77 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘

Resource Usage

tpcds — base (merge-base)

MetricValue
Wall time55.0s
Peak memory2.0 GiB
Avg memory1.5 GiB
CPU user228.4s
CPU sys6.2s
Peak spill0 B

tpcds — branch

MetricValue
Wall time55.0s
Peak memory2.2 GiB
Avg memory1.5 GiB
CPU user232.7s
CPU sys6.3s
Peak spill0 B

File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

group HEAD fast-batch-wise-skip
----- ---- --------------------
bench_merge_sorted_preserving/multiple_large_string_columns_with_1m_rows 1.00 1788.1±47.71ms ? ?/sec 1.01 1808.3±63.59ms ? ?/sec
bench_merge_sorted_preserving/multiple_u64_columns_with_1m_rows 1.10 77.0±3.72ms ? ?/sec 1.00 69.9±5.47ms ? ?/sec
bench_merge_sorted_preserving/single_large_string_column_with_1m_rows 1.00 250.5±2.72ms ? ?/sec 1.03 258.4±7.39ms ? ?/sec
bench_merge_sorted_preserving/single_u64_column_with_1m_rows 1.00 30.7±0.17ms ? ?/sec 1.05 32.3±0.22ms ? ?/sec

Resource Usage

sort_preserving_merge — base (merge-base)

MetricValue
Wall time405.1s
Peak memory6.3 GiB
Avg memory2.3 GiB
CPU user122.8s
CPU sys122.8s
Peak spill0 B

sort_preserving_merge — branch

MetricValue
Wall time405.1s
Peak memory6.3 GiB
Avg memory2.3 GiB
CPU user123.0s
CPU sys124.6s
Peak spill0 B

File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and fast-batch-wise-skip
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ fast-batch-wise-skip ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.23 / 4.07 ±5.53 / 15.13 ms │ 1.20 / 4.03 ±5.51 / 15.04 ms │ no change │
│ QQuery 1 │ 12.77 / 12.87 ±0.11 / 13.07 ms │ 12.37 / 12.72 ±0.24 / 13.06 ms │ no change │
│ QQuery 2 │ 35.55 / 35.95 ±0.45 / 36.81 ms │ 35.78 / 35.98 ±0.27 / 36.52 ms │ no change │
│ QQuery 3 │ 30.78 / 31.66 ±0.79 / 32.95 ms │ 30.32 / 30.53 ±0.14 / 30.74 ms │ no change │
│ QQuery 4 │ 224.62 / 228.73 ±2.97 / 231.78 ms │ 223.07 / 227.70 ±3.05 / 231.70 ms │ no change │
│ QQuery 5 │ 276.29 / 279.56 ±2.56 / 282.61 ms │ 272.24 / 275.62 ±2.09 / 278.05 ms │ no change │
│ QQuery 6 │ 1.27 / 1.42 ±0.23 / 1.86 ms │ 1.24 / 1.40 ±0.23 / 1.84 ms │ no change │
│ QQuery 7 │ 13.77 / 13.90 ±0.09 / 14.03 ms │ 13.51 / 13.92 ±0.23 / 14.15 ms │ no change │
│ QQuery 8 │ 326.23 / 328.80 ±2.12 / 331.30 ms │ 322.59 / 325.98 ±3.78 / 332.95 ms │ no change │
│ QQuery 9 │ 463.58 / 469.96 ±4.70 / 477.94 ms │ 460.87 / 470.11 ±8.58 / 485.24 ms │ no change │
│ QQuery 10 │ 71.37 / 72.02 ±0.50 / 72.60 ms │ 70.57 / 71.77 ±1.21 / 73.96 ms │ no change │
│ QQuery 11 │ 81.75 / 82.81 ±1.09 / 84.26 ms │ 81.69 / 82.77 ±1.35 / 84.92 ms │ no change │
│ QQuery 12 │ 268.95 / 274.42 ±6.00 / 285.41 ms │ 267.09 / 271.04 ±2.85 / 275.63 ms │ no change │
│ QQuery 13 │ 366.91 / 376.31 ±6.78 / 386.61 ms │ 372.84 / 378.10 ±2.69 / 380.09 ms │ no change │
│ QQuery 14 │ 284.34 / 286.22 ±1.42 / 288.37 ms │ 285.25 / 289.83 ±5.38 / 296.59 ms │ no change │
│ QQuery 15 │ 276.07 / 279.74 ±2.39 / 282.89 ms │ 274.76 / 282.83 ±5.52 / 291.55 ms │ no change │
│ QQuery 16 │ 619.51 / 624.30 ±4.39 / 632.45 ms │ 618.72 / 659.53 ±50.77 / 759.43 ms │ 1.06x slower │
│ QQuery 17 │ 626.24 / 633.69 ±7.56 / 642.92 ms │ 628.93 / 659.78 ±51.09 / 761.80 ms │ no change │
│ QQuery 18 │ 1256.42 / 1287.88 ±19.04 / 1315.08 ms │ 1274.48 / 1286.35 ±7.12 / 1295.86 ms │ no change │
│ QQuery 19 │ 27.97 / 31.08 ±4.96 / 40.90 ms │ 27.93 / 33.35 ±9.89 / 53.12 ms │ 1.07x slower │
│ QQuery 20 │ 514.00 / 523.96 ±12.32 / 547.79 ms │ 514.94 / 532.56 ±17.45 / 562.20 ms │ no change │
│ QQuery 21 │ 517.49 / 522.42 ±3.83 / 528.10 ms │ 514.56 / 522.29 ±7.26 / 535.58 ms │ no change │
│ QQuery 22 │ 995.42 / 1004.69 ±9.57 / 1022.21 ms │ 997.33 / 1003.76 ±3.56 / 1007.13 ms │ no change │
│ QQuery 23 │ 3113.78 / 3138.93 ±23.02 / 3171.14 ms │ 3125.86 / 3146.74 ±14.74 / 3165.05 ms │ no change │
│ QQuery 24 │ 41.53 / 44.88 ±5.43 / 55.66 ms │ 42.24 / 47.56 ±10.27 / 68.09 ms │ 1.06x slower │
│ QQuery 25 │ 112.60 / 119.53 ±11.27 / 141.99 ms │ 113.70 / 119.15 ±5.28 / 128.00 ms │ no change │
│ QQuery 26 │ 41.97 / 42.77 ±0.67 / 43.64 ms │ 42.67 / 44.63 ±3.03 / 50.67 ms │ no change │
│ QQuery 27 │ 667.39 / 674.92 ±4.73 / 681.77 ms │ 682.56 / 691.28 ±8.10 / 702.75 ms │ no change │
│ QQuery 28 │ 3042.94 / 3073.61 ±20.44 / 3101.79 ms │ 3062.10 / 3093.41 ±28.08 / 3129.48 ms │ no change │
│ QQuery 29 │ 40.77 / 67.49 ±17.07 / 88.20 ms │ 41.51 / 51.65 ±19.49 / 90.64 ms │ +1.31x faster │
│ QQuery 30 │ 303.67 / 307.63 ±3.99 / 314.73 ms │ 309.20 / 314.09 ±4.17 / 320.61 ms │ no change │
│ QQuery 31 │ 282.77 / 296.24 ±14.91 / 325.17 ms │ 284.27 / 304.20 ±19.57 / 339.81 ms │ no change │
│ QQuery 32 │ 919.34 / 970.38 ±48.43 / 1048.25 ms │ 971.31 / 994.30 ±15.14 / 1011.14 ms │ no change │
│ QQuery 33 │ 1457.29 / 1505.26 ±30.34 / 1541.67 ms │ 1535.60 / 1573.03 ±33.81 / 1620.85 ms │ no change │
│ QQuery 34 │ 1476.09 / 1530.91 ±74.90 / 1673.78 ms │ 1581.87 / 1603.11 ±15.91 / 1621.36 ms │ no change │
│ QQuery 35 │ 279.74 / 316.35 ±38.71 / 389.13 ms │ 321.33 / 326.91 ±7.55 / 341.79 ms │ no change │
│ QQuery 36 │ 66.28 / 76.64 ±8.21 / 89.74 ms │ 68.20 / 74.90 ±4.45 / 81.10 ms │ no change │
│ QQuery 37 │ 35.66 / 36.53 ±0.85 / 37.95 ms │ 37.96 / 46.44 ±6.55 / 58.09 ms │ 1.27x slower │
│ QQuery 38 │ 42.24 / 45.02 ±3.81 / 52.57 ms │ 42.19 / 45.40 ±2.87 / 50.53 ms │ no change │
│ QQuery 39 │ 134.33 / 150.42 ±10.03 / 162.35 ms │ 142.68 / 159.06 ±9.09 / 170.21 ms │ 1.06x slower │
│ QQuery 40 │ 13.88 / 14.19 ±0.47 / 15.12 ms │ 15.28 / 17.45 ±3.52 / 24.46 ms │ 1.23x slower │
│ QQuery 41 │ 13.44 / 13.63 ±0.12 / 13.79 ms │ 14.55 / 15.09 ±0.44 / 15.54 ms │ 1.11x slower │
│ QQuery 42 │ 13.26 / 14.54 ±2.10 / 18.71 ms │ 13.95 / 14.25 ±0.24 / 14.50 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 19846.35ms │
│ Total Time (fast-batch-wise-skip) │ 20154.61ms │
│ Average Time (HEAD) │ 461.54ms │
│ Average Time (fast-batch-wise-skip) │ 468.71ms │
│ Queries Faster │ 1 │
│ Queries Slower │ 7 │
│ Queries with No Change │ 35 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘

Resource Usage

clickbench_partitioned — base (merge-base)

MetricValue
Wall time100.0s
Peak memory12.1 GiB
Avg memory4.6 GiB
CPU user1019.6s
CPU sys69.1s
Peak spill0 B

clickbench_partitioned — branch

MetricValue
Wall time105.0s
Peak memory12.7 GiB
Avg memory4.4 GiB
CPU user1035.3s
CPU sys73.1s
Peak spill0 B

File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark sort_preserving_merge

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4896133554-873-8pdqc 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (1d5183b) to 0365d3c (merge-base) diff using: sort_preserving_merge
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

group HEAD fast-batch-wise-skip
----- ---- --------------------
bench_merge_sorted_preserving/multiple_large_string_columns_with_1m_rows 1.00 1753.5±10.15ms ? ?/sec 1.01 1779.1±10.65ms ? ?/sec
bench_merge_sorted_preserving/multiple_u64_columns_with_1m_rows 1.00 65.5±0.52ms ? ?/sec 1.03 67.4±0.33ms ? ?/sec
bench_merge_sorted_preserving/single_large_string_column_with_1m_rows 1.00 252.0±2.29ms ? ?/sec 1.02 257.4±3.32ms ? ?/sec
bench_merge_sorted_preserving/single_u64_column_with_1m_rows 1.00 30.8±0.16ms ? ?/sec 1.05 32.5±0.27ms ? ?/sec
spm/nearly_sorted/complex/payload_100 1.00 48.7±0.18ms ? ?/sec
spm/nearly_sorted/complex/payload_20 1.00 18.8±0.09ms ? ?/sec
spm/nearly_sorted/complex/payload_5 1.00 13.1±0.02ms ? ?/sec
spm/nearly_sorted/string/payload_100 1.00 46.4±0.08ms ? ?/sec
spm/nearly_sorted/string/payload_20 1.00 16.5±0.02ms ? ?/sec
spm/nearly_sorted/string/payload_5 1.00 10.9±0.01ms ? ?/sec
spm/nearly_sorted/u64/payload_100 1.00 41.6±0.08ms ? ?/sec
spm/nearly_sorted/u64/payload_20 1.00 11.9±0.02ms ? ?/sec
spm/nearly_sorted/u64/payload_5 1.00 6.3±0.01ms ? ?/sec
spm/reverse/complex/payload_100 1.00 16.0±0.03ms ? ?/sec
spm/reverse/complex/payload_20 1.00 7.4±0.02ms ? ?/sec
spm/reverse/complex/payload_5 1.00 5.8±0.01ms ? ?/sec
spm/reverse/string/payload_100 1.00 12.9±0.03ms ? ?/sec
spm/reverse/string/payload_20 1.00 4.4±0.01ms ? ?/sec
spm/reverse/string/payload_5 1.00 2.7±0.00ms ? ?/sec
spm/reverse/u64/payload_100 1.00 11.7±0.03ms ? ?/sec
spm/reverse/u64/payload_20 1.00 3.2±0.00ms ? ?/sec
spm/reverse/u64/payload_5 1.00 1640.6±7.76µs ? ?/sec
spm/sorted/complex/payload_100 1.00 4.8±0.01ms ? ?/sec
spm/sorted/complex/payload_20 1.00 3.7±0.01ms ? ?/sec
spm/sorted/complex/payload_5 1.00 3.4±0.03ms ? ?/sec
spm/sorted/string/payload_100 1.00 1448.1±7.46µs ? ?/sec
spm/sorted/string/payload_20 1.00 320.1±1.41µs ? ?/sec
spm/sorted/string/payload_5 1.00 116.3±0.38µs ? ?/sec
spm/sorted/u64/payload_100 1.00 1447.3±12.29µs ? ?/sec
spm/sorted/u64/payload_20 1.00 309.2±2.31µs ? ?/sec
spm/sorted/u64/payload_5 1.00 106.5±0.72µs ? ?/sec
spm/unsorted/complex/payload_100 1.00 55.8±0.08ms ? ?/sec
spm/unsorted/complex/payload_20 1.00 26.0±0.17ms ? ?/sec
spm/unsorted/complex/payload_5 1.00 20.2±0.03ms ? ?/sec
spm/unsorted/string/payload_100 1.00 52.2±0.14ms ? ?/sec
spm/unsorted/string/payload_20 1.00 22.4±0.04ms ? ?/sec
spm/unsorted/string/payload_5 1.00 16.6±0.03ms ? ?/sec
spm/unsorted/u64/payload_100 1.00 46.2±0.08ms ? ?/sec
spm/unsorted/u64/payload_20 1.00 16.5±0.03ms ? ?/sec
spm/unsorted/u64/payload_5 1.00 10.9±0.03ms ? ?/sec

Resource Usage

sort_preserving_merge — base (merge-base)

MetricValue
Wall time350.1s
Peak memory6.3 GiB
Avg memory2.6 GiB
CPU user119.3s
CPU sys121.2s
Peak spill0 B

sort_preserving_merge — branch

MetricValue
Wall time740.2s
Peak memory6.4 GiB
Avg memory1.4 GiB
CPU user594.2s
CPU sys124.1s
Peak spill0 B

File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark sort_preserving_merge

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4896500200-874-bl5dv 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (5a5e1f0) to 0365d3c (merge-base) diff using: sort_preserving_merge
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

group HEAD fast-batch-wise-skip
----- ---- --------------------
bench_merge_sorted_preserving/multiple_large_string_columns_with_1m_rows 1.00 1814.1±22.63ms ? ?/sec 1.00 1813.9±23.47ms ? ?/sec
bench_merge_sorted_preserving/multiple_u64_columns_with_1m_rows 1.00 65.4±0.38ms ? ?/sec 1.03 67.3±3.50ms ? ?/sec
bench_merge_sorted_preserving/single_large_string_column_with_1m_rows 1.00 255.0±2.10ms ? ?/sec 1.00 255.6±3.78ms ? ?/sec
bench_merge_sorted_preserving/single_u64_column_with_1m_rows 1.00 30.6±0.19ms ? ?/sec 1.01 30.9±0.28ms ? ?/sec
spm/nearly_sorted/complex/payload_100 1.00 48.2±0.09ms ? ?/sec
spm/nearly_sorted/complex/payload_20 1.00 18.7±0.07ms ? ?/sec
spm/nearly_sorted/complex/payload_5 1.00 13.1±0.11ms ? ?/sec
spm/nearly_sorted/string/payload_100 1.00 45.5±0.11ms ? ?/sec
spm/nearly_sorted/string/payload_20 1.00 16.0±0.05ms ? ?/sec
spm/nearly_sorted/string/payload_5 1.00 10.4±0.07ms ? ?/sec
spm/nearly_sorted/u64/payload_100 1.00 41.4±0.08ms ? ?/sec
spm/nearly_sorted/u64/payload_20 1.00 11.8±0.08ms ? ?/sec
spm/nearly_sorted/u64/payload_5 1.00 6.1±0.02ms ? ?/sec
spm/reverse/complex/payload_100 1.00 48.1±0.13ms ? ?/sec
spm/reverse/complex/payload_20 1.00 18.2±0.04ms ? ?/sec
spm/reverse/complex/payload_5 1.00 12.7±0.15ms ? ?/sec
spm/reverse/string/payload_100 1.00 45.4±0.15ms ? ?/sec
spm/reverse/string/payload_20 1.00 15.7±0.05ms ? ?/sec
spm/reverse/string/payload_5 1.00 10.1±0.02ms ? ?/sec
spm/reverse/u64/payload_100 1.00 41.3±0.14ms ? ?/sec
spm/reverse/u64/payload_20 1.00 11.7±0.08ms ? ?/sec
spm/reverse/u64/payload_5 1.00 6.1±0.07ms ? ?/sec
spm/sorted/complex/payload_100 1.00 48.0±0.12ms ? ?/sec
spm/sorted/complex/payload_20 1.00 18.3±0.16ms ? ?/sec
spm/sorted/complex/payload_5 1.00 12.8±0.12ms ? ?/sec
spm/sorted/string/payload_100 1.00 45.5±0.10ms ? ?/sec
spm/sorted/string/payload_20 1.00 15.7±0.03ms ? ?/sec
spm/sorted/string/payload_5 1.00 10.2±0.07ms ? ?/sec
spm/sorted/u64/payload_100 1.00 41.2±0.12ms ? ?/sec
spm/sorted/u64/payload_20 1.00 11.6±0.07ms ? ?/sec
spm/sorted/u64/payload_5 1.00 6.0±0.02ms ? ?/sec
spm/unsorted/complex/payload_100 1.00 55.9±0.16ms ? ?/sec
spm/unsorted/complex/payload_20 1.00 25.8±0.04ms ? ?/sec
spm/unsorted/complex/payload_5 1.00 20.0±0.10ms ? ?/sec
spm/unsorted/string/payload_100 1.00 51.6±0.12ms ? ?/sec
spm/unsorted/string/payload_20 1.00 21.9±0.03ms ? ?/sec
spm/unsorted/string/payload_5 1.00 16.2±0.04ms ? ?/sec
spm/unsorted/u64/payload_100 1.00 45.8±0.06ms ? ?/sec
spm/unsorted/u64/payload_20 1.00 16.1±0.05ms ? ?/sec
spm/unsorted/u64/payload_5 1.00 10.5±0.06ms ? ?/sec

Resource Usage

sort_preserving_merge — base (merge-base)

MetricValue
Wall time355.1s
Peak memory6.3 GiB
Avg memory2.6 GiB
CPU user120.2s
CPU sys127.4s
Peak spill0 B

sort_preserving_merge — branch

MetricValue
Wall time800.2s
Peak memory6.3 GiB
Avg memory1.4 GiB
CPU user608.1s
CPU sys127.4s
Peak spill0 B

File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark clickbench_sorted

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4896780949-875-vww69 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (80d85bf) to 0365d3c (merge-base) diff using: clickbench_sorted
Results will be posted here when complete


File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark sort_tpch10

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4896828325-876-rt6zg 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (80d85bf) to 0365d3c (merge-base) diff using: sort_tpch10
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and fast-batch-wise-skip
--------------------
Benchmark sort_tpch10.json
--------------------
┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ fast-batch-wise-skip ┃ Change ┃
┡━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ Q1 │ 1631.98 / 1643.06 ±8.49 / 1658.03 ms │ 1396.34 / 1418.26 ±17.19 / 1437.64 ms │ +1.16x faster │
│ Q2 │ 1421.45 / 1431.48 ±10.65 / 1451.01 ms │ 201.29 / 208.62 ±6.77 / 217.57 ms │ +6.86x faster │
│ Q3 │ 8893.78 / 9189.49 ±218.44 / 9425.68 ms │ 8880.60 / 9062.28 ±184.88 / 9413.73 ms │ no change │
│ Q4 │ 2741.65 / 2770.88 ±16.58 / 2789.94 ms │ 392.40 / 423.10 ±20.71 / 453.79 ms │ +6.55x faster │
│ Q5 │ 2644.71 / 2680.47 ±22.80 / 2708.24 ms │ 2700.25 / 2725.43 ±19.05 / 2745.08 ms │ no change │
│ Q6 │ 2819.27 / 2853.20 ±33.86 / 2901.49 ms │ 2827.71 / 2850.04 ±17.91 / 2878.97 ms │ no change │
│ Q7 │ 5602.34 / 5670.57 ±63.47 / 5780.55 ms │ 5569.72 / 5676.77 ±82.71 / 5824.64 ms │ no change │
│ Q8 │ 3978.51 / 4065.05 ±75.19 / 4176.24 ms │ 3456.58 / 3553.36 ±82.71 / 3672.65 ms │ +1.14x faster │
│ Q9 │ 4369.52 / 4443.83 ±58.81 / 4508.15 ms │ 3719.71 / 3864.12 ±88.71 / 3970.66 ms │ +1.15x faster │
│ Q10 │ 7035.70 / 7185.30 ±128.68 / 7353.68 ms │ 5843.73 / 6066.70 ±184.81 / 6299.54 ms │ +1.18x faster │
│ Q11 │ 3674.51 / 3745.29 ±64.70 / 3854.17 ms │ 3834.03 / 3984.65 ±202.64 / 4361.24 ms │ 1.06x slower │
└───────┴────────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 45678.61ms │
│ Total Time (fast-batch-wise-skip) │ 39833.34ms │
│ Average Time (HEAD) │ 4152.60ms │
│ Average Time (fast-batch-wise-skip) │ 3621.21ms │
│ Queries Faster │ 6 │
│ Queries Slower │ 1 │
│ Queries with No Change │ 4 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴────────────┘

Resource Usage

sort_tpch10 — base (merge-base)

MetricValue
Wall time230.1s
Peak memory19.0 GiB
Avg memory7.5 GiB
CPU user909.4s
CPU sys70.8s
Peak spill0 B

sort_tpch10 — branch

MetricValue
Wall time200.0s
Peak memory20.8 GiB
Avg memory8.9 GiB
CPU user828.9s
CPU sys51.8s
Peak spill0 B

File an issue against this benchmark runner

@rluvaton
rluvaton marked this pull request as ready for review July 6, 2026 21:01
@rluvatonrluvaton added the performance Make DataFusion faster label Jul 6, 2026
@rluvatonrluvaton changed the title perf(sort): fast path for full batch when it sorts before all other streams in SortPreservingMergeStreamperf(sort): fast path for full batch when it sorts before all other streams in SortPreservingMergeStreamJul 6, 2026
@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark tpch10

env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G

@adriangbot

Copy link
Copy Markdown

Hi @rluvaton, your benchmark configuration could not be parsed (#23345 (comment)).

Error:invalid configuration: found character that cannot start any token at line 2 column 1, while scanning for the next token

Usage:

run benchmark <name> # run specific benchmark(s)
run benchmarks # run default suite
run benchmarks <name1> <name2> # run specific benchmarks

Any benchmark name is accepted: bench.sh suite names (e.g. tpch, clickbench_partitioned, wide_schema) and Criterion bench targets (e.g. sql_planner) are resolved automatically. A name that matches neither fails on the runner.

Per-side configuration (run benchmark tpch followed by):

env:
# shared env is inherited by BOTH the build and the run, so build# flags go here. Builds default to no debuginfo for speed; opt back# in for hung-job gdb dumps and cap jobs to stay within memory:CARGO_PROFILE_RELEASE_DEBUG: "1"CARGO_BUILD_JOBS: "1"baseline:
ref: v45.0.0env:
# per-side env only reaches the benchmark run, not the buildDATAFUSION_RUNTIME_MEMORY_LIMIT: 1Gchanged:
ref: v46.0.0env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G

File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark tpch10

baseline:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G
changed:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4902817145-892-j4g6f 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (92505cf) to 0365d3c (merge-base) diff using: tpch10
Results will be posted here when complete


File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

show benchmark queue

@adriangbot

Copy link
Copy Markdown

Hi @rluvaton, you asked to view the benchmark queue (#23345 (comment)).

CommentRepoPRUserBenchmarksStatus
#4902817145apache/datafusion#23345rluvaton["tpch10"]running

File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

Benchmark for this request hit the 7200s job deadline before finishing.

Benchmarks requested: tpch10

Kubernetes message
Job was active longer than specified deadline

File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark tpch10

baseline:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 4G
changed:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 4G

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4904293557-894-h7jgf 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (92505cf) to 0365d3c (merge-base) diff using: tpch10
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and fast-batch-wise-skip
--------------------
Benchmark tpch_sf10.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ fast-batch-wise-skip ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1 │ 321.53 / 323.77 ±2.07 / 327.47 ms │ 321.15 / 322.54 ±0.79 / 323.61 ms │ no change │
│ QQuery 2 │ 108.18 / 111.42 ±2.92 / 116.60 ms │ 107.60 / 111.95 ±4.09 / 119.52 ms │ no change │
│ QQuery 3 │ 246.45 / 252.19 ±2.93 / 254.52 ms │ 248.00 / 250.94 ±2.38 / 253.80 ms │ no change │
│ QQuery 4 │ 119.54 / 121.41 ±1.51 / 123.41 ms │ 119.94 / 123.13 ±2.57 / 127.01 ms │ no change │
│ QQuery 5 │ 375.11 / 386.63 ±7.00 / 395.98 ms │ 377.79 / 390.02 ±8.50 / 400.51 ms │ no change │
│ QQuery 6 │ 128.33 / 129.91 ±2.52 / 134.92 ms │ 130.05 / 132.98 ±4.03 / 140.90 ms │ no change │
│ QQuery 7 │ 495.13 / 500.49 ±5.72 / 510.55 ms │ 495.44 / 509.62 ±10.64 / 526.66 ms │ no change │
│ QQuery 8 │ 397.36 / 401.64 ±5.73 / 412.82 ms │ 404.83 / 410.83 ±4.71 / 418.28 ms │ no change │
│ QQuery 9 │ 572.08 / 578.18 ±5.67 / 587.95 ms │ 597.85 / 610.96 ±9.21 / 621.92 ms │ 1.06x slower │
│ QQuery 10 │ 316.38 / 324.01 ±12.45 / 348.74 ms │ 317.13 / 331.49 ±12.66 / 354.45 ms │ no change │
│ QQuery 11 │ 90.93 / 93.93 ±2.14 / 96.32 ms │ 94.08 / 99.50 ±8.01 / 115.31 ms │ 1.06x slower │
│ QQuery 12 │ 181.26 / 185.21 ±2.70 / 188.79 ms │ 185.13 / 191.86 ±6.35 / 202.10 ms │ no change │
│ QQuery 13 │ 306.61 / 316.12 ±9.69 / 330.25 ms │ 317.41 / 325.82 ±9.19 / 342.70 ms │ no change │
│ QQuery 14 │ 177.22 / 182.92 ±6.72 / 192.46 ms │ 183.42 / 186.20 ±2.97 / 191.17 ms │ no change │
│ QQuery 15 │ 315.46 / 318.39 ±3.02 / 324.19 ms │ 319.76 / 326.54 ±3.93 / 330.63 ms │ no change │
│ QQuery 16 │ 67.70 / 69.33 ±1.81 / 72.70 ms │ 68.10 / 74.87 ±7.58 / 89.05 ms │ 1.08x slower │
│ QQuery 17 │ 660.32 / 675.38 ±10.02 / 691.93 ms │ 694.86 / 712.80 ±13.76 / 733.59 ms │ 1.06x slower │
│ QQuery 18 │ 818.54 / 835.52 ±13.27 / 856.21 ms │ 820.47 / 845.86 ±23.43 / 884.79 ms │ no change │
│ QQuery 19 │ 257.94 / 268.36 ±9.28 / 285.35 ms │ 254.47 / 266.02 ±12.01 / 287.87 ms │ no change │
│ QQuery 20 │ 285.84 / 304.79 ±11.98 / 319.92 ms │ 283.26 / 303.23 ±12.78 / 316.11 ms │ no change │
│ QQuery 21 │ 705.35 / 716.46 ±8.38 / 728.58 ms │ 677.33 / 687.71 ±5.87 / 693.47 ms │ no change │
│ QQuery 22 │ 64.40 / 67.68 ±2.33 / 70.51 ms │ 65.10 / 66.46 ±1.22 / 68.64 ms │ no change │
└───────────┴────────────────────────────────────┴────────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD) │ 7163.75ms │
│ Total Time (fast-batch-wise-skip) │ 7281.31ms │
│ Average Time (HEAD) │ 325.62ms │
│ Average Time (fast-batch-wise-skip) │ 330.97ms │
│ Queries Faster │ 0 │
│ Queries Slower │ 4 │
│ Queries with No Change │ 18 │
│ Queries with Failure │ 0 │
└─────────────────────────────────────┴───────────┘

Resource Usage

tpch10 — base (merge-base)

MetricValue
Wall time40.0s
Peak memory5.4 GiB
Avg memory1.5 GiB
CPU user362.9s
CPU sys22.3s
Peak spill0 B

tpch10 — branch

MetricValue
Wall time40.0s
Peak memory5.8 GiB
Avg memory1.8 GiB
CPU user368.0s
CPU sys22.7s
Peak spill0 B

File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark tpch10

baseline:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G
DATAFUSION_EXECUTION_TARGET_PARTITIONS: 8
changed:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G
DATAFUSION_EXECUTION_TARGET_PARTITIONS: 8

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4904647847-895-8tzbq 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (92505cf) to 0365d3c (merge-base) diff using: tpch10
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and fast-batch-wise-skip
--------------------
Benchmark tpch_sf10.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ fast-batch-wise-skip ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1 │ 459.46 / 460.66 ±0.83 / 461.90 ms │ 459.37 / 462.02 ±2.57 / 466.07 ms │ no change │
│ QQuery 2 │ 117.02 / 119.54 ±1.45 / 120.97 ms │ 117.84 / 119.20 ±0.84 / 120.26 ms │ no change │
│ QQuery 3 │ 296.73 / 301.91 ±3.62 / 306.34 ms │ 300.85 / 303.19 ±2.52 / 307.90 ms │ no change │
│ QQuery 4 │ 138.84 / 140.16 ±1.34 / 142.61 ms │ 138.49 / 141.39 ±2.89 / 146.83 ms │ no change │
│ QQuery 5 │ 404.75 / 408.43 ±2.51 / 411.93 ms │ 404.95 / 410.95 ±4.93 / 416.93 ms │ no change │
│ QQuery 6 │ 180.81 / 183.16 ±3.59 / 190.30 ms │ 183.78 / 185.97 ±2.70 / 191.18 ms │ no change │
│ QQuery 7 │ 523.26 / 530.23 ±4.88 / 536.98 ms │ 515.85 / 522.06 ±4.91 / 529.07 ms │ no change │
│ QQuery 8 │ 529.86 / 536.42 ±4.74 / 544.10 ms │ 532.16 / 535.07 ±2.15 / 538.01 ms │ no change │
│ QQuery 9 │ 711.34 / 721.01 ±5.89 / 728.76 ms │ 709.60 / 714.64 ±4.25 / 722.15 ms │ no change │
│ QQuery 10 │ 372.27 / 375.43 ±4.12 / 383.25 ms │ 362.86 / 380.34 ±14.22 / 402.37 ms │ no change │
│ QQuery 11 │ 98.96 / 106.92 ±11.79 / 130.38 ms │ 98.04 / 103.74 ±8.09 / 119.79 ms │ no change │
│ QQuery 12 │ 241.65 / 243.90 ±2.95 / 249.47 ms │ 239.06 / 245.73 ±4.70 / 251.18 ms │ no change │
│ QQuery 13 │ 340.86 / 351.30 ±7.68 / 360.02 ms │ 330.42 / 346.73 ±10.77 / 359.95 ms │ no change │
│ QQuery 14 │ 249.43 / 253.89 ±5.29 / 263.58 ms │ 248.76 / 253.01 ±5.02 / 261.39 ms │ no change │
│ QQuery 15 │ 444.45 / 445.41 ±0.86 / 446.72 ms │ 441.93 / 444.80 ±2.28 / 446.95 ms │ no change │
│ QQuery 16 │ 167.49 / 170.52 ±2.30 / 174.58 ms │ 159.82 / 166.00 ±7.11 / 179.59 ms │ no change │
│ QQuery 17 │ 773.72 / 791.65 ±18.90 / 826.71 ms │ 753.51 / 777.03 ±13.36 / 792.11 ms │ no change │
│ QQuery 18 │ FAIL │ FAIL │ incomparable │
│ QQuery 19 │ 353.09 / 362.38 ±15.38 / 393.01 ms │ 354.32 / 360.99 ±4.15 / 366.87 ms │ no change │
│ QQuery 20 │ 515.90 / 523.34 ±3.82 / 526.89 ms │ 513.19 / 517.04 ±1.98 / 518.43 ms │ no change │
│ QQuery 21 │ 797.57 / 807.02 ±4.90 / 810.69 ms │ 798.68 / 807.63 ±8.52 / 823.40 ms │ no change │
│ QQuery 22 │ 81.18 / 86.38 ±5.31 / 95.89 ms │ 81.29 / 83.05 ±1.24 / 84.85 ms │ no change │
└───────────┴────────────────────────────────────┴────────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD) │ 7919.66ms │
│ Total Time (fast-batch-wise-skip) │ 7880.59ms │
│ Average Time (HEAD) │ 377.13ms │
│ Average Time (fast-batch-wise-skip) │ 375.27ms │
│ Queries Faster │ 0 │
│ Queries Slower │ 0 │
│ Queries with No Change │ 21 │
│ Queries with Failure │ 1 │
└─────────────────────────────────────┴───────────┘

Resource Usage

tpch10 — base (merge-base)

MetricValue
Wall time45.0s
Peak memory2.3 GiB
Avg memory985.1 MiB
CPU user329.4s
CPU sys20.0s
Peak spill0 B

tpch10 — branch

MetricValue
Wall time45.0s
Peak memory2.3 GiB
Avg memory1016.5 MiB
CPU user328.2s
CPU sys20.4s
Peak spill0 B

File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark tpch10

baseline:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G
DATAFUSION_EXECUTION_TARGET_PARTITIONS: 6
changed:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G
DATAFUSION_EXECUTION_TARGET_PARTITIONS: 6

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4904814635-896-tm4cz 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (92505cf) to 0365d3c (merge-base) diff using: tpch10
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and fast-batch-wise-skip
--------------------
Benchmark tpch_sf10.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ fast-batch-wise-skip ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1 │ 599.77 / 604.11 ±2.38 / 606.66 ms │ 609.99 / 611.53 ±1.19 / 612.94 ms │ no change │
│ QQuery 2 │ 141.66 / 147.71 ±5.62 / 158.16 ms │ 140.26 / 143.32 ±2.42 / 146.59 ms │ no change │
│ QQuery 3 │ 390.80 / 400.76 ±5.31 / 406.81 ms │ 381.55 / 388.27 ±4.08 / 394.20 ms │ no change │
│ QQuery 4 │ 192.47 / 196.49 ±4.62 / 205.10 ms │ 189.03 / 191.77 ±2.02 / 194.61 ms │ no change │
│ QQuery 5 │ 541.57 / 549.46 ±7.31 / 558.70 ms │ 526.63 / 530.54 ±2.52 / 534.49 ms │ no change │
│ QQuery 6 │ 239.02 / 240.96 ±2.10 / 244.95 ms │ 237.61 / 239.77 ±2.62 / 244.58 ms │ no change │
│ QQuery 7 │ 580.70 / 590.44 ±8.74 / 603.88 ms │ 573.75 / 586.52 ±8.05 / 598.33 ms │ no change │
│ QQuery 8 │ 676.04 / 680.30 ±4.83 / 687.97 ms │ 666.54 / 672.98 ±5.19 / 681.60 ms │ no change │
│ QQuery 9 │ 856.11 / 872.83 ±9.99 / 887.20 ms │ 860.04 / 872.06 ±9.65 / 883.47 ms │ no change │
│ QQuery 10 │ 469.96 / 475.06 ±5.08 / 483.50 ms │ 464.97 / 474.04 ±6.19 / 483.48 ms │ no change │
│ QQuery 11 │ 125.24 / 131.89 ±6.76 / 141.53 ms │ 124.07 / 132.48 ±9.45 / 150.67 ms │ no change │
│ QQuery 12 │ 316.13 / 319.73 ±3.16 / 324.44 ms │ 316.93 / 321.37 ±5.65 / 332.53 ms │ no change │
│ QQuery 13 │ 438.00 / 440.42 ±1.77 / 442.17 ms │ 429.15 / 441.04 ±8.93 / 455.62 ms │ no change │
│ QQuery 14 │ 325.12 / 329.99 ±6.78 / 343.39 ms │ 324.52 / 330.82 ±7.56 / 344.65 ms │ no change │
│ QQuery 15 │ 581.08 / 585.55 ±2.86 / 589.85 ms │ 574.48 / 580.77 ±5.05 / 589.14 ms │ no change │
│ QQuery 16 │ 86.12 / 91.62 ±6.20 / 103.29 ms │ 84.65 / 88.38 ±4.51 / 96.92 ms │ no change │
│ QQuery 17 │ 1023.61 / 1039.23 ±24.40 / 1087.74 ms │ 1001.94 / 1012.11 ±10.74 / 1028.63 ms │ no change │
│ QQuery 18 │ FAIL │ FAIL │ incomparable │
│ QQuery 19 │ 466.16 / 477.44 ±10.78 / 494.74 ms │ 465.82 / 474.82 ±9.70 / 493.44 ms │ no change │
│ QQuery 20 │ 698.87 / 700.56 ±1.65 / 702.66 ms │ 697.66 / 703.92 ±5.98 / 714.18 ms │ no change │
│ QQuery 21 │ 1073.03 / 1083.61 ±10.01 / 1100.46 ms │ 1056.32 / 1062.46 ±4.59 / 1070.04 ms │ no change │
│ QQuery 22 │ 97.50 / 99.44 ±1.71 / 101.88 ms │ 98.17 / 101.33 ±3.45 / 106.15 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 10057.58ms │
│ Total Time (fast-batch-wise-skip) │ 9960.29ms │
│ Average Time (HEAD) │ 478.93ms │
│ Average Time (fast-batch-wise-skip) │ 474.30ms │
│ Queries Faster │ 0 │
│ Queries Slower │ 0 │
│ Queries with No Change │ 21 │
│ Queries with Failure │ 1 │
└─────────────────────────────────────┴────────────┘

Resource Usage

tpch10 — base (merge-base)

MetricValue
Wall time55.0s
Peak memory2.3 GiB
Avg memory988.4 MiB
CPU user316.6s
CPU sys20.9s
Peak spill0 B

tpch10 — branch

MetricValue
Wall time55.0s
Peak memory3.1 GiB
Avg memory972.7 MiB
CPU user313.3s
CPU sys20.6s
Peak spill0 B

File an issue against this benchmark runner

@rluvaton

Copy link
Copy Markdown
MemberAuthor

run benchmark tpch10

baseline:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 3G
DATAFUSION_EXECUTION_TARGET_PARTITIONS: 6
changed:
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 3G
DATAFUSION_EXECUTION_TARGET_PARTITIONS: 6

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4905002217-897-p5b4w 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing fast-batch-wise-skip (92505cf) to 0365d3c (merge-base) diff using: tpch10
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

Benchmark for this request hit the 7200s job deadline before finishing.

Benchmarks requested: tpch10

Kubernetes message
Job was active longer than specified deadline

File an issue against this benchmark runner

@alamb

alamb commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Hi @rluvaton -- I am sort of lost on this PR -- it seems like it adds some non trivial complexity but I can't quite seem what benchmarks are made faster.

is it really tpch_sort (aka #23345 (comment)) ?

Is there any way to make this PR (and the logic) simpler?

@rluvaton

Copy link
Copy Markdown
MemberAuthor

Hi @rluvaton -- I am sort of lost on this PR -- it seems like it adds some non trivial complexity but I can't quite seem what benchmarks are made faster.

is it really tpch_sort (aka (earlier comment)) ?

Yes, I also created dedicated micro benchmarks:

Is there any way to make this PR (and the logic) simpler?

tried to simplify it

@rluvaton

rluvaton commented Jul 9, 2026

Copy link
Copy Markdown
MemberAuthor

@alamb In order to make the code simpler I'm refactoring the code before:

@rluvaton
rluvaton marked this pull request as draft July 9, 2026 20:38
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coreCore DataFusion crateffiChanges to the ffi crateperformanceMake DataFusion fasterphysical-planChanges to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@rluvaton@adriangbot@alamb