Skip to content

Fix final hash aggregate output regression by materializing once - #23182

Merged
alamb merged 5 commits into
apache:mainfrom
hhhizzz:codex/hashagg-terminal-output-materialize-v1-20260625
Jun 26, 2026
Merged

Fix final hash aggregate output regression by materializing once#23182
alamb merged 5 commits into
apache:mainfrom
hhhizzz:codex/hashagg-terminal-output-materialize-v1-20260625

Conversation

@hhhizzz

@hhhizzzhhhizzz commented Jun 25, 2026•
edited by alamb
Loading

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

PR #23055 changed final hash aggregate output to emit groups incrementally with
EmitTo::First(batch_size). For terminal final aggregate output, this can cause
the group value state to be repeatedly compacted while output batches are being
produced. On TPC-DS q23 this showed up as a significant regression.

This PR implements the short-term approach discussed in #23178: materialize the
final aggregate output once, then return slices of that materialized
RecordBatch according to batch_size.

This avoids changing the GroupValues API while preserving bounded downstream
batch sizes.

What changes are included in this PR?

  • Adds an OutputtingMaterialized hash aggregate state.
  • Adds MaterializedOutput, a small wrapper around a RecordBatch plus output
    offset.
  • Changes final hash aggregate output to:
    • emit all final groups once,
    • evaluate all final aggregate values once,
    • slice the materialized batch for subsequent output polling.
  • Leaves partial aggregate output behavior unchanged.
  • Adds focused tests for materialized output slicing and final hash aggregate
    output state transitions.

Performance

TPC-DS SF10 full 99 queries, 10 rounds:

  • Total runtime ratio: 0.857051
  • Geomean ratio: 0.976652 (~2.4% faster)
  • q23 ratio: 0.313770 (~218.7% faster), faster in 10/10 rounds

Regressions over 5% were observed in 10 queries. Most have small absolute
deltas, but the largest slowdowns were:

  • q67: 1.055907, +170.996 ms
  • q39: 1.060436, +98.544 ms
  • q9: 1.050135, +37.858 ms
  • q70: 1.061124, +11.848 ms
  • q35: 1.052392, +9.386 ms
  • q33: 1.063655, +6.995 ms
  • q98: 1.071688, +6.515 ms
  • q91: 1.109819, +5.362 ms
  • q15: 1.058356, +5.072 ms
  • q27: 1.057686, +0.815 ms

Overall, this recovers the q23 regression strongly and improves full-query
geomean, but q39 and q67 are worth calling out as residual per-query slowdowns.

Testing

  • cargo fmt --all -- --check
  • cargo test -p datafusion-physical-plan materializ
  • cargo test -p datafusion-physical-plan aggregates::
  • TPC-DS SF10 q23, 3 rounds
  • TPC-DS SF10 full 99 queries, 10 rounds

@github-actionsgithub-actionsBot added the physical-plan Changes to the physical-plan crate label Jun 25, 2026
@hhhizzz
hhhizzz marked this pull request as ready for review June 25, 2026 11:05
@hhhizzz

hhhizzz commented Jun 26, 2026

Copy link
Copy Markdown
ContributorAuthor

@2010YOUY01 Do you have time to take a look? ☺️

2010YOUY01
2010YOUY01 approved these changes Jun 26, 2026
View reviewed changes

@2010YOUY012010YOUY01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you! Just some minor suggestions.

Comment threaddatafusion/physical-plan/src/aggregates/aggregate_hash_table/common.rs
Comment threaddatafusion/physical-plan/src/aggregates/aggregate_hash_table/final_table.rs Outdated
@hhhizzz
hhhizzzforce-pushed the codex/hashagg-terminal-output-materialize-v1-20260625 branch from bc2a13c to 170d4baCompareJune 26, 2026 04:11
@Dandandan

Dandandan commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

run benchmarks

@2010YOUY012010YOUY01 mentioned this pull request Jun 26, 2026
Closed
@adriangbot

adriangbot commented Jun 26, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4806294902-689-ttvl7 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing codex/hashagg-terminal-output-materialize-v1-20260625 (170d4ba) to a00f749 (merge-base) diff using: clickbench_partitioned
Results will be posted here when complete

File an issue against this benchmark runner

@adriangbot

adriangbot commented Jun 26, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4806294902-690-gmg48 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing codex/hashagg-terminal-output-materialize-v1-20260625 (170d4ba) to a00f749 (merge-base) diff using: tpcds
Results will be posted here when complete

File an issue against this benchmark runner

@adriangbot

adriangbot commented Jun 26, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance:c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4806294902-691-j8n45 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Comparing codex/hashagg-terminal-output-materialize-v1-20260625 (170d4ba) to a00f749 (merge-base) diff using: tpch
Results will be posted here when complete

File an issue against this benchmark runner

@adriangbot

adriangbot commented Jun 26, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and codex_hashagg-terminal-output-materialize-v1-20260625
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ codex_hashagg-terminal-output-materialize-v1-20260625 ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 39.09 / 39.69 ±0.87 / 41.39 ms │ 38.64 / 40.02 ±1.97 / 43.92 ms │ no change │
│ QQuery 2 │ 19.74 / 19.95 ±0.17 / 20.25 ms │ 19.62 / 20.22 ±0.75 / 21.68 ms │ no change │
│ QQuery 3 │ 31.77 / 33.70 ±1.03 / 34.77 ms │ 31.24 / 32.29 ±0.97 / 34.04 ms │ no change │
│ QQuery 4 │ 18.16 / 18.56 ±0.53 / 19.60 ms │ 17.91 / 18.05 ±0.14 / 18.32 ms │ no change │
│ QQuery 5 │ 40.83 / 41.40 ±0.38 / 42.03 ms │ 41.43 / 41.68 ±0.24 / 42.03 ms │ no change │
│ QQuery 6 │ 16.74 / 16.90 ±0.10 / 16.98 ms │ 16.72 / 17.35 ±1.01 / 19.36 ms │ no change │
│ QQuery 7 │ 44.27 / 46.96 ±2.33 / 51.05 ms │ 44.95 / 47.56 ±2.74 / 52.78 ms │ no change │
│ QQuery 8 │ 43.96 / 44.58 ±0.66 / 45.79 ms │ 44.07 / 44.47 ±0.31 / 44.96 ms │ no change │
│ QQuery 9 │ 51.72 / 52.67 ±0.94 / 54.46 ms │ 50.32 / 51.13 ±0.67 / 52.33 ms │ no change │
│ QQuery 10 │ 42.90 / 43.72 ±0.97 / 45.61 ms │ 42.98 / 43.07 ±0.07 / 43.18 ms │ no change │
│ QQuery 11 │ 13.81 / 13.99 ±0.11 / 14.12 ms │ 13.73 / 14.00 ±0.19 / 14.31 ms │ no change │
│ QQuery 12 │ 24.38 / 24.71 ±0.20 / 25.01 ms │ 24.05 / 24.49 ±0.27 / 24.79 ms │ no change │
│ QQuery 13 │ 33.32 / 34.90 ±1.17 / 36.91 ms │ 32.70 / 35.36 ±2.10 / 38.61 ms │ no change │
│ QQuery 14 │ 24.22 / 24.57 ±0.40 / 25.35 ms │ 24.38 / 24.62 ±0.29 / 25.17 ms │ no change │
│ QQuery 15 │ 31.83 / 32.13 ±0.16 / 32.29 ms │ 31.64 / 32.26 ±0.81 / 33.85 ms │ no change │
│ QQuery 16 │ 14.37 / 14.64 ±0.20 / 14.92 ms │ 14.11 / 14.69 ±0.36 / 15.18 ms │ no change │
│ QQuery 17 │ 91.24 / 92.45 ±1.14 / 94.33 ms │ 90.05 / 90.60 ±0.91 / 92.40 ms │ no change │
│ QQuery 18 │ 73.97 / 75.14 ±1.00 / 76.73 ms │ 66.61 / 68.75 ±1.53 / 71.17 ms │ +1.09x faster │
│ QQuery 19 │ 34.04 / 34.20 ±0.18 / 34.47 ms │ 33.39 / 33.96 ±0.47 / 34.73 ms │ no change │
│ QQuery 20 │ 35.83 / 36.37 ±0.53 / 37.21 ms │ 34.90 / 35.82 ±0.92 / 37.45 ms │ no change │
│ QQuery 21 │ 56.91 / 59.38 ±2.04 / 62.07 ms │ 56.58 / 57.96 ±1.02 / 59.13 ms │ no change │
│ QQuery 22 │ 14.58 / 15.07 ±0.36 / 15.67 ms │ 14.18 / 15.96 ±2.46 / 20.78 ms │ 1.06x slower │
└───────────┴────────────────────────────────┴───────────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD) │ 815.69ms │
│ Total Time (codex_hashagg-terminal-output-materialize-v1-20260625) │ 804.31ms │
│ Average Time (HEAD) │ 37.08ms │
│ Average Time (codex_hashagg-terminal-output-materialize-v1-20260625) │ 36.56ms │
│ Queries Faster │ 1 │
│ Queries Slower │ 1 │
│ Queries with No Change │ 20 │
│ Queries with Failure │ 0 │
└──────────────────────────────────────────────────────────────────────┴──────────┘

Resource Usage

tpch — base (merge-base)

MetricValue
Wall time5.0s
Peak memory980.5 MiB
Avg memory638.1 MiB
CPU user24.3s
CPU sys1.6s
Peak spill0 B

tpch — branch

MetricValue
Wall time5.0s
Peak memory1.1 GiB
Avg memory664.3 MiB
CPU user23.7s
CPU sys1.8s
Peak spill0 B

File an issue against this benchmark runner

@adriangbot

adriangbot commented Jun 26, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and codex_hashagg-terminal-output-materialize-v1-20260625
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ codex_hashagg-terminal-output-materialize-v1-20260625 ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 5.69 / 6.23 ±0.85 / 7.91 ms │ 5.70 / 6.20 ±0.85 / 7.90 ms │ no change │
│ QQuery 2 │ 81.91 / 82.33 ±0.29 / 82.77 ms │ 81.66 / 81.87 ±0.26 / 82.37 ms │ no change │
│ QQuery 3 │ 30.05 / 30.19 ±0.09 / 30.30 ms │ 30.03 / 30.25 ±0.21 / 30.62 ms │ no change │
│ QQuery 4 │ 499.54 / 510.92 ±7.07 / 521.62 ms │ 492.04 / 495.17 ±3.59 / 500.98 ms │ no change │
│ QQuery 5 │ 52.31 / 53.05 ±0.59 / 54.09 ms │ 52.27 / 53.35 ±0.83 / 54.50 ms │ no change │
│ QQuery 6 │ 37.12 / 37.32 ±0.18 / 37.65 ms │ 37.02 / 37.42 ±0.25 / 37.69 ms │ no change │
│ QQuery 7 │ 94.22 / 96.32 ±2.25 / 100.67 ms │ 94.40 / 95.27 ±0.61 / 96.17 ms │ no change │
│ QQuery 8 │ 36.88 / 37.61 ±0.50 / 38.45 ms │ 37.39 / 39.72 ±4.20 / 48.10 ms │ 1.06x slower │
│ QQuery 9 │ 51.63 / 54.31 ±1.63 / 56.05 ms │ 52.34 / 55.32 ±1.92 / 58.30 ms │ no change │
│ QQuery 10 │ 63.43 / 63.95 ±0.56 / 64.90 ms │ 63.35 / 63.69 ±0.25 / 64.07 ms │ no change │
│ QQuery 11 │ 302.16 / 307.47 ±4.12 / 312.17 ms │ 298.72 / 304.23 ±5.41 / 312.92 ms │ no change │
│ QQuery 12 │ 28.71 / 29.10 ±0.30 / 29.62 ms │ 28.90 / 29.46 ±0.36 / 29.90 ms │ no change │
│ QQuery 13 │ 119.05 / 120.86 ±2.81 / 126.45 ms │ 118.98 / 120.50 ±1.05 / 122.04 ms │ no change │
│ QQuery 14 │ 415.37 / 422.32 ±5.86 / 431.82 ms │ 415.53 / 418.88 ±3.26 / 424.29 ms │ no change │
│ QQuery 15 │ 58.78 / 59.75 ±0.82 / 61.05 ms │ 58.45 / 59.23 ±0.47 / 59.89 ms │ no change │
│ QQuery 16 │ 6.92 / 7.09 ±0.26 / 7.61 ms │ 6.80 / 6.92 ±0.15 / 7.21 ms │ no change │
│ QQuery 17 │ 81.09 / 82.46 ±1.44 / 85.12 ms │ 80.56 / 81.31 ±0.71 / 82.64 ms │ no change │
│ QQuery 18 │ 125.53 / 126.31 ±0.57 / 127.14 ms │ 124.23 / 126.65 ±2.91 / 132.30 ms │ no change │
│ QQuery 19 │ 41.49 / 41.85 ±0.32 / 42.39 ms │ 41.79 / 42.15 ±0.35 / 42.72 ms │ no change │
│ QQuery 20 │ 35.79 / 36.90 ±1.17 / 39.05 ms │ 36.07 / 36.90 ±0.55 / 37.69 ms │ no change │
│ QQuery 21 │ 17.97 / 18.56 ±0.65 / 19.76 ms │ 17.80 / 17.95 ±0.16 / 18.22 ms │ no change │
│ QQuery 22 │ 62.57 / 63.70 ±0.93 / 65.11 ms │ 62.75 / 62.97 ±0.19 / 63.32 ms │ no change │
│ QQuery 23 │ 409.39 / 418.38 ±5.37 / 424.08 ms │ 359.32 / 366.39 ±5.71 / 376.68 ms │ +1.14x faster │
│ QQuery 24 │ 226.72 / 230.58 ±2.92 / 234.27 ms │ 229.49 / 232.73 ±5.88 / 244.48 ms │ no change │
│ QQuery 25 │ 111.85 / 114.63 ±3.44 / 121.23 ms │ 111.77 / 112.67 ±0.91 / 114.25 ms │ no change │
│ QQuery 26 │ 58.10 / 58.50 ±0.32 / 58.99 ms │ 58.63 / 58.98 ±0.19 / 59.16 ms │ no change │
│ QQuery 27 │ 6.54 / 6.65 ±0.16 / 6.97 ms │ 6.59 / 6.72 ±0.15 / 7.01 ms │ no change │
│ QQuery 28 │ 57.81 / 61.07 ±1.66 / 62.46 ms │ 61.84 / 62.44 ±0.36 / 62.88 ms │ no change │
│ QQuery 29 │ 98.83 / 101.60 ±2.68 / 105.78 ms │ 98.26 / 99.06 ±0.80 / 100.50 ms │ no change │
│ QQuery 30 │ 33.11 / 33.22 ±0.16 / 33.53 ms │ 33.11 / 33.60 ±0.72 / 34.99 ms │ no change │
│ QQuery 31 │ 111.85 / 114.38 ±2.13 / 118.23 ms │ 112.01 / 112.93 ±0.75 / 114.26 ms │ no change │
│ QQuery 32 │ 21.05 / 21.29 ±0.21 / 21.58 ms │ 20.45 / 20.88 ±0.38 / 21.58 ms │ no change │
│ QQuery 33 │ 38.55 / 38.97 ±0.36 / 39.53 ms │ 37.97 / 38.30 ±0.36 / 38.77 ms │ no change │
│ QQuery 34 │ 10.43 / 10.58 ±0.20 / 10.97 ms │ 9.92 / 10.11 ±0.15 / 10.33 ms │ no change │
│ QQuery 35 │ 72.63 / 73.41 ±0.45 / 73.78 ms │ 72.96 / 74.12 ±1.24 / 76.21 ms │ no change │
│ QQuery 36 │ 5.91 / 6.14 ±0.20 / 6.49 ms │ 5.89 / 6.07 ±0.16 / 6.34 ms │ no change │
│ QQuery 37 │ 7.07 / 7.22 ±0.10 / 7.34 ms │ 7.03 / 7.10 ±0.04 / 7.15 ms │ no change │
│ QQuery 38 │ 63.83 / 65.01 ±1.19 / 67.19 ms │ 63.61 / 64.18 ±0.36 / 64.58 ms │ no change │
│ QQuery 39 │ 86.86 / 88.20 ±1.60 / 91.34 ms │ 87.35 / 88.83 ±1.64 / 91.35 ms │ no change │
│ QQuery 40 │ 23.63 / 23.86 ±0.23 / 24.20 ms │ 23.49 / 24.09 ±0.45 / 24.60 ms │ no change │
│ QQuery 41 │ 11.70 / 11.82 ±0.14 / 12.08 ms │ 11.77 / 12.01 ±0.23 / 12.45 ms │ no change │
│ QQuery 42 │ 23.74 / 24.60 ±1.23 / 27.05 ms │ 24.59 / 25.07 ±0.51 / 26.04 ms │ no change │
│ QQuery 43 │ 4.97 / 5.08 ±0.10 / 5.27 ms │ 5.16 / 5.23 ±0.08 / 5.38 ms │ no change │
│ QQuery 44 │ 9.45 / 10.15 ±1.09 / 12.32 ms │ 9.60 / 9.84 ±0.15 / 10.01 ms │ no change │
│ QQuery 45 │ 39.74 / 40.81 ±0.86 / 42.30 ms │ 40.45 / 42.28 ±2.19 / 46.54 ms │ no change │
│ QQuery 46 │ 11.95 / 12.61 ±0.50 / 13.20 ms │ 12.64 / 13.15 ±0.35 / 13.58 ms │ no change │
│ QQuery 47 │ 238.98 / 242.97 ±4.12 / 250.95 ms │ 240.18 / 243.84 ±5.66 / 255.13 ms │ no change │
│ QQuery 48 │ 96.69 / 96.99 ±0.35 / 97.67 ms │ 97.09 / 97.76 ±0.56 / 98.64 ms │ no change │
│ QQuery 49 │ 77.03 / 77.77 ±0.56 / 78.39 ms │ 77.52 / 80.19 ±3.84 / 87.83 ms │ no change │
│ QQuery 50 │ 59.01 / 59.39 ±0.26 / 59.65 ms │ 59.36 / 60.45 ±0.62 / 60.99 ms │ no change │
│ QQuery 51 │ 98.94 / 101.92 ±2.63 / 106.83 ms │ 98.73 / 101.32 ±1.33 / 102.38 ms │ no change │
│ QQuery 52 │ 23.81 / 24.13 ±0.26 / 24.47 ms │ 24.33 / 25.33 ±0.64 / 26.12 ms │ no change │
│ QQuery 53 │ 29.61 / 29.95 ±0.19 / 30.20 ms │ 30.25 / 30.70 ±0.44 / 31.45 ms │ no change │
│ QQuery 54 │ 55.33 / 56.16 ±0.45 / 56.70 ms │ 56.44 / 56.89 ±0.37 / 57.45 ms │ no change │
│ QQuery 55 │ 23.53 / 23.60 ±0.07 / 23.72 ms │ 23.81 / 24.20 ±0.21 / 24.40 ms │ no change │
│ QQuery 56 │ 39.17 / 41.92 ±3.68 / 49.06 ms │ 39.19 / 39.66 ±0.43 / 40.33 ms │ +1.06x faster │
│ QQuery 57 │ 180.63 / 182.33 ±1.22 / 184.12 ms │ 180.05 / 181.70 ±1.02 / 182.90 ms │ no change │
│ QQuery 58 │ 116.15 / 118.18 ±2.14 / 122.02 ms │ 117.75 / 119.61 ±2.33 / 124.08 ms │ no change │
│ QQuery 59 │ 118.98 / 119.11 ±0.14 / 119.35 ms │ 119.08 / 119.79 ±0.50 / 120.35 ms │ no change │
│ QQuery 60 │ 40.25 / 40.73 ±0.51 / 41.44 ms │ 39.64 / 41.75 ±2.70 / 47.08 ms │ no change │
│ QQuery 61 │ 12.71 / 12.95 ±0.14 / 13.13 ms │ 12.81 / 12.98 ±0.18 / 13.33 ms │ no change │
│ QQuery 62 │ 46.73 / 47.26 ±0.32 / 47.74 ms │ 47.35 / 47.39 ±0.04 / 47.44 ms │ no change │
│ QQuery 63 │ 30.12 / 30.32 ±0.22 / 30.74 ms │ 30.26 / 30.45 ±0.16 / 30.72 ms │ no change │
│ QQuery 64 │ 409.43 / 413.94 ±3.23 / 417.61 ms │ 409.85 / 414.82 ±4.10 / 421.51 ms │ no change │
│ QQuery 65 │ 149.80 / 151.29 ±1.46 / 153.97 ms │ 150.11 / 156.70 ±5.48 / 164.47 ms │ no change │
│ QQuery 66 │ 79.60 / 81.60 ±3.19 / 87.91 ms │ 79.93 / 80.63 ±0.64 / 81.66 ms │ no change │
│ QQuery 67 │ 244.42 / 249.34 ±4.06 / 256.71 ms │ 246.85 / 250.09 ±2.97 / 255.12 ms │ no change │
│ QQuery 68 │ 11.98 / 12.14 ±0.20 / 12.50 ms │ 12.15 / 12.97 ±1.23 / 15.42 ms │ 1.07x slower │
│ QQuery 69 │ 57.73 / 58.21 ±0.29 / 58.57 ms │ 57.65 / 58.32 ±0.44 / 58.96 ms │ no change │
│ QQuery 70 │ 105.02 / 110.59 ±8.86 / 128.14 ms │ 105.44 / 106.06 ±0.81 / 107.65 ms │ no change │
│ QQuery 71 │ 35.71 / 36.14 ±0.46 / 37.01 ms │ 36.02 / 38.05 ±2.18 / 41.84 ms │ 1.05x slower │
│ QQuery 72 │ 2179.61 / 2235.82 ±39.44 / 2280.86 ms │ 2138.28 / 2272.04 ±112.52 / 2462.04 ms │ no change │
│ QQuery 73 │ 9.73 / 9.91 ±0.25 / 10.41 ms │ 9.86 / 9.99 ±0.13 / 10.22 ms │ no change │
│ QQuery 74 │ 170.94 / 177.04 ±5.10 / 186.22 ms │ 178.38 / 187.13 ±6.60 / 194.94 ms │ 1.06x slower │
│ QQuery 75 │ 150.79 / 156.07 ±7.37 / 170.63 ms │ 154.30 / 159.29 ±8.33 / 175.88 ms │ no change │
│ QQuery 76 │ 35.75 / 36.05 ±0.24 / 36.38 ms │ 35.85 / 36.52 ±0.58 / 37.52 ms │ no change │
│ QQuery 77 │ 61.04 / 61.69 ±0.46 / 62.43 ms │ 62.43 / 63.73 ±1.22 / 65.66 ms │ no change │
│ QQuery 78 │ 187.28 / 191.43 ±4.19 / 199.48 ms │ 195.00 / 199.56 ±5.03 / 208.79 ms │ no change │
│ QQuery 79 │ 67.93 / 70.62 ±4.91 / 80.44 ms │ 69.47 / 70.55 ±1.57 / 73.66 ms │ no change │
│ QQuery 80 │ 101.18 / 103.37 ±3.11 / 109.51 ms │ 101.77 / 103.96 ±2.03 / 107.25 ms │ no change │
│ QQuery 81 │ 26.90 / 27.20 ±0.29 / 27.76 ms │ 26.97 / 27.30 ±0.20 / 27.50 ms │ no change │
│ QQuery 82 │ 16.96 / 17.12 ±0.15 / 17.37 ms │ 17.21 / 17.39 ±0.18 / 17.68 ms │ no change │
│ QQuery 83 │ 41.40 / 41.77 ±0.28 / 42.16 ms │ 42.37 / 45.65 ±5.12 / 55.84 ms │ 1.09x slower │
│ QQuery 84 │ 31.86 / 33.92 ±3.06 / 39.98 ms │ 31.99 / 32.93 ±0.95 / 34.71 ms │ no change │
│ QQuery 85 │ 109.25 / 110.26 ±0.87 / 111.81 ms │ 110.96 / 111.71 ±0.58 / 112.56 ms │ no change │
│ QQuery 86 │ 25.48 / 25.69 ±0.29 / 26.26 ms │ 25.73 / 26.44 ±0.46 / 27.15 ms │ no change │
│ QQuery 87 │ 65.22 / 66.07 ±0.57 / 66.64 ms │ 67.75 / 69.33 ±1.24 / 71.46 ms │ no change │
│ QQuery 88 │ 64.14 / 64.75 ±0.35 / 65.18 ms │ 65.33 / 65.65 ±0.31 / 66.17 ms │ no change │
│ QQuery 89 │ 36.10 / 36.90 ±0.63 / 37.86 ms │ 36.90 / 37.22 ±0.42 / 38.05 ms │ no change │
│ QQuery 90 │ 17.32 / 17.49 ±0.18 / 17.83 ms │ 18.12 / 18.39 ±0.14 / 18.51 ms │ 1.05x slower │
│ QQuery 91 │ 46.58 / 48.36 ±2.99 / 54.32 ms │ 47.70 / 50.87 ±4.47 / 59.56 ms │ 1.05x slower │
│ QQuery 92 │ 30.63 / 31.62 ±1.46 / 34.49 ms │ 32.03 / 32.23 ±0.16 / 32.50 ms │ no change │
│ QQuery 93 │ 50.10 / 51.45 ±1.34 / 53.94 ms │ 52.37 / 52.96 ±0.81 / 54.53 ms │ no change │
│ QQuery 94 │ 38.74 / 39.05 ±0.27 / 39.51 ms │ 40.83 / 41.01 ±0.17 / 41.21 ms │ 1.05x slower │
│ QQuery 95 │ 81.75 / 83.64 ±2.83 / 89.26 ms │ 86.04 / 88.07 ±1.93 / 91.54 ms │ 1.05x slower │
│ QQuery 96 │ 24.68 / 25.24 ±0.90 / 27.00 ms │ 25.53 / 25.74 ±0.13 / 25.87 ms │ no change │
│ QQuery 97 │ 56.93 / 57.70 ±0.59 / 58.67 ms │ 56.99 / 57.80 ±0.50 / 58.35 ms │ no change │
│ QQuery 98 │ 43.05 / 43.83 ±0.57 / 44.60 ms │ 44.90 / 45.65 ±0.77 / 47.10 ms │ no change │
│ QQuery 99 │ 70.62 / 72.89 ±3.44 / 79.75 ms │ 72.47 / 73.09 ±0.71 / 74.39 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 10223.26ms │
│ Total Time (codex_hashagg-terminal-output-materialize-v1-20260625) │ 10246.02ms │
│ Average Time (HEAD) │ 103.27ms │
│ Average Time (codex_hashagg-terminal-output-materialize-v1-20260625) │ 103.50ms │
│ Queries Faster │ 2 │
│ Queries Slower │ 9 │
│ Queries with No Change │ 88 │
│ Queries with Failure │ 0 │
└──────────────────────────────────────────────────────────────────────┴────────────┘

Resource Usage

tpcds — base (merge-base)

MetricValue
Wall time55.0s
Peak memory2.0 GiB
Avg memory1.3 GiB
CPU user232.8s
CPU sys5.8s
Peak spill0 B

tpcds — branch

MetricValue
Wall time55.0s
Peak memory2.1 GiB
Avg memory1.4 GiB
CPU user234.0s
CPU sys5.8s
Peak spill0 B

File an issue against this benchmark runner

@hhhizzz

hhhizzz commented Jun 26, 2026

Copy link
Copy Markdown
ContributorAuthor

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected

Details

Comparing HEAD and codex_hashagg-terminal-output-materialize-v1-20260625
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ codex_hashagg-terminal-output-materialize-v1-20260625 ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 5.69 / 6.23 ±0.85 / 7.91 ms │ 5.70 / 6.20 ±0.85 / 7.90 ms │ no change │
│ QQuery 2 │ 81.91 / 82.33 ±0.29 / 82.77 ms │ 81.66 / 81.87 ±0.26 / 82.37 ms │ no change │
│ QQuery 3 │ 30.05 / 30.19 ±0.09 / 30.30 ms │ 30.03 / 30.25 ±0.21 / 30.62 ms │ no change │
│ QQuery 4 │ 499.54 / 510.92 ±7.07 / 521.62 ms │ 492.04 / 495.17 ±3.59 / 500.98 ms │ no change │
│ QQuery 5 │ 52.31 / 53.05 ±0.59 / 54.09 ms │ 52.27 / 53.35 ±0.83 / 54.50 ms │ no change │
│ QQuery 6 │ 37.12 / 37.32 ±0.18 / 37.65 ms │ 37.02 / 37.42 ±0.25 / 37.69 ms │ no change │
│ QQuery 7 │ 94.22 / 96.32 ±2.25 / 100.67 ms │ 94.40 / 95.27 ±0.61 / 96.17 ms │ no change │
│ QQuery 8 │ 36.88 / 37.61 ±0.50 / 38.45 ms │ 37.39 / 39.72 ±4.20 / 48.10 ms │ 1.06x slower │
│ QQuery 9 │ 51.63 / 54.31 ±1.63 / 56.05 ms │ 52.34 / 55.32 ±1.92 / 58.30 ms │ no change │
│ QQuery 10 │ 63.43 / 63.95 ±0.56 / 64.90 ms │ 63.35 / 63.69 ±0.25 / 64.07 ms │ no change │
│ QQuery 11 │ 302.16 / 307.47 ±4.12 / 312.17 ms │ 298.72 / 304.23 ±5.41 / 312.92 ms │ no change │
│ QQuery 12 │ 28.71 / 29.10 ±0.30 / 29.62 ms │ 28.90 / 29.46 ±0.36 / 29.90 ms │ no change │
│ QQuery 13 │ 119.05 / 120.86 ±2.81 / 126.45 ms │ 118.98 / 120.50 ±1.05 / 122.04 ms │ no change │
│ QQuery 14 │ 415.37 / 422.32 ±5.86 / 431.82 ms │ 415.53 / 418.88 ±3.26 / 424.29 ms │ no change │
│ QQuery 15 │ 58.78 / 59.75 ±0.82 / 61.05 ms │ 58.45 / 59.23 ±0.47 / 59.89 ms │ no change │
│ QQuery 16 │ 6.92 / 7.09 ±0.26 / 7.61 ms │ 6.80 / 6.92 ±0.15 / 7.21 ms │ no change │
│ QQuery 17 │ 81.09 / 82.46 ±1.44 / 85.12 ms │ 80.56 / 81.31 ±0.71 / 82.64 ms │ no change │
│ QQuery 18 │ 125.53 / 126.31 ±0.57 / 127.14 ms │ 124.23 / 126.65 ±2.91 / 132.30 ms │ no change │
│ QQuery 19 │ 41.49 / 41.85 ±0.32 / 42.39 ms │ 41.79 / 42.15 ±0.35 / 42.72 ms │ no change │
│ QQuery 20 │ 35.79 / 36.90 ±1.17 / 39.05 ms │ 36.07 / 36.90 ±0.55 / 37.69 ms │ no change │
│ QQuery 21 │ 17.97 / 18.56 ±0.65 / 19.76 ms │ 17.80 / 17.95 ±0.16 / 18.22 ms │ no change │
│ QQuery 22 │ 62.57 / 63.70 ±0.93 / 65.11 ms │ 62.75 / 62.97 ±0.19 / 63.32 ms │ no change │
│ QQuery 23 │ 409.39 / 418.38 ±5.37 / 424.08 ms │ 359.32 / 366.39 ±5.71 / 376.68 ms │ +1.14x faster │
│ QQuery 24 │ 226.72 / 230.58 ±2.92 / 234.27 ms │ 229.49 / 232.73 ±5.88 / 244.48 ms │ no change │
│ QQuery 25 │ 111.85 / 114.63 ±3.44 / 121.23 ms │ 111.77 / 112.67 ±0.91 / 114.25 ms │ no change │
│ QQuery 26 │ 58.10 / 58.50 ±0.32 / 58.99 ms │ 58.63 / 58.98 ±0.19 / 59.16 ms │ no change │
│ QQuery 27 │ 6.54 / 6.65 ±0.16 / 6.97 ms │ 6.59 / 6.72 ±0.15 / 7.01 ms │ no change │
│ QQuery 28 │ 57.81 / 61.07 ±1.66 / 62.46 ms │ 61.84 / 62.44 ±0.36 / 62.88 ms │ no change │
│ QQuery 29 │ 98.83 / 101.60 ±2.68 / 105.78 ms │ 98.26 / 99.06 ±0.80 / 100.50 ms │ no change │
│ QQuery 30 │ 33.11 / 33.22 ±0.16 / 33.53 ms │ 33.11 / 33.60 ±0.72 / 34.99 ms │ no change │
│ QQuery 31 │ 111.85 / 114.38 ±2.13 / 118.23 ms │ 112.01 / 112.93 ±0.75 / 114.26 ms │ no change │
│ QQuery 32 │ 21.05 / 21.29 ±0.21 / 21.58 ms │ 20.45 / 20.88 ±0.38 / 21.58 ms │ no change │
│ QQuery 33 │ 38.55 / 38.97 ±0.36 / 39.53 ms │ 37.97 / 38.30 ±0.36 / 38.77 ms │ no change │
│ QQuery 34 │ 10.43 / 10.58 ±0.20 / 10.97 ms │ 9.92 / 10.11 ±0.15 / 10.33 ms │ no change │
│ QQuery 35 │ 72.63 / 73.41 ±0.45 / 73.78 ms │ 72.96 / 74.12 ±1.24 / 76.21 ms │ no change │
│ QQuery 36 │ 5.91 / 6.14 ±0.20 / 6.49 ms │ 5.89 / 6.07 ±0.16 / 6.34 ms │ no change │
│ QQuery 37 │ 7.07 / 7.22 ±0.10 / 7.34 ms │ 7.03 / 7.10 ±0.04 / 7.15 ms │ no change │
│ QQuery 38 │ 63.83 / 65.01 ±1.19 / 67.19 ms │ 63.61 / 64.18 ±0.36 / 64.58 ms │ no change │
│ QQuery 39 │ 86.86 / 88.20 ±1.60 / 91.34 ms │ 87.35 / 88.83 ±1.64 / 91.35 ms │ no change │
│ QQuery 40 │ 23.63 / 23.86 ±0.23 / 24.20 ms │ 23.49 / 24.09 ±0.45 / 24.60 ms │ no change │
│ QQuery 41 │ 11.70 / 11.82 ±0.14 / 12.08 ms │ 11.77 / 12.01 ±0.23 / 12.45 ms │ no change │
│ QQuery 42 │ 23.74 / 24.60 ±1.23 / 27.05 ms │ 24.59 / 25.07 ±0.51 / 26.04 ms │ no change │
│ QQuery 43 │ 4.97 / 5.08 ±0.10 / 5.27 ms │ 5.16 / 5.23 ±0.08 / 5.38 ms │ no change │
│ QQuery 44 │ 9.45 / 10.15 ±1.09 / 12.32 ms │ 9.60 / 9.84 ±0.15 / 10.01 ms │ no change │
│ QQuery 45 │ 39.74 / 40.81 ±0.86 / 42.30 ms │ 40.45 / 42.28 ±2.19 / 46.54 ms │ no change │
│ QQuery 46 │ 11.95 / 12.61 ±0.50 / 13.20 ms │ 12.64 / 13.15 ±0.35 / 13.58 ms │ no change │
│ QQuery 47 │ 238.98 / 242.97 ±4.12 / 250.95 ms │ 240.18 / 243.84 ±5.66 / 255.13 ms │ no change │
│ QQuery 48 │ 96.69 / 96.99 ±0.35 / 97.67 ms │ 97.09 / 97.76 ±0.56 / 98.64 ms │ no change │
│ QQuery 49 │ 77.03 / 77.77 ±0.56 / 78.39 ms │ 77.52 / 80.19 ±3.84 / 87.83 ms │ no change │
│ QQuery 50 │ 59.01 / 59.39 ±0.26 / 59.65 ms │ 59.36 / 60.45 ±0.62 / 60.99 ms │ no change │
│ QQuery 51 │ 98.94 / 101.92 ±2.63 / 106.83 ms │ 98.73 / 101.32 ±1.33 / 102.38 ms │ no change │
│ QQuery 52 │ 23.81 / 24.13 ±0.26 / 24.47 ms │ 24.33 / 25.33 ±0.64 / 26.12 ms │ no change │
│ QQuery 53 │ 29.61 / 29.95 ±0.19 / 30.20 ms │ 30.25 / 30.70 ±0.44 / 31.45 ms │ no change │
│ QQuery 54 │ 55.33 / 56.16 ±0.45 / 56.70 ms │ 56.44 / 56.89 ±0.37 / 57.45 ms │ no change │
│ QQuery 55 │ 23.53 / 23.60 ±0.07 / 23.72 ms │ 23.81 / 24.20 ±0.21 / 24.40 ms │ no change │
│ QQuery 56 │ 39.17 / 41.92 ±3.68 / 49.06 ms │ 39.19 / 39.66 ±0.43 / 40.33 ms │ +1.06x faster │
│ QQuery 57 │ 180.63 / 182.33 ±1.22 / 184.12 ms │ 180.05 / 181.70 ±1.02 / 182.90 ms │ no change │
│ QQuery 58 │ 116.15 / 118.18 ±2.14 / 122.02 ms │ 117.75 / 119.61 ±2.33 / 124.08 ms │ no change │
│ QQuery 59 │ 118.98 / 119.11 ±0.14 / 119.35 ms │ 119.08 / 119.79 ±0.50 / 120.35 ms │ no change │
│ QQuery 60 │ 40.25 / 40.73 ±0.51 / 41.44 ms │ 39.64 / 41.75 ±2.70 / 47.08 ms │ no change │
│ QQuery 61 │ 12.71 / 12.95 ±0.14 / 13.13 ms │ 12.81 / 12.98 ±0.18 / 13.33 ms │ no change │
│ QQuery 62 │ 46.73 / 47.26 ±0.32 / 47.74 ms │ 47.35 / 47.39 ±0.04 / 47.44 ms │ no change │
│ QQuery 63 │ 30.12 / 30.32 ±0.22 / 30.74 ms │ 30.26 / 30.45 ±0.16 / 30.72 ms │ no change │
│ QQuery 64 │ 409.43 / 413.94 ±3.23 / 417.61 ms │ 409.85 / 414.82 ±4.10 / 421.51 ms │ no change │
│ QQuery 65 │ 149.80 / 151.29 ±1.46 / 153.97 ms │ 150.11 / 156.70 ±5.48 / 164.47 ms │ no change │
│ QQuery 66 │ 79.60 / 81.60 ±3.19 / 87.91 ms │ 79.93 / 80.63 ±0.64 / 81.66 ms │ no change │
│ QQuery 67 │ 244.42 / 249.34 ±4.06 / 256.71 ms │ 246.85 / 250.09 ±2.97 / 255.12 ms │ no change │
│ QQuery 68 │ 11.98 / 12.14 ±0.20 / 12.50 ms │ 12.15 / 12.97 ±1.23 / 15.42 ms │ 1.07x slower │
│ QQuery 69 │ 57.73 / 58.21 ±0.29 / 58.57 ms │ 57.65 / 58.32 ±0.44 / 58.96 ms │ no change │
│ QQuery 70 │ 105.02 / 110.59 ±8.86 / 128.14 ms │ 105.44 / 106.06 ±0.81 / 107.65 ms │ no change │
│ QQuery 71 │ 35.71 / 36.14 ±0.46 / 37.01 ms │ 36.02 / 38.05 ±2.18 / 41.84 ms │ 1.05x slower │
│ QQuery 72 │ 2179.61 / 2235.82 ±39.44 / 2280.86 ms │ 2138.28 / 2272.04 ±112.52 / 2462.04 ms │ no change │
│ QQuery 73 │ 9.73 / 9.91 ±0.25 / 10.41 ms │ 9.86 / 9.99 ±0.13 / 10.22 ms │ no change │
│ QQuery 74 │ 170.94 / 177.04 ±5.10 / 186.22 ms │ 178.38 / 187.13 ±6.60 / 194.94 ms │ 1.06x slower │
│ QQuery 75 │ 150.79 / 156.07 ±7.37 / 170.63 ms │ 154.30 / 159.29 ±8.33 / 175.88 ms │ no change │
│ QQuery 76 │ 35.75 / 36.05 ±0.24 / 36.38 ms │ 35.85 / 36.52 ±0.58 / 37.52 ms │ no change │
│ QQuery 77 │ 61.04 / 61.69 ±0.46 / 62.43 ms │ 62.43 / 63.73 ±1.22 / 65.66 ms │ no change │
│ QQuery 78 │ 187.28 / 191.43 ±4.19 / 199.48 ms │ 195.00 / 199.56 ±5.03 / 208.79 ms │ no change │
│ QQuery 79 │ 67.93 / 70.62 ±4.91 / 80.44 ms │ 69.47 / 70.55 ±1.57 / 73.66 ms │ no change │
│ QQuery 80 │ 101.18 / 103.37 ±3.11 / 109.51 ms │ 101.77 / 103.96 ±2.03 / 107.25 ms │ no change │
│ QQuery 81 │ 26.90 / 27.20 ±0.29 / 27.76 ms │ 26.97 / 27.30 ±0.20 / 27.50 ms │ no change │
│ QQuery 82 │ 16.96 / 17.12 ±0.15 / 17.37 ms │ 17.21 / 17.39 ±0.18 / 17.68 ms │ no change │
│ QQuery 83 │ 41.40 / 41.77 ±0.28 / 42.16 ms │ 42.37 / 45.65 ±5.12 / 55.84 ms │ 1.09x slower │
│ QQuery 84 │ 31.86 / 33.92 ±3.06 / 39.98 ms │ 31.99 / 32.93 ±0.95 / 34.71 ms │ no change │
│ QQuery 85 │ 109.25 / 110.26 ±0.87 / 111.81 ms │ 110.96 / 111.71 ±0.58 / 112.56 ms │ no change │
│ QQuery 86 │ 25.48 / 25.69 ±0.29 / 26.26 ms │ 25.73 / 26.44 ±0.46 / 27.15 ms │ no change │
│ QQuery 87 │ 65.22 / 66.07 ±0.57 / 66.64 ms │ 67.75 / 69.33 ±1.24 / 71.46 ms │ no change │
│ QQuery 88 │ 64.14 / 64.75 ±0.35 / 65.18 ms │ 65.33 / 65.65 ±0.31 / 66.17 ms │ no change │
│ QQuery 89 │ 36.10 / 36.90 ±0.63 / 37.86 ms │ 36.90 / 37.22 ±0.42 / 38.05 ms │ no change │
│ QQuery 90 │ 17.32 / 17.49 ±0.18 / 17.83 ms │ 18.12 / 18.39 ±0.14 / 18.51 ms │ 1.05x slower │
│ QQuery 91 │ 46.58 / 48.36 ±2.99 / 54.32 ms │ 47.70 / 50.87 ±4.47 / 59.56 ms │ 1.05x slower │
│ QQuery 92 │ 30.63 / 31.62 ±1.46 / 34.49 ms │ 32.03 / 32.23 ±0.16 / 32.50 ms │ no change │
│ QQuery 93 │ 50.10 / 51.45 ±1.34 / 53.94 ms │ 52.37 / 52.96 ±0.81 / 54.53 ms │ no change │
│ QQuery 94 │ 38.74 / 39.05 ±0.27 / 39.51 ms │ 40.83 / 41.01 ±0.17 / 41.21 ms │ 1.05x slower │
│ QQuery 95 │ 81.75 / 83.64 ±2.83 / 89.26 ms │ 86.04 / 88.07 ±1.93 / 91.54 ms │ 1.05x slower │
│ QQuery 96 │ 24.68 / 25.24 ±0.90 / 27.00 ms │ 25.53 / 25.74 ±0.13 / 25.87 ms │ no change │
│ QQuery 97 │ 56.93 / 57.70 ±0.59 / 58.67 ms │ 56.99 / 57.80 ±0.50 / 58.35 ms │ no change │
│ QQuery 98 │ 43.05 / 43.83 ±0.57 / 44.60 ms │ 44.90 / 45.65 ±0.77 / 47.10 ms │ no change │
│ QQuery 99 │ 70.62 / 72.89 ±3.44 / 79.75 ms │ 72.47 / 73.09 ±0.71 / 74.39 ms │ no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 10223.26ms │
│ Total Time (codex_hashagg-terminal-output-materialize-v1-20260625) │ 10246.02ms │
│ Average Time (HEAD) │ 103.27ms │
│ Average Time (codex_hashagg-terminal-output-materialize-v1-20260625) │ 103.50ms │
│ Queries Faster │ 2 │
│ Queries Slower │ 9 │
│ Queries with No Change │ 88 │
│ Queries with Failure │ 0 │
└──────────────────────────────────────────────────────────────────────┴────────────┘

Resource Usage
tpcds — base (merge-base)

Metric Value
Wall time 55.0s
Peak memory 2.0 GiB
Avg memory 1.3 GiB
CPU user 232.8s
CPU sys 5.8s
Peak spill 0 B
tpcds — branch

Metric Value
Wall time 55.0s
Peak memory 2.1 GiB
Avg memory 1.4 GiB
CPU user 234.0s
CPU sys 5.8s
Peak spill 0 B
File an issue against this benchmark runner

Looks like the bench here is using SF1, let me repro on my server to see if the big regression is only in SF10 or higher.

@2010YOUY01

2010YOUY01 commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

I remember 5% ish noise is likely for short-running queries

@hhhizzz

hhhizzz commented Jun 26, 2026

Copy link
Copy Markdown
ContributorAuthor

I remember 5% ish noise is likely for short-running queries

Yes, my point is that q23 was only about 1.14x faster on SF1, but on my server, running SF10 produced around a 3x speedup. This makes me wonder whether we should require at least SF10+ runs before merging performance-related PRs.

@adriangbot

adriangbot commented Jun 26, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance:c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
Details

Comparing HEAD and codex_hashagg-terminal-output-materialize-v1-20260625
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ codex_hashagg-terminal-output-materialize-v1-20260625 ┃ Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1.20 / 4.00 ±5.46 / 14.92 ms │ 1.23 / 4.04 ±5.48 / 15.00 ms │ no change │
│ QQuery 1 │ 12.54 / 12.89 ±0.18 / 13.03 ms │ 12.98 / 13.05 ±0.06 / 13.13 ms │ no change │
│ QQuery 2 │ 36.69 / 36.94 ±0.26 / 37.42 ms │ 35.63 / 35.81 ±0.20 / 36.16 ms │ no change │
│ QQuery 3 │ 30.43 / 31.11 ±0.75 / 32.05 ms │ 30.19 / 30.65 ±0.32 / 31.01 ms │ no change │
│ QQuery 4 │ 2411.94 / 2442.52 ±20.27 / 2465.92 ms │ 1652.15 / 1708.50 ±39.67 / 1771.07 ms │ +1.43x faster │
│ QQuery 5 │ 2546.47 / 2634.27 ±73.61 / 2733.48 ms │ 1715.58 / 1801.19 ±80.18 / 1946.40 ms │ +1.46x faster │
│ QQuery 6 │ 1.25 / 1.42 ±0.23 / 1.88 ms │ 1.26 / 1.42 ±0.25 / 1.92 ms │ no change │
│ QQuery 7 │ 13.75 / 14.00 ±0.15 / 14.23 ms │ 14.32 / 14.38 ±0.06 / 14.46 ms │ no change │
│ QQuery 8 │ 2923.75 / 2976.94 ±37.38 / 3035.60 ms │ 2003.37 / 2076.85 ±57.90 / 2160.24 ms │ +1.43x faster │
│ QQuery 9 │ 480.19 / 514.27 ±42.23 / 597.04 ms │ 476.13 / 507.95 ±28.25 / 559.45 ms │ no change │
│ QQuery 10 │ 79.50 / 82.49 ±1.53 / 83.61 ms │ 75.98 / 83.08 ±11.22 / 105.46 ms │ no change │
│ QQuery 11 │ 95.10 / 105.62 ±18.49 / 142.58 ms │ 88.17 / 90.99 ±2.68 / 95.68 ms │ +1.16x faster │
│ QQuery 12 │ 2578.21 / 2713.78 ±102.23 / 2895.53 ms │ 1796.68 / 1911.22 ±92.77 / 2073.01 ms │ +1.42x faster │
│ QQuery 13 │ 1875.47 / 2105.11 ±143.47 / 2319.28 ms │ 447.15 / 555.41 ±100.24 / 725.23 ms │ +3.79x faster │
│ QQuery 14 │ 737.32 / 756.34 ±17.94 / 786.02 ms │ 547.91 / 557.62 ±6.52 / 566.35 ms │ +1.36x faster │
│ QQuery 15 │ 2756.91 / 2808.16 ±46.29 / 2884.49 ms │ 1885.94 / 1960.47 ±67.66 / 2079.77 ms │ +1.43x faster │
│ QQuery 16 │ 6978.66 / 7107.13 ±95.19 / 7218.65 ms │ 4296.75 / 4415.86 ±121.40 / 4631.94 ms │ +1.61x faster │
│ QQuery 17 │ 4218.06 / 4383.42 ±159.67 / 4677.87 ms │ 4217.47 / 4528.07 ±164.82 / 4698.87 ms │ no change │
│ QQuery 18 │ 32116.84 / 32721.53 ±429.69 / 33247.91 ms │ 17924.16 / 18536.93 ±600.67 / 19572.03 ms │ +1.77x faster │
│ QQuery 19 │ 27.56 / 28.84 ±1.61 / 31.93 ms │ 28.24 / 29.04 ±1.05 / 31.08 ms │ no change │
│ QQuery 20 │ 510.63 / 524.87 ±9.22 / 534.82 ms │ 510.69 / 521.76 ±10.68 / 541.31 ms │ no change │
│ QQuery 21 │ 511.34 / 523.84 ±11.54 / 545.55 ms │ 508.99 / 523.43 ±11.55 / 537.12 ms │ no change │
│ QQuery 22 │ 979.33 / 990.55 ±8.95 / 1006.55 ms │ 978.79 / 989.28 ±5.66 / 995.12 ms │ no change │
│ QQuery 23 │ 3009.80 / 3066.50 ±47.18 / 3148.57 ms │ 3017.75 / 3068.06 ±38.96 / 3123.75 ms │ no change │
│ QQuery 24 │ 41.25 / 44.28 ±3.67 / 50.36 ms │ 40.93 / 41.22 ±0.23 / 41.63 ms │ +1.07x faster │
│ QQuery 25 │ 111.25 / 115.40 ±7.09 / 129.53 ms │ 109.81 / 117.52 ±7.88 / 128.50 ms │ no change │
│ QQuery 26 │ 41.49 / 41.91 ±0.33 / 42.34 ms │ 41.47 / 41.91 ±0.42 / 42.53 ms │ no change │
│ QQuery 27 │ 669.77 / 673.09 ±3.06 / 677.88 ms │ 661.64 / 669.67 ±6.48 / 680.18 ms │ no change │
│ QQuery 28 │ 3584.62 / 3755.57 ±133.88 / 3928.41 ms │ 3478.62 / 3633.00 ±114.45 / 3793.43 ms │ no change │
│ QQuery 29 │ 40.60 / 77.27 ±31.64 / 120.02 ms │ 40.52 / 68.44 ±44.09 / 155.75 ms │ +1.13x faster │
│ QQuery 30 │ 703.25 / 723.86 ±16.38 / 742.23 ms │ 549.76 / 564.77 ±13.39 / 589.62 ms │ +1.28x faster │
│ QQuery 31 │ 1031.49 / 1058.02 ±31.39 / 1115.37 ms │ 284.66 / 321.37 ±53.23 / 426.75 ms │ +3.29x faster │
│ QQuery 32 │ 42332.63 / 42545.32 ±358.90 / 43260.75 ms │ 973.36 / 993.73 ±25.73 / 1037.91 ms │ +42.81x faster │
│ QQuery 33 │ 40302.59 / 41884.95 ±1144.35 / 43551.13 ms │ 24525.14 / 27730.32 ±2673.52 / 31755.12 ms │ +1.51x faster │
│ QQuery 34 │ 40435.45 / 42264.76 ±1364.76 / 43879.58 ms │ 25024.43 / 26582.62 ±1352.03 / 29066.98 ms │ +1.59x faster │
│ QQuery 35 │ 1220.32 / 1251.43 ±29.37 / 1301.81 ms │ 984.76 / 1025.84 ±46.16 / 1088.00 ms │ +1.22x faster │
│ QQuery 36 │ 168.56 / 176.76 ±9.55 / 192.61 ms │ 165.88 / 178.42 ±14.02 / 203.90 ms │ no change │
│ QQuery 37 │ 37.11 / 40.46 ±2.28 / 43.05 ms │ 37.14 / 43.11 ±6.69 / 52.51 ms │ 1.07x slower │
│ QQuery 38 │ 41.97 / 43.44 ±0.78 / 44.04 ms │ 41.96 / 45.69 ±2.71 / 48.94 ms │ 1.05x slower │
│ QQuery 39 │ 179.50 / 192.55 ±14.69 / 219.89 ms │ 183.75 / 192.65 ±7.95 / 205.32 ms │ no change │
│ QQuery 40 │ 14.30 / 14.42 ±0.08 / 14.57 ms │ 14.05 / 14.79 ±0.42 / 15.31 ms │ no change │
│ QQuery 41 │ 13.81 / 19.00 ±6.16 / 28.98 ms │ 13.42 / 13.66 ±0.16 / 13.93 ms │ +1.39x faster │
│ QQuery 42 │ 13.09 / 16.89 ±5.51 / 27.69 ms │ 12.69 / 13.06 ±0.23 / 13.36 ms │ +1.29x faster │
└───────────┴────────────────────────────────────────────┴───────────────────────────────────────────────────────┴────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Benchmark Summary ┃ ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ Total Time (HEAD) │ 201525.92ms │
│ Total Time (codex_hashagg-terminal-output-materialize-v1-20260625) │ 106256.87ms │
│ Average Time (HEAD) │ 4686.65ms │
│ Average Time (codex_hashagg-terminal-output-materialize-v1-20260625) │ 2471.09ms │
│ Queries Faster │ 20 │
│ Queries Slower │ 2 │
│ Queries with No Change │ 21 │
│ Queries with Failure │ 0 │
└──────────────────────────────────────────────────────────────────────┴─────────────┘

Resource Usage

clickbench_partitioned — base (merge-base)

MetricValue
Wall time1010.2s
Peak memory12.3 GiB
Avg memory6.5 GiB
CPU user10269.2s
CPU sys476.2s
Peak spill0 B

clickbench_partitioned — branch

MetricValue
Wall time535.1s
Peak memory12.1 GiB
Avg memory6.4 GiB
CPU user4829.7s
CPU sys315.1s
Peak spill0 B

File an issue against this benchmark runner

@hhhizzz

hhhizzz commented Jun 26, 2026

Copy link
Copy Markdown
ContributorAuthor

I remember 5% ish noise is likely for short-running queries

Yes, my point is that q23 was only about 1.14x faster on SF1, but on my server, running SF10 produced around a 3x speedup. This makes me wonder whether we should require at least SF10+ runs before merging performance-related PRs.

I reran the benchmarks on my machine with SF1, and it looks like there's essentially no difference. It seems TPC-DS SF1 just isn't suitable for evaluating this kind of optimization. In hindsight, it also wouldn't have been able to reveal the impact of your previous changes.

That said, I did notice a +42.81x improvement on ClickBench, so the optimization still appears to be effective.😁

@hhhizzz

hhhizzz commented Jun 26, 2026

Copy link
Copy Markdown
ContributorAuthor

@alamb Can you help take a look when you have time?

@alamb

alamb commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

I reran the benchmarks on my machine with SF1, and it looks like there's essentially no difference. It seems TPC-DS SF1 just isn't suitable for evaluating this kind of optimization. In hindsight, it also wouldn't have been able to reveal the impact of your previous changes.

TPCH queries don't have many high cardinality groupings so I am not surprosed they don't show much of a difference (there is one with a subquery I think)

@alamb

alamb commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

clickbench looks much better

alamb
alamb approved these changes Jun 26, 2026
View reviewed changes

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @hhhizzz and @2010YOUY01

Comment threaddatafusion/physical-plan/src/aggregates/aggregate_hash_table/common.rs
Outputting(AggregateHashTableBuffer),
/// Materialize all the output results, and then incrementally output in the `OutputtingMaterializedFinal` state.
///
/// Note this is a temporary solution until the `GroupValues` issue is solved:

@alambalambJun 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 it also keeps the existing behavior (of DataFusion 54)

Comment threaddatafusion/physical-plan/src/aggregates/aggregate_hash_table/final_table.rs
let output_schema = Arc::clone(&self.output_schema);
let batch_size = self.batch_size;
match &mut self.state {
match std::mem::replace(&mut self.state, AggregateHashTableState::Done) {

@alambalambJun 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found it confusing here that self.state was note replaced in this function (so it looks like self.state is lost)

i think what actually happens is that emit_next_materialized_batch sets it

I think it would be easier to understand what is happening if we inlined emit_next_materialized_batch here, but maybe just a comment would be good enough.

@alambalambJun 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a commit with a comment explanining this

@alamb
alamb enabled auto-merge June 26, 2026 18:02
@alamb
alamb added this pull request to the merge queueJun 26, 2026
Merged via the queue into apache:main with commit d58e0c6Jun 26, 2026
39 checks passed
@Rachelint

Rachelint commented Jun 30, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Thanks, the problem exist in partial path now actually, using emit::First is usually a bad idea unless the sorted input case.
I think at least we should rename the emit::First and add comments to tell that.

@hhhizzzhhhizzz mentioned this pull request Jun 30, 2026
Closed
@hhhizzz

hhhizzz commented Jun 30, 2026

Copy link
Copy Markdown
ContributorAuthor

Thanks, Opened #23249 to track this separately.

I described it as an investigation for now because we still need benchmark/profile evidence for the partial path before deciding whether to materialize-once-and-slice there too, or only rename/comment EmitTo::First to make its destructive renumbering semantics clearer.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alambalambalamb approved these changes
@2010YOUY012010YOUY012010YOUY01 approved these changes

Assignees

No one assigned

Labels

physical-planChanges to the physical-plan crate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

HashAggregate output regression after #23055: repeated EmitTo::First maintains unused GroupValues lookup state

6 participants

@hhhizzz@Dandandan@adriangbot@2010YOUY01@alamb@Rachelint