Skip to content

fix(engine): give MODE a deterministic tie-break (P41) - #67

Merged
TimelordUK merged 1 commit into
mainfrom
fix/p41-mode-tiebreak
Sep 6, 2026
Merged

fix(engine): give MODE a deterministic tie-break (P41)#67
TimelordUK merged 1 commit into
mainfrom
fix/p41-mode-tiebreak

Conversation

@TimelordUK

Copy link
Copy Markdown
Owner

MODE tallied into a HashMap and took the highest count with no tie-break rule, so on a tie the winner was whichever entry hash iteration surfaced last. The same binary on the same data returned different answers between runs: six runs of the 25/25 parity tie gave 0 1 1 0 0 1.

Break ties by first occurrence in the input, which is what DuckDB does (mode.cpp tracks a first_row per distinct value and compares count > best.count || (count == best.count && first_row < best.first_row)). The entry had proposed "smallest value wins"; probing the reference showed that is a different rule — ('b','b','a','a') gives 'b', and (5,3,9,1) gives 5, not 1 — so this follows the reference per the doc's own convention.

The finding named ModeState in src/sql/aggregates/mod.rs. Fixing it there changed nothing: there are two aggregate registries and ArithmeticEvaluator checks the newer one first, so the live MODE is CollectorState in src/sql/aggregate_functions/mod.rs. Both are fixed here; the duplication is filed as R12.

  • Corpus: four new cases in tier 10, including one where first-seen and "smallest wins" disagree, so the two rules are distinguishable. 177 -> 181 cases, 152 -> 156 AGREE, no other bucket moved.
  • Rust regression tests in both modules, six cases each.
  • examples/stats_examples.sql and statistical_analysis.sql promoted to FORMAL — they were blocked on this exact nondeterminism.
  • Spun off P42 (MODE is numeric-only and returns a float for integers) and P43 (our RANGE stop bound is inclusive; DuckDB's is exclusive).

Claude-Session: https://claude.ai/code/session_01JBEUeckCwmWXoWTpQUTDqP

MODE tallied into a HashMap and took the highest count with no tie-break
rule, so on a tie the winner was whichever entry hash iteration surfaced
last. The same binary on the same data returned different answers between
runs: six runs of the 25/25 parity tie gave `0 1 1 0 0 1`.

Break ties by first occurrence in the input, which is what DuckDB does
(mode.cpp tracks a first_row per distinct value and compares
`count > best.count || (count == best.count && first_row < best.first_row)`).
The entry had proposed "smallest value wins"; probing the reference showed
that is a different rule — ('b','b','a','a') gives 'b', and (5,3,9,1) gives
5, not 1 — so this follows the reference per the doc's own convention.

The finding named `ModeState` in src/sql/aggregates/mod.rs. Fixing it there
changed nothing: there are two aggregate registries and ArithmeticEvaluator
checks the newer one first, so the live MODE is CollectorState in
src/sql/aggregate_functions/mod.rs. Both are fixed here; the duplication is
filed as R12.

- Corpus: four new cases in tier 10, including one where first-seen and
  "smallest wins" disagree, so the two rules are distinguishable. 177 -> 181
  cases, 152 -> 156 AGREE, no other bucket moved.
- Rust regression tests in both modules, six cases each.
- examples/stats_examples.sql and statistical_analysis.sql promoted to
  FORMAL — they were blocked on this exact nondeterminism.
- Spun off P42 (MODE is numeric-only and returns a float for integers) and
  P43 (our RANGE stop bound is inclusive; DuckDB's is exclusive).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBEUeckCwmWXoWTpQUTDqP
@TimelordUK
TimelordUK merged commit 6df0765 into main Sep 6, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant