fix(parity): NULL ordering in ORDER BY — P17 + P13 stage 2 - #64
Merged
Conversation
Closes both halves of the ORDER BY comparator's NULL rule in one slice, since they are the same comparator and both decisions were taken together (2026-08-02). Parity 141 -> 152 AGREE; eleven cases in one change. Default: NULLs now sort LAST in both directions, following the reference engine. Previously NULL sorted as the minimum value, so ASC put them first. The standard leaves this implementation-defined and the major engines disagree, so this is a recorded choice, not a correction — see the CHANGELOG's behaviour-change note. Explicit NULLS FIRST / NULLS LAST is now parsed and honoured, per ORDER BY item. NULLS, FIRST and LAST are matched contextually rather than promoted to keywords: all three are plausible column names, and reserving them would break unrelated queries against arbitrary CSV headers. A malformed clause errors rather than being ignored, per P13 stage 1's rule. One comparator, datavalue_compare::compare_for_order_by, now serves both sort sites — that shared function, not the corpus cases, is what stops them drifting apart again. The window site was worse than "a different rule": it compared through DataValue's derived PartialOrd, where Null is the last variant and so sorted as the MAXIMUM before DESC reversed it into first place. The same derived ordering compared cross-type values by variant rather than value, so Integer never met Float numerically; routing it through the shared comparator fixed that too. compare_datavalues is deliberately untouched — it is shared with aggregates and TUI sorting, where NULL-as-minimum is a different question. Also here: - examples/jsonl_logs.sql: [SKIP] dropped; both forms now return the same top 5. - corpus: renamed order_by_nulls_first_limit_nullfree, which had collided with the null_edges case of the same name so --check reported one id twice. docs/SQL_PARITY.md P13/P17 updated; P37 moves to NEXT. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015fQ6qjYnQjmUAXgaQMk4Qq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes both halves of the
ORDER BYcomparator's NULL rule in one slice. Theyare the same comparator and both decisions were taken together on 2026-08-02, so
this was implementation rather than deliberation.
Parity 141 → 152 AGREE (177 cases, contract holds). Eleven cases closed in
one change — the largest single movement in the effort so far.
What changes for users
Previously NULL sorted as the minimum value, so
ORDER BY scoreput NULLs firstand
ORDER BY score DESCput them last.This is a recorded choice, not a bug fix — standard SQL leaves NULL placement
implementation-defined and the major engines genuinely disagree (SQLite/MySQL
treat NULL as smallest, PostgreSQL as largest, DuckDB pins NULLS LAST). We follow
the reference engine, which is the whole point of having one. Anyone relying on
NULLs-first in an ascending sort now writes it explicitly. Called out in
CHANGELOG.md.✨
NULLS FIRST/NULLS LASTis now parsed and honoured, per ORDER BY item:Before this the clause was not parsed at all. Until P13 stage 1 it was silently
discarded along with every clause after it, so
ORDER BY amount DESC NULLS LAST LIMIT 3quietly returned every row instead of 3; since stage 1 it has been aparse error. Both are now gone.
Three things worth reviewing
1. One shared comparator, not two matching edits.
datavalue_compare::compare_for_order_by(a, b, ascending, nulls_first)nowserves both sort sites —
DataView::apply_multi_sortandwindow_context::compare_by_sort_cols. That shared function, not the corpuscases, is what stops the two drifting apart again; the cases only cover one shape
each. NULL placement is applied before direction and is never reversed by it —
NULLS LASTmeans last in the output whichever way the values sort. A comparatorthat reversed the NULL arm along with the values would pass every ASC case and
fail the DESC ones, so both directions are asserted.
compare_datavaluesis deliberately untouched. It still sorts NULL as theminimum because it is shared with aggregates,
MIN/MAXand TUI column sorting,where that is not the same question.
2. The window site was worse than the finding recorded.
It was filed as "follows a different NULL rule". In fact it compared
DataValuesthrough their derived
PartialOrd, which orders by variant index — andNullis the last variant, so NULL sorted as the maximum, then got reversed by DESC
into first place. The same derived ordering compared cross-type values by variant
rather than by value, so
Integer(100)sorted belowFloat(1.0)in a window'sinternal sort. Routing this site through the shared comparator fixed that too; it
has its own regression test.
3.
NULLS,FIRSTandLASTare not reserved words.They are matched contextually, in the one position they can appear. All three are
plausible column names —
firstandlastespecially — and reserving them wouldbreak queries that have nothing to do with NULL ordering, in a tool that reads
user CSVs with arbitrary headers.
SELECT nulls FROM t ORDER BY nullsstillparses, and there is a test that says so. A malformed clause errors naming the
offending token rather than being ignored, per P13 stage 1's rule.
Also here
examples/jsonl_logs.sql: the[SKIP]is dropped. Both forms now return theidentical top 5 — the filtered set contains a row with a NULL
latency_ms, soNULLS LASTis doing real work there and the default agreeing with it is theP17 change visible end-to-end.
order_by_nulls_first_limit, so--checkreported one id twice and an id filter would have run both. The NULL-free one
is now
order_by_nulls_first_limit_nullfree.NullsOrder::Unspecifiedis distinct fromLasteven though the two mean thesame thing to the comparator. Printing
NULLS LASTonto a query the user neverwrote it in would be a silent edit.
Not fixed, deliberately
A third comparator exists:
csv_datasource.rs::sort_resultssortsserde_json::Values and still places NULLs first. It is unreachable — it hangsoff
CsvApiClient, whichbuffer.rskeeps only "for API compatibility" and nevercalls, and the
DataSourceAdapterthat would reach it has no callers either.Left as-is rather than fixed blind, and recorded in the P17 entry so that whoever
revives that path knows it needs the same rule. Reviving it without this is a
silent divergence, not a compile error.
R11 (ORDER BY's private column resolver) was kept out of this slice — it is a
separate behaviour change on unquoted dotted names and wants its own parity run.
Verification
holds. All five
null_edges.csvacceptance cases flipped, including the sharpone (
order_by_nulls_first_limitreturns exactly ids 3, 10, 11), plus thewindow second site
win_first_value_unfiltered.cargo test: 749 + 469 pass, 0 failures. New unit tests cover the comparator(both directions, missing-cell vs explicit NULL, mixed numerics) and the parser
(per-item clause, case-insensitivity, non-reserved words, loud error).
expander_rewritersquery 7, the example deliberately left failing for P37.cargo fmtclean; no new clippy findings.The acceptance criteria mattered here: the three original
order_by_nulls_*cases run on a NULL-free fixture, so an implementation that parsed the clause into
the AST and then ignored it would have flipped all three to AGREE and passed the
gate. The
null_edges.csvcases added the day before were the real check.Docs:
docs/SQL_PARITY.mdP13 and P17 updated; P37 moves to NEXT in the fixqueue.
🤖 Generated with Claude Code
https://claude.ai/code/session_015fQ6qjYnQjmUAXgaQMk4Qq