fix: restore the merge_collapse invariant to prevent quadratic merges

[hey-jj]

Sorting uniform random input takes a number of comparisons that grows as n^2, against the O(n log n) the crate documents (src/lib.rs:2-3). Measured with a counting comparator on deterministic xorshift64* input, release build:

n	comparisons before	comparisons after this PR	std stable sort
128,000	130,193,008	3,538,277	2,285,401
1,000,000	7,952,525,755	30,461,538	20,801,825

At n = 1,000,000 that is 12.0 s before and 87 ms after on the same machine and data. The output is correctly sorted and stable in both cases. The defect surfaces purely as running time.

Root cause

Two spots in merge_collapse (src/sort.rs):

1. The first invariant check reads runs[l - 1].len <= runs[l - 2].len + runs[l - 1].len. That has the form x <= y + x, which holds for every usize, so the branch fires whenever the stack holds three runs. The corrected mergeCollapse from the envisage-project writeup cited in the comment above the function compares the third run from the top: runs[l - 3].len <= runs[l - 2].len + runs[l - 1].len.
2. When the invariant disjunction fails, the loop breaks unconditionally. The cited algorithm still merges the top two runs while runLen[n] <= runLen[n + 1] and only breaks otherwise. Without that rule nothing ever merges at stack depth 2.

Combined effect: the run stack keeps at most two runs beyond a transient third, every new run of length about min_run merges into one accumulating run, and total work is Theta(n^2 / min_run).

One more change rides along. The debug_assert_eq!(run1.len, run2.pos) in merge_force_collapse compares a run length to an absolute position. The intended adjacency check is run1.pos + run1.len == run2.pos, the form merge_collapse already asserts. It only held because the broken invariant kept run1.pos at 0, and it fires as soon as the merge rules are fixed.

Fix

Restore both rules so the loop maintains the standard timsort invariant, and correct the debug_assert. The merge-target selection and the merge itself are unchanged.

Verification

• cargo test passes in debug and release (54 tests, including the new one).
• Fuzz against slice::sort_by: 8,240 arrays across sizes 0 to 1,000,000 (including six sizes above 2^16), covering random, heavy-duplicate, sorted, reverse-sorted, sawtooth, organ-pipe, mostly-sorted, and adversarial run-block patterns, sorting (key, index) pairs so equality with std's stable sort also checks stability. 0 mismatches, with debug-assertions = true.
• The corrected debug_assert passes throughout. The uncorrected one fires at n = 2,000 once the merge rules are fixed.
• New regression test merge_collapse_comparison_bound counts comparisons on deterministic random input at n = 100,000 and asserts they stay under 3 * n * ceil(log2 n). With src/sort.rs reverted to master the test fails, and with this PR it passes.

This affects the published 0.1.3 as well, since the merge_collapse body there is byte-identical to master.

Summary by CodeRabbit

• Bug Fixes

  • Improved sorting reliability for complex data patterns.
  • Corrected merge handling to preserve proper ordering and performance.

• Tests

  • Added regression coverage for large randomized inputs.
  • Verified sorting correctness and expected O(n log n) comparison performance.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e8846947-f9db-45c9-98c2-ad278dd98358

📥 Commits

Reviewing files that changed from the base of the PR and between 4fcaf7f and 420e1ff.

📒 Files selected for processing (2)

• src/sort.rs
• src/sort/tests.rs

---

📝 Walkthrough

Walkthrough

Changes

The merge stack logic now checks run availability and adjacency before merging. A deterministic randomized test verifies sorting correctness and enforces a 3n log₂(n) comparison bound for 100,000 elements.

Merge stack invariant correction

Layer / File(s)	Summary
Run-stack collapse corrections src/sort.rs	merge_collapse safely evaluates stack invariants and merges the correct adjacent runs. merge_force_collapse verifies adjacency with each run’s position and length.
Sorting complexity regression test src/sort/tests.rs	A deterministic xorshift64* test checks sorted output and limits comparator calls for 100,000 elements.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes restoring the merge_collapse invariant to prevent quadratic merges.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}