Skip to content

[multi-vector] Verify Standard won't overflow in its constructor. - #757

Merged
Mark Hildebrand (hildebrandmw) merged 1 commit into
mainfrom
mhildebr/standard-new
Feb 11, 2026
Merged

[multi-vector] Verify Standard won't overflow in its constructor.#757
Mark Hildebrand (hildebrandmw) merged 1 commit into
mainfrom
mhildebr/standard-new

Conversation

@hildebrandmw

Copy link
Copy Markdown
Contributor

Move the check that:

  1. nrows * ncols will not overflow
  2. nrows * ncols * std::mem::size_of::<T>() will not exceed isize::MAX

into the constructor Standard::new(). This allows the calculation of the number of elements and allocation sizes to be performed with safely with reckless abandon in implementation code as we no longer need to worry about overflow. The constructor Standard::new() can now return an error, which is slightly less ergonomic, but I think the improved safety in the implementation is worth it.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the multi_vector::Standard matrix representation by moving overflow/size validation into Standard::new(), so downstream element-count and allocation-size computations can rely on a checked constructor invariant.

Changes:

  • Change Standard::new(nrows, ncols) to return Result<Standard<T>, Overflow> and validate element-count overflow and isize::MAX allocation bounds.
  • Make Standard::num_elements() infallible (usize) and remove the SliceError::Overflow path.
  • Update docs/tests/compile-fail fixtures to handle the new fallible constructor and add targeted overflow tests.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
diskann-quantization/src/multi_vector/matrix.rsImplements fallible Standard::new with overflow checks, adds Overflow error, updates num_elements and tests.
diskann-quantization/src/multi_vector/mod.rsUpdates doctest examples and re-exports the new Overflow type.
diskann-quantization/src/multi_vector/distance/simple.rsUpdates doctest/test helpers to unwrap the now-fallible Standard::new.
diskann-quantization/src/multi_vector/distance/mod.rsUpdates module-level doctest examples to unwrap Standard::new.
diskann-quantization/src/minmax/multi/mod.rsUpdates doctest examples to unwrap Standard::new.
diskann-quantization/src/minmax/multi/meta.rsUpdates tests to unwrap Standard::new when building MatRef.
diskann-quantization/src/minmax/multi/max_sim.rsUpdates tests to unwrap Standard::<f32>::new.
diskann-quantization/tests/compile-fail/multi/matref_rows.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/matref_get_row.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/matmut_rows_mut.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/matmut_rows.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/matmut_reborrow_mut.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/matmut_reborrow.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/matmut_get_row_mut.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/matmut_get_row.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/matmut_as_view_borrows.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/mat_rows_mut.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/mat_rows.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/mat_reborrow_mut.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/mat_reborrow.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/mat_get_row_mut.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/mat_get_row.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/mat_as_view_mut.rsUpdates compile-fail fixture to unwrap Standard::new.
diskann-quantization/tests/compile-fail/multi/mat_as_view.rsUpdates compile-fail fixture to unwrap Standard::new.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threaddiskann-quantization/src/multi_vector/matrix.rs
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.00%. Comparing base (e873811) to head (8a662c4).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@ Coverage Diff @@## main #757 +/- ##
=======================================
Coverage 88.99% 89.00% =======================================
Files 428 428 Lines 78235 78291 +56 =======================================
+ Hits 69627 69680 +53 - Misses 8608 8611 +3 
FlagCoverage Δ
miri89.00% <100.00%> (+<0.01%)⬆️
unittests89.00% <100.00%> (+<0.01%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing linesCoverage Δ
diskann-quantization/src/minmax/multi/max_sim.rs98.06% <100.00%> (ø)
diskann-quantization/src/minmax/multi/meta.rs97.66% <100.00%> (+0.03%)⬆️
...n-quantization/src/multi_vector/distance/simple.rs98.43% <100.00%> (ø)
diskann-quantization/src/multi_vector/matrix.rs97.30% <100.00%> (-0.85%)⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

CopilotAI commented Feb 11, 2026

Copy link
Copy Markdown
Contributor

Mark Hildebrand (@hildebrandmw) I've opened a new pull request, #758, to work on those changes. Once the pull request is ready, I'll request review from you.

@hildebrandmw

Copy link
Copy Markdown
ContributorAuthor

Mark Hildebrand (@hildebrandmw) I've opened a new pull request, #758, to work on those changes. Once the pull request is ready, I'll request review from you.

Ugh - that's not what I meant to happen 😞.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense

@hildebrandmw
Mark Hildebrand (hildebrandmw) merged commit 26fd1af into mainFeb 11, 2026
26 checks passed
@hildebrandmw
Mark Hildebrand (hildebrandmw) deleted the mhildebr/standard-new branch February 11, 2026 00:38
Mark Hildebrand (hildebrandmw) added a commit that referenced this pull request Feb 13, 2026
## What's Changed
### API Breaking Changes
* Remove the `experimental_avx512` feature. by @hildebrandmw in
#732
* Use VirtualStorageProvider::new_overlay(test_data_root()) in tests by
@Copilot in #726
* save and load max_record_size and leaf_page_size for bftrees by
@backurs in #724
* [multi-vector] Verify `Standard` won't overflow in its constructor. by
@hildebrandmw in #757
* VirtualStorageProvider: Make new() private, add new_physical by
@Copilot in #764
* [minmax] Refactor full query by @arkrishn94 in
#770
* Bump diskann-quantization to edition 2024. by @hildebrandmw in
#772
### Additions
* [multi-vector] Enable cloning of `Mat` and friends. by @hildebrandmw
in #759
* adding bftreepaths in mod.rs by @backurs in
#775
* [quantization] Add `as_raw_ptr`. by @hildebrandmw in
#774
### Bug Fixes
* Fix `diskann` compilation without default-features and add CI tests.
by @hildebrandmw in #722
### Docs and Comments
* Updating the benchmark README to use diskann-benchmark by @bryantower
in #709
* Fix doc comment: Windows line endings are \r\n not \n\r by @Copilot in
#717
* Fix spelling errors in streaming API documentation by @Copilot in
#715
* Add performance diagnostic to `diskann-benchmark` by @hildebrandmw in
#744
* Add agents.md onboarding guide for coding agents by @Copilot in
#765
* [doc] Fix lots of little typos in `diskann-wide` by @hildebrandmw in
#771
### Performance
* [diskann-wide] Optimize `load_simd_first` for 8-bit and 16-bit element
types. by @hildebrandmw in #747
### Dependencies
* Bump bytes from 1.11.0 to 1.11.1 by @dependabot[bot] in
#723
* [diskann] Add note on the selection of `PruneKind` in
`graph::config::Builder`. by @hildebrandmw in
#734
* [diskann-providers] Remove the LRU dependency and make `vfs` and
`serde_json` optional. by @hildebrandmw in
#733
### Infrastructure
* Add initial QEMU tests for `diskann-wide`. by @hildebrandmw in
#719
* [CI] Skip coverage for Dependabot. by @hildebrandmw in
#725
* Add miri test coverage to CI workflow by @Copilot in
#729
* [CI] Add minimal ARM checks by @hildebrandmw in
#745
* Enable CodeQL security analysis by @Copilot in
#754
## New Contributors
* @backurs made their first contribution in
#724
* @arkrishn94 made their first contribution in
#770
**Full Changelog**:
0.45.0...0.46.0
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@hildebrandmw@codecov-commenter@metajack@arkrishn94