Reduce scalar-quantization benchmark monomorphization by suri-kumkaran · Pull Request #1041 · microsoft/DiskANN

Suryansh Gupta (suri-kumkaran) · 2026-05-07T18:01:41Z

Summary

Trim ScalarQuantized benchmark registrations from 9 → 3 variants to cut diskann-benchmark compile time. Adds an inline doc comment for reintroducing variants.

Changes

diskann-benchmark/src/backend/index/scalar.rs — keep only f32 × {1, 4, 8}-bit registrations and matching impl_sq_build! invocations; drop all f16, i8, and f32/2-bit variants. Add a doc comment with explicit instructions for adding new variants.
diskann-benchmark/example/scalar.json — drop the num_bits: 2 job.
diskann-benchmark/perf_test_inputs/graph_index_scalar_oai_large.json — delete (orphaned; not referenced by any workflow, script, code, or doc).

Disk-index SQ_1_2.0 paths used by CI are a separate code path and are unaffected.

Notes for reviewers

Reintroducing a dropped variant is a documented two-line change: one benchmarks.register(...) + one matching impl_sq_build!(N, T);.

Copilot

Pull request overview

Reduces Rust compile time for diskann-benchmark by trimming the number of scalar-quantization benchmark instantiations (monomorphized (NBITS, T) combinations) that get registered/compiled.

Changes:

Reduce ScalarQuantized registrations to f32 × {1, 4, 8}-bit and remove f32/2-bit, all f16, and i8 variants.
Add inline documentation explaining the monomorphization/compile-time tradeoff and the steps to reintroduce variants.
Update the example scalar benchmark job list to remove the num_bits: 2 job.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
diskann-benchmark/src/backend/index/scalar.rs	Drops most SQ benchmark variants and documents how to add them back to reduce monomorphization/compile time.
diskann-benchmark/example/scalar.json	Removes the 2-bit SQ example job to match the reduced set of registered variants.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Codecov Comments Bot (codecov-commenter) · 2026-05-07T18:23:41Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.48%. Comparing base (9ca5928) to head (f322e6f).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1041   +/-   ##
=======================================
  Coverage   89.47%   89.48%           
=======================================
  Files         460      460           
  Lines       85369    85369           
=======================================
+ Hits        76387    76389    +2     
+ Misses       8982     8980    -2

Flag	Coverage Δ
miri	`89.48% <ø> (+<0.01%)`	⬆️
unittests	`89.32% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
diskann-benchmark/src/backend/index/scalar.rs	`100.00% <ø> (ø)`

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Aditya Krishnan (arkrishn94)

Just left a small comment about the subset being deleted.

@suhasjs

# DiskANN v0.52.0 Release Notes ## Breaking Changes An AI generated, human reviewed list of changes is summarized below. ### `get_degree_stats` signature changed ([#998](#998)) `DiskANNIndex::get_degree_stats` now takes an explicit iterator of IDs instead of requiring the data provider to implement `IntoIterator`. ```rust // Before — provider had to impl IntoIterator index.get_degree_stats(&mut accessor)?; // After — caller supplies the ID iterator index.get_degree_stats(&mut accessor, id_iter)?; ``` ### PQ dimension contract tightened; entries now `&[f32]` only ([#1044](#1044)) With `AlignedBoxWithSlice` removed from the PQ path, the dimension handling has been refactored into a three-layer contract: | Layer | Where | Contract | |---|---|---| | **Boundary (inmem)** | `QueryComputer::new`, `MultiQueryComputer::new`, `DistanceComputer::evaluate_similarity` | `len == dim` (returns `Err` on mismatch) | | **Boundary (disk)** | `PQScratch::set` | `len >= dim`, slices to `[..dim]` | | **Internal** | `TableL2/IP/Cosine::{new, populate}` | Trusted — no re-validation | **Other changes:** - PQ table populate/distance methods now accept `&[f32]` instead of `<U: Into<f32>>`. Callers must pre-decode quantized vectors via `VectorRepr::as_f32`. - Generic trampoline impls (`&Vec<u8>`, `&&[u8]`) on `QueryComputer` / `DistanceComputer` have been removed. ### `calculate_chunk_offsets` relocated to `ChunkOffsets` constructors ([#976](#976)) The free functions `calculate_chunk_offsets` and `calculate_chunk_offsets_auto` have been moved into constructors on `ChunkOffsets` / `ChunkOffsetsView` in `diskann-quantization::views`. ```rust // Before let offsets = calculate_chunk_offsets(dim, num_chunks); // After (allocating) let offsets = ChunkOffsets::partition(dim, num_chunks)?; // After (zero-alloc, borrows caller-owned scratch) let view = ChunkOffsetsView::partition_into(dim, &mut scratch)?; ``` Additionally, `get_chunk_from_training_data` has been moved from public API. ### `CachingProvider` removed ([#1052](#1052)) The entire `diskann_providers::model::graph::provider::async_::caching` module has been deleted. **Why:** The `CachingProvider` was an experiment in transparent caching over `DataProvider`. In practice it required double monomorphization of the indexing code, didn't save integration work for bulk methods like `on_elements_unordered`/`distances_unordered`, and was complex to maintain. An internal user who …migrated off it removed ~1,000 lines of code, improved compile times by ~20%, and substantially reduced complexity. **Upgrade:** Manage caching directly in your `DataProvider` implementation. ## New Features ### AVX-512 4-bit distance kernels ([#1045](#1045)) Native V4 (AVX-512) specializations for 4-bit packed vector distance computations: - **`SquaredL2`** — 16 × `u32` lanes per iteration via `_mm512_madd_epi16`. - **`InnerProduct`** — AVX-512 VNNI (`_mm512_dpbusd_epi32`) over `u8x64` / `i8x64` operands. Previously, V4 hardware fell back to two AVX2 (V3) kernel invocations per 512-bit chunk. The native kernels double per-instruction throughput. No API changes — existing code benefits automatically on AVX-512 capable hardware. ## Merged PRs * Deprecate 32-bit targets by @suhasjs in #1022 * Add a fast path to `Map::prepare`. by @hildebrandmw in #1023 * Add boundary checks in gen_associated_data_from_range() by @Copilot in #847 * [deps] Don't pull `rayon` as a dependency of `diskann`. by @hildebrandmw in #1024 * Bump openssl from 0.10.78 to 0.10.79 by @dependabot[bot] in #1026 * Cleaning up test work and changing the get_degree_stats signature. by @JordanMaples in #998 * Reduce scalar-quantization benchmark monomorphization by @suri-kumkaran in #1041 * [diskann-vector] Support truly unaligned distances. by @hildebrandmw in #981 * rename spherical.json to graph index with spherical quantization by @harsha-simhadri in #1042 * [PQ Cleanup] Part 2: Consolidate `calculate_chunk_offsets*` by @arkrishn94 in #976 * PQ: tighten dim contract; right-size scratch buffer by @wuw92 in #1044 * Add v4 distance kernels (4-bit SquaredL2 / InnerProduct) by @m3hm3t in #1045 * Remove the Caching Provider by @hildebrandmw in #1052 ## New Contributors * @suhasjs made their first contribution in #1022 * @m3hm3t made their first contribution in #1045 **Full Changelog**: v0.51.0...v0.52.0 Co-authored-by: Mark Hildebrand <mhildebrand@microsoft.com>

## Summary Trim `ScalarQuantized` benchmark registrations from **9 → 3** variants to cut `diskann-benchmark` compile time. Adds an inline doc comment for reintroducing variants. ## Changes - **`diskann-benchmark/src/backend/index/scalar.rs`** — keep only `f32` × {1, 4, 8}-bit registrations and matching `impl_sq_build!` invocations; drop all `f16`, `i8`, and `f32`/2-bit variants. Add a doc comment with explicit instructions for adding new variants. - **`diskann-benchmark/example/scalar.json`** — drop the `num_bits: 2` job. - **`diskann-benchmark/perf_test_inputs/graph_index_scalar_oai_large.json`** — delete (orphaned; not referenced by any workflow, script, code, or doc). Disk-index `SQ_1_2.0` paths used by CI are a separate code path and are unaffected. ## Notes for reviewers Reintroducing a dropped variant is a documented two-line change: one `benchmarks.register(...)` + one matching `impl_sq_build!(N, T);`.

@suhasjs

An AI generated, human reviewed list of changes is summarized below. ([microsoft#998](microsoft#998)) `DiskANNIndex::get_degree_stats` now takes an explicit iterator of IDs instead of requiring the data provider to implement `IntoIterator`. ```rust // Before — provider had to impl IntoIterator index.get_degree_stats(&mut accessor)?; // After — caller supplies the ID iterator index.get_degree_stats(&mut accessor, id_iter)?; ``` ([microsoft#1044](microsoft#1044)) With `AlignedBoxWithSlice` removed from the PQ path, the dimension handling has been refactored into a three-layer contract: | Layer | Where | Contract | |---|---|---| | **Boundary (inmem)** | `QueryComputer::new`, `MultiQueryComputer::new`, `DistanceComputer::evaluate_similarity` | `len == dim` (returns `Err` on mismatch) | | **Boundary (disk)** | `PQScratch::set` | `len >= dim`, slices to `[..dim]` | | **Internal** | `TableL2/IP/Cosine::{new, populate}` | Trusted — no re-validation | **Other changes:** - PQ table populate/distance methods now accept `&[f32]` instead of `<U: Into<f32>>`. Callers must pre-decode quantized vectors via `VectorRepr::as_f32`. - Generic trampoline impls (`&Vec<u8>`, `&&[u8]`) on `QueryComputer` / `DistanceComputer` have been removed. ([microsoft#976](microsoft#976)) The free functions `calculate_chunk_offsets` and `calculate_chunk_offsets_auto` have been moved into constructors on `ChunkOffsets` / `ChunkOffsetsView` in `diskann-quantization::views`. ```rust // Before let offsets = calculate_chunk_offsets(dim, num_chunks); // After (allocating) let offsets = ChunkOffsets::partition(dim, num_chunks)?; // After (zero-alloc, borrows caller-owned scratch) let view = ChunkOffsetsView::partition_into(dim, &mut scratch)?; ``` Additionally, `get_chunk_from_training_data` has been moved from public API. ([microsoft#1052](microsoft#1052)) The entire `diskann_providers::model::graph::provider::async_::caching` module has been deleted. **Why:** The `CachingProvider` was an experiment in transparent caching over `DataProvider`. In practice it required double monomorphization of the indexing code, didn't save integration work for bulk methods like `on_elements_unordered`/`distances_unordered`, and was complex to maintain. An internal user who …migrated off it removed ~1,000 lines of code, improved compile times by ~20%, and substantially reduced complexity. **Upgrade:** Manage caching directly in your `DataProvider` implementation. ([microsoft#1045](microsoft#1045)) Native V4 (AVX-512) specializations for 4-bit packed vector distance computations: - **`SquaredL2`** — 16 × `u32` lanes per iteration via `_mm512_madd_epi16`. - **`InnerProduct`** — AVX-512 VNNI (`_mm512_dpbusd_epi32`) over `u8x64` / `i8x64` operands. Previously, V4 hardware fell back to two AVX2 (V3) kernel invocations per 512-bit chunk. The native kernels double per-instruction throughput. No API changes — existing code benefits automatically on AVX-512 capable hardware. * Deprecate 32-bit targets by @suhasjs in microsoft#1022 * Add a fast path to `Map::prepare`. by @hildebrandmw in microsoft#1023 * Add boundary checks in gen_associated_data_from_range() by @Copilot in microsoft#847 * [deps] Don't pull `rayon` as a dependency of `diskann`. by @hildebrandmw in microsoft#1024 * Bump openssl from 0.10.78 to 0.10.79 by @dependabot[bot] in microsoft#1026 * Cleaning up test work and changing the get_degree_stats signature. by @JordanMaples in microsoft#998 * Reduce scalar-quantization benchmark monomorphization by @suri-kumkaran in microsoft#1041 * [diskann-vector] Support truly unaligned distances. by @hildebrandmw in microsoft#981 * rename spherical.json to graph index with spherical quantization by @harsha-simhadri in microsoft#1042 * [PQ Cleanup] Part 2: Consolidate `calculate_chunk_offsets*` by @arkrishn94 in microsoft#976 * PQ: tighten dim contract; right-size scratch buffer by @wuw92 in microsoft#1044 * Add v4 distance kernels (4-bit SquaredL2 / InnerProduct) by @m3hm3t in microsoft#1045 * Remove the Caching Provider by @hildebrandmw in microsoft#1052 * @suhasjs made their first contribution in microsoft#1022 * @m3hm3t made their first contribution in microsoft#1045 **Full Changelog**: microsoft/DiskANN@v0.51.0...v0.52.0 Co-authored-by: Mark Hildebrand <mhildebrand@microsoft.com>

Remove unsed Scalar benchmarks

398ab22

Suryansh Gupta (suri-kumkaran) requested review from a team and Copilot May 7, 2026 18:01

Mark Hildebrand (hildebrandmw) approved these changes May 7, 2026

View reviewed changes

Copilot started reviewing on behalf of Suryansh Gupta (suri-kumkaran) May 7, 2026 18:08 View session

Copilot AI reviewed May 7, 2026

View reviewed changes

Comment thread diskann-benchmark/src/backend/index/scalar.rs Outdated

Cleanup unsed scalar benchmark config file

f322e6f

Suryansh Gupta (suri-kumkaran) enabled auto-merge (squash) May 7, 2026 20:30

Aditya Krishnan (arkrishn94) approved these changes May 7, 2026

View reviewed changes

Comment thread diskann-benchmark/src/backend/index/scalar.rs

Mark Hildebrand (hildebrandmw) approved these changes May 7, 2026

View reviewed changes

Suryansh Gupta (suri-kumkaran) merged commit e222415 into main May 8, 2026
26 checks passed

Suryansh Gupta (suri-kumkaran) deleted the users/suryangupta/cleanup-scalar-benchmarks branch May 8, 2026 03:49

Mark Hildebrand (hildebrandmw) mentioned this pull request May 12, 2026

Set version to v0.52.0 #1056

Merged

Navigation Menu

Uh oh!

Reduce scalar-quantization benchmark monomorphization - #1041

Reduce scalar-quantization benchmark monomorphization#1041
Suryansh Gupta (suri-kumkaran) merged 2 commits into
mainfrom
users/suryangupta/cleanup-scalar-benchmarks

Suryansh Gupta (suri-kumkaran) commented May 7, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Codecov Comments Bot (codecov-commenter) commented May 7, 2026 •

edited

Loading

Uh oh!

Aditya Krishnan (arkrishn94) left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

Suryansh Gupta (suri-kumkaran) commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Notes for reviewers

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Codecov Comments Bot (codecov-commenter) commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Aditya Krishnan (arkrishn94) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Suryansh Gupta (suri-kumkaran) commented May 7, 2026 •

edited

Loading

Codecov Comments Bot (codecov-commenter) commented May 7, 2026 •

edited

Loading