TurboQuantSharp

A from-scratch C# / .NET implementation of Google Research's TurboQuant algorithm - a data-oblivious vector quantizer that compresses high-dimensional embeddings 8–16× and searches them with a SIMD FastScan kernel. No codebook training, no separate train phase: add vectors and they're indexed.

TurboQuantSharp is a clean-room .NET port inspired by turbovec (the Rust/Python implementation of the same algorithm).

• Online ingest. Add vectors and they're quantized immediately - no train step, no parameter tuning, no rebuilds as the corpus grows.
• SIMD FastScan search. A u8 nibble-LUT scored with byte-shuffles, dispatched to the widest register available at runtime: AVX-512BW (Vector512) → AVX2 (Vector256) → portable Vector128 (which also lowers to ARM NEON), with an exact-double scalar fallback. 6.7–8.8× faster than scalar on AVX2.
• 2, 3 and 4-bit quantization - trade recall for memory (a 1536-dim float32 vector shrinks from 6,144 bytes to 768 bytes at 4-bit, 384 bytes at 2-bit).
• Filter at search time.IdMapIndex carries stable ulong ids that survive deletes and accepts an id allowlist, honoured inside the search.
• Pure local. Managed .NET, no native dependencies, no data leaving your process.

Quick start

usingTurboQuantSharp.Core;usingTurboQuantSharp.Search;varindex=newTurboQuantIndex(dimension:1536,bitWidth:4);index.Add(vectors);index.Add(moreVectors);SearchResultsresults=index.Search(queries,k:10);ReadOnlySpan<long>ids=results.IndicesForQuery(0);ReadOnlySpan<double>scores=results.ScoresForQuery(0);index.Write("index.tqs");TurboQuantIndexloaded=TurboQuantIndex.Load("index.tqs");

Don't know the dimension up front? Construct lazily and let the first add lock it in:

varindex=TurboQuantIndex.CreateLazy(bitWidth:4);index.Add(vectors,dimension:1536);

Stable ids that survive deletes

usingTurboQuantSharp.Core;usingTurboQuantSharp.Search;varindex=newIdMapIndex(dimension:1536,bitWidth:4);index.AddWithIds(vectors,newulong[]{1001,1002,1003});IdSearchResultsresults=index.Search(queries,k:10);ReadOnlySpan<ulong>hits=results.IdsForQuery(0);index.Remove(1002);index.Write("index.tqsm");IdMapIndexreloaded=IdMapIndex.Load("index.tqsm");

Hybrid retrieval (filtered search)

Restrict results to a candidate set produced by another system (SQL, BM25, ACL, time window, …):

ulong[]allowed=GetCandidateIds();IdSearchResultsresults=index.Search(queries,k:10,allowlist:allowed);

The plain TurboQuantIndex.Search accepts an optional bool[] slot mask for the same purpose. The output length is min(k, allowedCount) - a smaller allowlist yields exactly that many results rather than padded fallbacks.

How it works

Each vector is a direction on a high-dimensional hypersphere. TurboQuant compresses these directions with one insight: after a random rotation, every coordinate follows a known distribution, regardless of the input data.

1. Normalize. Strip each vector's norm and keep it as a single scalar; every vector becomes a unit direction.
2. Random rotation. Multiply by a fixed random orthogonal matrix (built once as the Q factor of a QR decomposition of a seeded Gaussian matrix). After rotation each coordinate follows a Beta distribution that converges to Gaussian N(0, 1/d) in high dimensions - for any input.
3. Per-coordinate calibration (TQ+). At finite dimensions the coordinates drift from the asymptotic shape. TQ+ fits a shift and a scale per coordinate on the first add, mapping the empirical quantiles onto the canonical Beta marginal, then freezes them - no retraining on later adds.
4. Lloyd-Max scalar quantization. Because the target distribution is known, the optimal bucket boundaries and centroids are computed once from the math (4 buckets at 2-bit, 16 at 4-bit), not learned from the data.
5. Bit-pack. Each coordinate is now a small integer, packed tightly into bytes.
6. Length-renormalized scoring (RaBitQ-style). Scalar quantization biases inner products downward; one scalar per vector (‖v‖ / ⟨u, x̂⟩, computed at encode time) corrects it at zero query-time cost.

Search rotates the query once into the codebook domain and scores directly against the compressed codes via the nibble-LUT SIMD kernel - database vectors are never decompressed.

Benchmarks

Measured on an Intel Core i7-12700F (AVX2), .NET 10, via the BenchmarkDotNet suite in benchmarks/.

Note: These numbers are from one specific machine and are indicative only. Absolute timings and speedups will differ with your CPU (core count, available SIMD width such as AVX-512 vs AVX2 vs NEON), memory, .NET runtime version, and build configuration. Run the suite on your own hardware for representative results.

Scalar vs SIMD search

256-dim, 10,000 vectors, 64 queries, k=10:

Bit width	Scalar	SIMD (AVX2)	Speedup
2-bit	13.44 ms	1.52 ms	8.8×
3-bit	15.31 ms	1.96 ms	7.8×
4-bit	16.13 ms	2.35 ms	6.7×

Search scaling

768-dim, 4-bit, 100,000 vectors, k=10 - single-query latency vs batched throughput:

Query batch	Mean	Per query
1	3.92 ms	3.92 ms
16	8.35 ms	0.52 ms
64	29.23 ms	0.46 ms
256	115.03 ms	0.45 ms

Batching amortizes the per-query work across cores, dropping per-query latency ~8.5×.

Persistence

10,000 vectors, 256-dim, 2-bit: Write 16.7 ms, Load 4.0 ms.

Running the suite

BenchmarkDotNet requires a Release build:

# Full matrix (long-running)
dotnet run -c Release --project benchmarks/TurboQuantSharp.Benchmarks -- --filter *# A single class
dotnet run -c Release --project benchmarks/TurboQuantSharp.Benchmarks -- --filter *SearchBenchmarks*

Benchmark classes: SearchBenchmarks (scalar vs SIMD), SearchScalingBenchmarks (batch/k scaling), EncodeBenchmarks (encode + build), SerializationBenchmarks (Write/Load), CodebookBenchmarks and RotationBenchmarks (one-time init).

Building

dotnet build -c Release # build the solution
dotnet test# run the test suite

Project layout:

src/TurboQuantSharp - the library
tests/TurboQuantSharp.Tests - xUnit tests
samples/TurboQuantDemo - end-to-end demo
benchmarks/TurboQuantSharp.Benchmarks - BenchmarkDotNet suite

The SIMD kernels are written once against System.Runtime.Intrinsics and dispatched at runtime, so the same binary runs on any CPU - the wide kernels light up only where the hardware supports them.

Contributing

Contributions and feedback are welcome. Whether it's a bug report, a performance idea, a recall improvement, better docs, or a question - please open an issue or a pull request. If you're sending code, a quick dotnet build and dotnet test before submitting keeps things green.

References

• TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate - Amir Zandieh, Majid Daliri, Majid Hadian, Vahab Mirrokni. arXiv:2504.19874 (ICLR 2026). The paper this implements.
• RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search - arXiv:2405.12497 (SIGMOD 2024). Source of the length-renormalization correction in step 6.
• turbovec - the Rust/Python implementation of TurboQuant that inspired this port.
• FAISS FastScan - the nibble-LUT / byte-shuffle scoring strategy the SIMD search adapts.

Citation

If you use the TurboQuant algorithm, please cite the original paper:

@inproceedings{zandieh2026turboquant,
title = {TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate},
author = {Zandieh, Amir and Daliri, Majid and Hadian, Majid and Mirrokni, Vahab},
booktitle = {International Conference on Learning Representations (ICLR)},
year = {2026},
eprint = {2504.19874},
archivePrefix = {arXiv},
primaryClass = {cs.LG}
}