Skip to content

Get rid of utils/kmeans.rs #917

Description

@arrayka
  • utils/kmeans.rs: KMeans has been replaced for PQ related purposes by the faster implementation in diskann-quantization. The last user of this flow is the disk-index partitioning. It's possible that the BLAS based implementation of the diskann-providers is faster than the approach taken in diskann-quantization for full-dimensional data. To that end, we should either:
    • Audit and replace the use of KMeans in disk partitioning to use diskann-quantization and get rid of the diskann-providers implementation.
    • Upgrade the implementation in diskann-quantization to be on-par for full-dimensional data.
    • Move the implementation to diskann-disk where it's actually used.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions