VecSim - A unified interface for similarity servers

A standard, light-weight interface to all popular similarity servers.

The problems we are trying to solve:

1. Standard API - Different vector similarity servers have different APIs - so switching is not trivial.
2. Identifiers - Some vector similarity servers support string IDs, some do not - we keep track of the mapping.
3. Partitions - In most cases, pre-filtering is needed prior to querying, we abstract this concept away.
4. Aggregations - In some cases, one item is being indexed to multiple vectors.

Supported engines:

1. Scikit-learn, via NearestNeighbors
2. RediSearch
3. Faiss
4. ElasticSearch
5. Pinecone

QuickStart example

importnumpyasnp# Import a similarity server of your choice:# SKlearn (best for small datasets or testing)fromvecsimimportSciKitIndexsim=SciKitIndex(metric='cosine', dim=32)
user_ids= ["user_"+str(1+i) foriinrange(100)]
user_data=np.random.random((100,32))
item_ids=["item_"+str(101+i) foriinrange(100)]
item_data=np.random.random((100,32))
sim.add_items(user_data, user_ids, partition="users")
sim.add_items(item_data, item_ids, partition="items")
# Index the datasim.init()
# Run nearest neighbor vector searchquery=np.random.random(32)
dists, items=sim.search(query, k=10) # returns a list of users and itemsdists, items=sim.search(query, k=10, partition="users") # returns a list of users only

For more examples, please read our documentation