Added store_many_vectors on Mongo Storage - #87
Conversation
amorgun
left a comment
There was a problem hiding this comment.
Could you add some tests for store_many_vectors to the mongo test suite?
| self.mongo_object.insert_one(val_dict) | ||
| def _get_vector(self, hash_name, bucket_key, v, data): | ||
| """ |
There was a problem hiding this comment.
This docstring belongs to store_vector method
| def store_many_vectors(self, hash_name, bucket_keys, vs, data): | ||
| requests = [] | ||
| for v, d, bk in zip(vs, data, bucket_keys): |
There was a problem hiding this comment.
I suggest using from future.builtins import zip because it is more efficient in python2.7.
| def store_many_vectors(self, hash_name, bucket_keys, vs, data): | ||
| requests = [] | ||
| for v, d, bk in zip(vs, data, bucket_keys): |
| else: | ||
| vector = numpy.fromstring(val_dict['vector'], | ||
| vector = numpy.frombuffer(val_dict['vector'], |
There was a problem hiding this comment.
I updated this because I got some deprecation warnings
| Removes all buckets from all hashes and their content. | ||
| """ | ||
| self.mongo_object.remove( | ||
| self.mongo_object.delete_many( |
There was a problem hiding this comment.
remove method is deprecated so I replaced it with the suggested delete_many. It avoids annoying deprecation warnings
amorgun
commented
Nov 21, 2019
@etudor The test is broken because it cannot import |
etudor
commented
Nov 29, 2019
@amorgun I have updated this |
amorgun
commented
Nov 29, 2019
@etudor It looks like a lot of mongo tests are broken now in python2.7. Please, check if it is related to your changes. Maybe you should pin an older version of |
Because I found this method to be very useful on Redis storage, I've added it to the mongo storage as well.
I haven't done a benchmark to compare what is the speed increase vs single inserts.