Seeking advice: remove is painfully slow

[patknight]

I'm finding that adding and searching an objectbox database is really fast. However, the remove operation is really slow (1 second per object.) The database is on a local NVME SSD drive. It contains about 20,000 hashes and takes about 6GB.

My find_unique hash_box.query operation is fast - it's literally the call to hash_box.remove that takes the time.

What am I doing wrong?

@Entity()
class ImHash:
 id = Id
 key = String(index=Index(IndexType.HASH), unique=True)
 cos_value = Float32Vector(index=HnswIndex(
 dimensions=62720,
 distance_type=VectorDistanceType.COSINE,
 ))


def hash_image(im: Image.Image) -> list[float]:
 vector = img2vec.get_vec(im, tensor=True)
 return vector.detach().cpu().numpy().flatten()


def hash_and_store(name_or_fp, key: str):
 im = Image.open(name_or_fp)
 h = hash_image(im)
 ih = find_unique(key)
 if ih is None:
 # create
 ih = ImHash()
 ih.key = key
 ih.cos_value = h
 with store_lock:
 hash_box.put(ih)


def init(db_dir: pathlib.Path):
 global store, hash_box, img2vec
 store = Store(directory=str(db_dir / directory_name),
 model_json_file=str(db_dir / json_model_name),
 max_db_size_in_kb=10 * 1024 * 1024)
 hash_box = store.box(ImHash)
 img2vec = Img2Vec(cuda=False, model='efficientnet_b0')


def close():
 store.close()


def find_unique(key: str):
 with store_lock:
 query = hash_box.query(ImHash.key.equals(key)).build()
 result = query.find()
 if len(result) == 0:
 return None
 elif len(result) > 1:
 print('Multiple matches found')
 return None
 else:
 return result[0]


def find_similar(key: str) -> list[tuple[ImHash, float]]:
 target = find_unique(key)
 with store_lock:
 query = hash_box.query(ImHash.cos_value.nearest_neighbor(target.cos_value, 8)).build()
 results = query.find_with_scores()
 results.sort(key=lambda x: x[1])
 return results


def remove(key: str):
 target = find_unique(key)
 if target is not None:
 with store_lock:
 hash_box.remove(target)


def remove_many(keys: list[str]):
 with store.write_tx():
 for k in keys:
 i = find_unique(k)
 if i is None:
 print('Hash key "%s" was already gone' % k)
 else:
 with store_lock:
 hash_box.remove(i.id)

[ElizioMartins]

Hi! I noticed something in your entity definition that may be related to your issue:

@Entity()
class ImHash:
 id = Id # ❌ missing parentheses
 key = String(...)

Id and String must be instantiated with () to work correctly as field descriptors. Without the parentheses, id holds a reference to the class itself instead of a descriptor instance, which can cause unexpected behavior at runtime (e.g. TypeError when instantiating the entity, or incorrect serialization).

The correct syntax is:

@Entity()
class ImHash:
 id = Id() # ✅ correct
 key = String(index=Index(IndexType.HASH), unique=True)

This bug was reported back in 2021 (issue #10) and was addressed in PR #32, which corrects the README example. Fixing this in your code might resolve or at least change the behavior you're seeing.

That said, even with correct syntax, removing objects with a 62,720-dimension HNSW vector index will inherently be slow — each removal requires repairing the HNSW graph, which involves distance calculations across all those dimensions for multiple neighbor nodes. If removal performance is critical, consider:

• Using box.remove_all() for bulk deletes instead of one-by-one
• Reducing vector dimensions if possible
• Running removes inside a single explicit write transaction via store.write_tx()

Hope this helps! 🙌

[patknight]

Thanks for the response. I'd found a better model that produced much shorter vectors and that improved performance.

Good catch on the Id parentheses. I won't be able to test this as I recently switched to another vector DB and no longer have an object box DB. It would take too long to build one as I'd have to regenerate the vectors.

I'm closing this issue.