Skip to content

Support a custom vacuum or optimize command on vec0 tables #185

Description

@asg017

If you DELETE a lot of vectors in a vec0 index, the vector chunks will contain a lot unused space. Sometimes it can be re-claimed when INSERT'ing new vectors (see #184), but there's no way to re-claim the disk space entirely.

So we should add a new "insert command" like FTS5 that allows you do manually fixup chunks and reclaim space:

INSERT INTO vec_items(vec_items) VALUES ('optimize');

This is easier said than done. This requires:

  • Finding unused entries in older chunks
  • Moving newer chunks into available old positions
  • Deleting "empty" older chunks
  • Making sure parition keys/metadata values stay the same

oh boy

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions