imgRetrieval

A local, cross-platform (macOS / Windows / Linux) image retrieval system. Index the images already on your machine — without copying them — and search by example image, by text, or by text + image together.

Demo

Try it in one command on a small set of public images (no personal data):

pip install -r requirements.txt -r requirements-ml.txt
python demo/run.py # fetch public images → index → http://127.0.0.1:8000

VLAD + RANSAC retrieves transformed copies of the same scene, ranked by geometric-consistency (inlier count) — coincidental matches sink to the bottom:

Text → image with CLIP — describe what you want:

More examples (CLIP image-to-image, fusion, captions) in demo/README.md.

Search methods

Query	Method	Engine	Notes
image → image	CLIP	CLIP embed	Best general visual similarity (semantic)
text → image	Text	CLIP embed	Type a description; matches images in shared CLIP space
text + image → image	Fusion	CLIP embed	Blends a text and image query (weight slider)
image → image	VLAD	SIFT	Local features → residual aggregation (+PCA whitening) → cosine
image → image	BoW	SIFT	Local features → visual-word histogram → TF-IDF cosine

Two engines: SIFT (BoW/VLAD — great for near-duplicate / same-object) and CLIP (image/text/fusion — semantic & text search). BoW/VLAD results are geometrically re-ranked with RANSAC on SIFT keypoints, so coincidental visual-word overlap is filtered out and true scene/object matches rise to the top.

Each result can be opened or revealed in your file manager (Finder / Explorer / Files), and shows an auto-generated BLIP caption.

• No duplication. Only file paths + a content hash are stored. The same image found in two folders is indexed once (the second path is kept as an alias).
• Incremental. The codebook (and CLIP/PCA models) are trained once and reused. New images are encoded and appended — existing entries are never re-processed. Re-run index any time you add images; retrain rebuilds the vocabulary + PCA.
• Caching. Per-image SIFT features (descriptors + keypoints, float16) are cached by content hash, so re-indexing / re-ranking never re-read pixels.
• Parallel. Feature extraction runs across threads (OpenCV releases the GIL).

Everything lives under data_dir (default ~/.imgretrieval/data: SQLite metadata, codebooks, CLIP/PCA models, descriptor cache, thumbnails) — delete it to start over. Your original images are never modified.

Approximate footprint (≈4,500 images)

• Search index in RAM: ~32 MB (VLAD 4.6 MB after PCA + BoW 18 MB + CLIP 9 MB)
• Descriptor cache on disk: ~450 MB (float16; lazy — only for retrain/re-rank)
• Models in RAM when used: CLIP ~0.6 GB (text/clip/fusion queries), BLIP ~1 GB (captioning only)

Setup

cp config.example.yaml config.yaml # then edit `folders`
python3 -m venv .venv &&source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt

BoW/VLAD (image→image) work with just the core requirements. Text, CLIP and fusion search (and BLIP captions) need PyTorch + transformers — a large download that runs on GPU (CUDA / Apple MPS) when available:

pip install -r requirements-ml.txt # torch + transformers (CLIP + BLIP)

CLIP is enabled by default (clip.enabled: true); set it false to skip the ML deps and use BoW/VLAD only. Captions are off by default (captioning.enabled).

Usage

# 1) index your configured folders (first run trains the vocabulary + CLIP)
python -m imgret.cli index
# 2) launch the browser UI
python -m imgret.cli serve # → http://127.0.0.1:8000# or query from the CLI
python -m imgret.cli query --image some.jpg --method clip --top-k 10
python -m imgret.cli query --image some.jpg --method vlad --top-k 10
python -m imgret.cli query --text "a dog on the beach" --top-k 10
python -m imgret.cli query --text "sunset" --image q.jpg --method fusion --alpha 0.5
python -m imgret.cli status

In the web UI: pick a method, drop an image and/or type text (fusion uses both, with a text↔image weight slider), and click a result to Open or Reveal in file manager. Scan & index new images adds new files incrementally; Retrain rebuilds the vocabulary + PCA.

Tuning (config.yaml)

• feature.type: sift (best) or orb (faster). feature.workers: parallel extraction (0 = auto).
• codebook.vlad_pca_dim: PCA-whiten VLAD to this many dims (big memory cut + accuracy gain; 0 = off).
• codebook.bow_clusters / vlad_clusters: vocabulary sizes.
• search.rerank + rerank_candidates + rerank_min_inliers: geometric (RANSAC) re-ranking of BoW/VLAD.
• clip.enabled / clip.model, captioning.enabled / captioning.model.

Architecture

• features.py SIFT/ORB extraction · extract.py parallel + cache · codebook.py k-means
• encoders.py BoW/VLAD · vlad_pca.py PCA whitening · geomverify.py RANSAC re-rank
• clip_embed.py CLIP · captioner.py BLIP · index.py orchestration · search.py query routing
• db.py SQLite · app.py FastAPI + static/index.html UI · cli.py CLI · reveal.py file-manager