structflo.cser

Chemical structure and label extraction from scientific documents.

Installation • Quick Start • Step-by-Step • Matchers • Downstream Processing • Notebooks

---

structflo.cser extracts chemical structure–label pairs from images and PDF pages. It uses a fine-tuned YOLO detector trained on synthetic chemical structure data to locate structures and compound labels on a page, then pairs them using Learned Pair Scorer (LPS) model or a simpler Hungarian Matcher.

The extracted crops can be passed to any structure-to-SMILES converter (DECIMER, MolScribe) and any OCR engine for label text. DECIMER and EasyOCR are bundled for convenience, but any downstream tools can be swapped in.

Two-step process:

1. Detect — A fine-tuned YOLO detector finds all chemical structures and compound labels in the image
2. Match — A matcher pairs each structure with its corresponding label, producing cropped image pairs

	LearnedMatcher (default)	HungarianMatcher
Approach	Neural Pair Scorer (LPS)	Geometric (centroid distance)
Setup	Auto-downloads weights	Zero config
Speed	Fast (GPU accelerated)	Instantaneous
Accuracy	Better for complex or crowded pages	Good for simple layouts
Output	CompoundPair	CompoundPair (identical)

Installation

pip install structflo-cser

# or with uv
uv add structflo-cser

This also installs DECIMER and EasyOCR for downstream SMILES and text extraction. The core pipeline does not depend on them — any extractor implementation can be swapped in.

Quick Start

One call from image to (SMILES, label) pairs:

fromstructflo.cser.pipelineimportChemPipelinefromstructflo.cser.lpsimportLearnedMatcherpipeline=ChemPipeline(matcher=LearnedMatcher())
results=pipeline.process("page.png")
forpairinresults:
print(pair.smiles, pair.label_text)

Weights for both the detector and the LPS are auto-downloaded from HuggingFace Hub on first use.

Export to a pandas DataFrame or JSON:

df=ChemPipeline.to_dataframe(results)
data=ChemPipeline.to_json(results)

 match_distance match_confidence smiles label_text
0 135.19 0.9844 CN1CCC2=C(C1)SC(=N2)C(=O)NC3=... 7178-39-6
1 208.40 0.9973 C1=CC(=CC=C1C2=C(C(=O)O)N=NN2... 72804-12-9
2 126.25 0.9997 COC1=CC=C(C=C1)C=C2C(=O)N(C3=... ZINC2978 720

PDF input

For PDFs, use process_pdf() — it renders each page and returns one result list per page:

fromstructflo.cser.pipelineimportChemPipelinefromstructflo.cser.lpsimportLearnedMatcherpipeline=ChemPipeline(matcher=LearnedMatcher())
# Returns list[list[CompoundPair]] — one inner list per pageall_pages=pipeline.process_pdf("paper.pdf")
forpage_num, pairsinenumerate(all_pages):
print(f"Page {page_num+1}: {len(pairs)} compound pairs")
forpairinpairs:
print(f" {pair.label_text:20s}{pair.smiles}")

Pass output_pdf to save an annotated copy with bounding boxes and extracted data overlaid:

pipeline.process_pdf("paper.pdf", output_pdf="paper_annotated.pdf")

Step-by-Step Pipeline

For finer control, each stage is exposed individually.

1. Create the pipeline

fromstructflo.cser.pipelineimportChemPipeline# Default: LearnedMatcher — auto-downloads LPS weights on first usepipeline=ChemPipeline(tile=False, conf=0.70)

For a heuristic based approach, use HungarianMatcher:

fromstructflo.cser.pipelineimportChemPipeline, HungarianMatcherpipeline=ChemPipeline(
tile=False,
conf=0.70,
matcher=HungarianMatcher(max_distance=500),
)

The pipeline is lazy — detector weights, DECIMER, and EasyOCR are loaded on first use only.

2. Detect

detections=pipeline.detect("page.png")
n_struct=sum(1fordindetectionsifd.class_id==0)
n_label=sum(1fordindetectionsifd.class_id==1)
print(f"Found {n_struct} structures and {n_label} labels")
# Found 6 structures and 6 labels

class_id=0 = chemical structure | class_id=1 = compound label

3. Match

pairs=pipeline.match(detections)
# Matched 6 structure–label pairs# Pair 0: distance=135px structure@(490,421) label@(489,285)# Pair 1: distance=208px structure@(258,194) label@(466,195)

4. Visualise

fromstructflo.cser.vizimportplot_detections, plot_pairs, plot_crops, plot_resultsfig=plot_detections(img, detections) # green = structure, blue = labelfig=plot_pairs(img, pairs) # orange lines connect matched pairsfig=plot_crops(img, pairs) # cropped structure and label regionsfig=plot_results(img, results) # final annotated output

5. Enrich — SMILES and label text

enriched=pipeline.enrich(pairs, "page.png")
fori, pinenumerate(enriched):
print(f"Pair {i}:")
print(f" SMILES: {p.smiles}")
print(f" Label text: {p.label_text}")

Pair 0:
SMILES: CN1CCC2=C(C1)SC(=N2)C(=O)NC3=C(C=CC=C3)CNC(=O)C4=CC=CC(=C4)Cl
Label text: 7178-39-6
Pair 1:
SMILES: C1=CC(=CC=C1C2=C(C(=O)O)N=NN2C3=CC=C(C=C3)S(=O)(=O)N)Br
Label text: 72804-12-9

Matchers

Learned Pair Scorer — LearnedMatcher (default)

A neural matcher trained to score structure–label compatibility using both visual crops and geometric features. It replaces the raw distance cost matrix with a learned association probability, then solves global assignment with the Hungarian algorithm.

Weights are auto-downloaded from HuggingFace Hub on first use — no manual setup needed. Models are hosted at:

• Detector: huggingface.co/sidxz/structflo-cser-detector
• LPS scorer: huggingface.co/sidxz/structflo-cser-lps

fromstructflo.cser.pipelineimportChemPipelinefromstructflo.cser.lpsimportLearnedMatcherpipeline=ChemPipeline(
matcher=LearnedMatcher(
min_score=0.5, # drop pairs below this confidencemax_dist_px=None, # optional centroid pre-filter to save compute
)
)

min_score — pairs scoring below this threshold are discarded as unlabelled structures.

Hungarian Matcher — HungarianMatcher (fallback)

Pairs structures and labels by minimising total centroid-to-centroid distance. Zero config, zero weights download. Useful for simple document layouts or as a fast sanity check.

fromstructflo.cser.pipelineimportChemPipeline, HungarianMatcherpipeline=ChemPipeline(
matcher=HungarianMatcher(max_distance=500),
)

max_distance — maximum pixel distance for a valid pair. Increase for large pages; reduce to avoid false pairings on dense layouts.

Downstream Processing

structflo.cser outputs cropped image pairs. Plug in any converter for SMILES and any OCR for label text.

SMILES extraction

DECIMER is bundled by default. Swap for MolScribe or any custom BaseSmilesExtractor:

fromstructflo.cser.pipeline.smiles_extractorimportBaseSmilesExtractorclassMyExtractor(BaseSmilesExtractor):
defextract(self, image) ->str:
returnmy_model.predict(image)
pipeline=ChemPipeline(smiles_extractor=MyExtractor())

OCR

EasyOCR is bundled by default. Swap for any custom BaseOCR:

fromstructflo.cser.pipeline.ocrimportBaseOCRclassMyOCR(BaseOCR):
defextract(self, image) ->str:
returnmy_ocr.read(image)
pipeline=ChemPipeline(ocr=MyOCR())

CLI

Run extraction directly from the terminal:

# Detect and pair structures/labels in a directory of images
sf-detect --image_dir data/test_images/ --conf 0.60 --no_tile --pair --max_dist 500
# Full pipeline: detect → match → SMILES + OCR
sf-extract page.png

All available commands:

Command	Description
sf-detect	Run YOLO detection on images
sf-extract	Full pipeline: detect → match → extract
sf-generate	Generate synthetic training data
sf-train	Train the YOLO detection model
sf-train-lps	Train the Learned Pair Scorer
sf-eval-lps	Evaluate LPS on a test set
sf-fetch-smiles	Download SMILES from ChEMBL
sf-download-distractors	Download distractor images for generation
sf-annotate	Launch the web annotation server

Notebooks

Notebook	Description
01-quickstart.ipynb	Step-by-step pipeline walkthrough: detect → match → enrich, then one-call convenience API
02-LPS.ipynb	Using the Learned Pair Scorer for improved matching on complex document pages

License

Apache License 2.0