Image Preprocessing Pipeline for OCR

This project is an OCR-focused image preprocessing pipeline built to improve text readability in difficult images, especially where text is blurred, low-contrast, or surrounded by noisy/dark backgrounds.

The core idea is practical: improve OCR accuracy by combining foreground localization (rembg mask-based cropping) with document-aware enhancement steps (contrast, illumination correction, denoising, and controlled sharpening).

Problem Statement

The input set includes invoices and document-like photos with issues such as:

• Text-level blur
• Uneven lighting and shadows
• Dark/irrelevant background regions
• Low contrast between text and paper
• Mixed quality across images

This repository contains multiple iterations of solutions, from simple enhancement to a more adaptive, production-style OCR preprocessing pipeline.

What Was Tried (Evolution)

1) Baseline OCR Enhancement (enhance.py)

Initial enhancement path:

• Convert to grayscale
• Apply CLAHE (clipLimit=2.0, tileGridSize=(8,8))
• Apply very conservative unsharp masking

Also includes commented experimental variants:

• LAB-space CLAHE on luminance only
• Luminance-only sharpening with reduced high-frequency gain

Goal: improve clarity without causing stroke fattening.

2) Background Localization with rembg (rembg_process.py)

Instead of only alpha compositing, you used rembg mask prediction to localize the document and crop the original image:

• Reused ONNX session on CPU (new_session(providers=["CPUExecutionProvider"]))
• Optional downscale before mask inference (MAX_SIDE=1024) for speed
• Thresholded mask (threshold=10) and computed bounding box
• Added configurable padding (pad=8)
• Mapped bbox back to original resolution

This avoids aggressive blending artifacts and keeps original text pixels while removing irrelevant background regions.

3) Text Darkening + Sharpening Experiments (enhance_darken.py)

Additional OCR-focused intensity pipeline:

• Gamma correction to darken (gamma < 1)
• Contrast boost via PIL ImageEnhance.Contrast
• Black point push (gray < threshold to pure black)
• Optional luminance sharpening

Combined in enhance_text_darkness(...) with tunable parameters:

• gamma
• contrast_factor
• black_threshold
• sharpen_amount

Purpose: make weak text strokes more OCR-visible, especially on faded scans/photos.

4) Combined Processing Orchestration (app.py)

A directory-level orchestrator with two tracks:

• Full background removal output (remove_background)
• Mask-based crop + darkening/sharpening output (darken_image)

Config toggles allow running either or both:

• DO_BACKGROUND_REMOVAL
• DO_DARKENING
• USE_CROP

This script reflects your intermediate production attempt before the fully adaptive pipeline.

5) Adaptive, Document-Aware OCR Pipeline (modern_ocr.py)

This is the most complete and robust approach in the repo.

It computes quality metrics first:

• Blur (Laplacian variance)
• Noise (MAD-based estimate)
• Illumination variance (block mean std-dev)
• Contrast (global std-dev)

Then runs staged preprocessing:

1. Input normalization and OCR-friendly scaling toward target DPI
2. Skew detection/correction using Hough lines
3. Illumination correction with blur-based background normalization (cv2.divide)
4. Adaptive CLAHE based on measured contrast
5. Adaptive unsharp masking based on blur score
6. Bilateral denoising based on estimated noise
7. Optional adaptive binarization

Design intent:

• Prefer signal-preserving operations
• Reduce over-processing that can make OCR worse
• Adapt strength by measured image quality instead of fixed parameters

6) Batch OCR Runner (ocr.py)

End-to-end runner for folders:

• Optional rembg mask-based cropping
• Calls preprocess_for_ocr(...) from modern_ocr.py
• Saves outputs as ocr_<name>.png
• Reports per-image quality metrics and summary stats

This is the current recommended pipeline entry point for OCR preprocessing.

7) Earlier Batch Script (run_pipeline.py)

An older batch script tying:

• remove_background_pil(...)
• enhance_for_ocr(...)

Useful as a historical baseline for comparing simpler enhancement flow vs the newer adaptive pipeline.

Repository Structure

• modern_ocr.py: adaptive OCR preprocessing engine
• ocr.py: main batch runner (recommended)
• app.py: earlier combined background-removal + darkening workflow
• enhance.py: baseline contrast + unsharp enhancement
• enhance_darken.py: darkening/black-point/sharpen experiments
• rembg_process.py: rembg mask-based cropping helper
• run_pipeline.py: legacy batch runner
• input_images/: source images
• output_ocr_enhanced/: modern pipeline outputs
• output_darkened/: darkening pipeline outputs

Installation

python -m venv .venv
# Windows PowerShell
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Usage

Recommended: Adaptive OCR Pipeline

python ocr.py

Default behavior in ocr.py:

• Input directory: input_images
• Output directory: output_ocr_enhanced
• Uses rembg mask-based crop (use_crop=True)
• Target DPI: 300
• Adaptive mode: enabled
• Binarization: disabled by default

Run the Core Engine on One Image

python modern_ocr.py path/to/image.jpg

Run the Darkening Pipeline

python app.py

Tuning Notes

If text is still faint:

• In enhance_darken.py, lower gamma slightly (for stronger darkening)
• Increase contrast_factor carefully
• Raise black_threshold in small steps

If text becomes thick/blobby:

• Reduce sharpen_amount
• Reduce CLAHE strength (clip_limit)
• Disable binarization or tune threshold settings

If background interference remains:

• Keep use_crop=True
• Increase crop padding only if characters near borders are cut

Current Best Approach

For mixed-quality document photos, the best current path in this repo is:

• rembg mask-based crop (localize document)
• Adaptive preprocessing from modern_ocr.py
• Optional binarization only when your OCR engine benefits from binary input

Dependencies

From requirements.txt:

• opencv-python
• numpy
• pillow
• matplotlib
• rembg

Known Limitations

• Very severe motion blur cannot be fully recovered by preprocessing alone.
• Mask quality from rembg may vary on unusual backgrounds.
• Perspective correction is currently skew-focused; full quadrilateral document rectification is not implemented.
• Parameter defaults are tuned for document-like images and may need adjustment for other domains.

Potential repo updates !

Potential upgrades if you continue this pipeline:

• Add OCR benchmark loop (CER/WER) for parameter tuning
• Add automatic document boundary detection + perspective warp
• Add per-image config auto-selection and experiment logging
• Add unit tests for each stage and regression image set