Scriptify

End-to-end handwriting synthesis: train → package → serve → web.

Built on PyTorch with an LSTM + attention model trained on the IAM On‑Line Handwriting Database. Based on Alex Graves' "Generating Sequences With Recurrent Neural Networks" (https://arxiv.org/abs/1308.0850).

Features

• Train LSTM + attention with GMM output on IAM
• Multi‑GPU training (DDP) and resumable checkpoints
• Weights & Biases logging (optional)
• FastAPI inference service + React frontend
• HuggingFace Space deployment template

Project Structure

scriptify/
├── ml/ # Training pipeline (PyTorch)
│ ├── src/ # models/, data/, training/, utils/
│ ├── config/ # config.yaml (paths, hyperparams)
│ ├── scripts/ # run.sh, run_ddp.sh
│ ├── outputs/ # runs/, checkpoints/, logs
│ └── packaged_models/ # packaged artifacts for serving
├── api/ # FastAPI inference service
│ └── src/
│ ├── main.py # API entry
│ └── inference_utils.py
└── frontend/ # React + TypeScript app
└── src/ # components/, services/, App.tsx

Quickstart

1) Setup

• Python 3.9+, Conda, Node 18+, CUDA GPU recommended
• Obtain IAM On‑Line Handwriting (ascii/ and lineStrokes/)

2) Prepare data

cd ml
# Edit config/config.yaml → set:# paths.raw_data_root (contains ascii/ and lineStrokes/)# paths.processed_data_dir (write location for .npy files)
conda env create -f environment.yml && conda activate scriptify_ml_env
python -m src.data.dataset # writes processed arrays to paths.processed_data_dir

3) Train

cd ml
wandb login # optional
bash scripts/run.sh # single GPU (torchrun)# Multi-node/GPU (requires ml/.env with DISTRIBUTED settings)# Save this as ml/.env (example values):# SCRIPTIFY_DIST_MASTER_ADDR=10.0.0.1# SCRIPTIFY_DIST_MASTER_PORT=29500# SCRIPTIFY_DIST_NNODES=2# SCRIPTIFY_DIST_NPROC_PER_NODE=2
bash scripts/run_ddp.sh 0 # node rank 0

4) Package model for serving

The API loads from ml/packaged_models and expects these filenames:

• handwriting_model.pt
• handwriting_model.scripted.pt

Generate matching files with:

cd ml
python -m src.package_model --pkg_name handwriting_model

Or specify paths explicitly:

python -m src.package_model \
--checkpoint <path/to/checkpoint> \
--config <path/to/config.yaml> \
--pkg_name handwriting_model

If you change names or locations, update api/src/main.py constants (MODEL_DIR, SCRIPTED_MODEL_NAME, METADATA_MODEL_NAME) or rename the files accordingly.

5) Run API

cd api
pip install -r requirements.txt
cd src
python main.py # http://localhost:8000 (see /docs)# Alternatively: uvicorn main:app --reload

6) Run frontend

cd frontend
npm install
npm run dev # http://localhost:5173

Model (brief)

• 3‑layer LSTM with soft attention over characters
• GMM output for (dx, dy, pen‑up) stroke prediction
• Tunable via ml/config/config.yaml

Configuration

Edit ml/config/config.yaml (or set SCRIPTIFY_CONFIG_PATH). Key fields:

• paths.raw_data_root — IAM dataset root (contains ascii/ and lineStrokes/)
• paths.processed_data_dir — where processed .npy are saved
• paths.outputs_dir — training runs and checkpoints
• dataset.max_stroke_len, dataset.max_text_len, dataset.alphabet_string
• wandb.enabled, wandb.project_name

Demo

• Frontend: https://scriptify.h3nok.dev/
• Backend: HuggingFace Spaces — https://huggingface.co/spaces/h3nock/scriptify-api