CineMA: Development and validation of a versatile foundation model for cine cardiac magnetic resonance image analysis 🎥🫀

Tip

Check out our interactive demos on Hugging Face to see CineMA in action!

Multiple training and inference examples/scripts have also been provided in cinema/examples.

📝 Overview

CineMA is a vision foundation model for Cine cardiac magnetic resonance (CMR) imaging, built on Masked-Autoencoder. Pre-trained on the extensive UK Biobank dataset, CineMA has been fine-tuned for various clinically relevant tasks:

• 🫀 Ventricle and myocardium segmentation
• 📊 Ejection fraction (EF) regression
• 🏥 Cardiovascular disease (CVD) detection and classification
• 📍 Mid-valve plane and apical landmark localization

The model has demonstrated improved or comparative performance against convolutional neural network baselines (UNet, ResNet) across multiple datasets, including ACDC, M&Ms, M&Ms2, Kaggle, Rescan, and Landmark.

🚀 Getting Started

Installation

Option 1: Quick Install with pip

pip install git+https://github.com/mathpluscode/CineMA

Note: This method does not install dependencies automatically.

Option 2: Full Installation with Dependencies

git clone https://github.com/mathpluscode/CineMA.git
cd CineMA
conda env update -f envs/environment.yml
conda activate cinema
pip install -e .

⚠️Important: Install PyTorch separately following the official instructions.

🎯 Using Fine-tuned Models

All fine-tuned models are available on HuggingFace. Try them out using our example inference scripts:

# Segmentation
python examples/inference/segmentation_sax.py
python examples/inference/segmentation_lax_4c.py
# Classification
python examples/inference/classification_cvd.py
python examples/inference/classification_sex.py
python examples/inference/classification_vendor.py
# Regression
python examples/inference/regression_ef.py
python examples/inference/regression_bmi.py
python examples/inference/regression_age.py
# Landmark Detection
python examples/inference/landmark_heatmap.py
python examples/inference/landmark_coordinate.py

Available tasks and models are listed below.

Task	Input View	Input Timeframes	Inference Script
Ventricle and myocardium segmentation	SAX	1	segmentation_sax.py
Ventricle and myocardium segmentation	LAX 4C	1	segmentation_lax_4c.py
CVD classification	SAX or LAX 4C	2 (ED and ES)	classification_cvd.py
Patient sex classification	SAX	2 (ED and ES)	classification_sex.py
CMR machine vendor classification	SAX or LAX 4C	2 (ED and ES)	classification_vendor.py
EF regression	SAX or LAX 4C	2 (ED and ES)	regression_ef.py
Patient BMI regression	SAX	2 (ED and ES)	regression_bmi.py
Patient age regression	SAX	2 (ED and ES)	regression_age.py
Landmark localization by heatmap regression	LAX 2C or LAX 4C	1	landmark_heatmap.py
Landmark localization by coordinates regression	LAX 2C or LAX 4C	1	landmark_coordinate.py

🔧 Using Pre-trained Models

The pre-trained CineMA backbone is available at HuggingFace. Fine-tune it using our example scripts and the preprocessed ACDC dataset for following tasks:

Task	Fine-tuning Script
Ventricle and myocardium segmentation	segmentation.py
Cardiovascular disease classification	classification.py
Ejection fraction regression	regression.py

The commandlines are:

# Fine-tuning Scripts
python examples/train/segmentation.py
python examples/train/classification.py
python examples/train/regression.py

You can also explore the reconstruction performance of extract features using following example scripts.

# MAE Examples
python examples/inference/mae.py
python examples/inference/mae_feature_extraction.py

📚 Dataset Support

For fine-tuning CineMA on other datasets, pre-process can be performed using the provided scripts following the documentations. It is recommended to download the data under ~/.cache/cinema_datasets as the integration tests uses this path. For instance, the mnms preprocessed data would be ~/.cache/cinema_datasets/mnms/processed. Otherwise define the path using environment variable CINEMA_DATA_DIR.

Dataset	Documentation
ACDC	README.md
M&Ms	README.md
M&Ms2	README.md
Kaggle	README.md
Rescan	README.md
Landmark	README.md
EMIDEC	README.md
Myops2020	README.md

The code for training and evaluating models on these datasets are available.

Task	Data	Documentation
Ventricle and myocardium segmentation	ACDC	cinema/segmentation/acdc/README.md
Ventricle and myocardium segmentation	M&Ms	cinema/segmentation/mnms/README.md
Ventricle and myocardium segmentation	M&Ms2	cinema/segmentation/mnms2/README.md
Ventricle and myocardium segmentation	Kaggle	cinema/segmentation/kaggle/README.md
Ventricle and myocardium segmentation	Rescan	cinema/segmentation/rescan/README.md
Scar segmentation	EMIDEC	cinema/segmentation/emidec/README.md
Scar segmentation	Myops2020	cinema/segmentation/myops2020/README.md
Image classification	ACDC	cinema/classification/acdc/README.md
Image classification	M&Ms	cinema/classification/mnms/README.md
Image classification	M&Ms2	cinema/classification/mnms2/README.md
Image regression	ACDC	cinema/regression/acdc/README.md
Image regression	M&Ms	cinema/regression/mnms/README.md
Image regression	M&Ms2	cinema/regression/mnms2/README.md
Landmark localization by heatmap regression	Landmark	cinema/segmentation/landmark/README.md
Landmark localization by coordinates regression	Landmark	cinema/regression/landmark/README.md

🏗️ Training Your Own Foundation Model

Start with our simplified pretraining script:

python examples/train/pretrain.py

For distributed training support, check cinema/mae/pretrain.py.

❤️ Acknowledgement

This research has been conducted using the UK Biobank Resource under Application Number 71702. The authors acknowledge the use of resources provided by the Isambard-AI National AI Research Resource (AIRR). Isambard-AI is operated by the University of Bristol and is funded by the UK Government’s Department for Science, Innovation and Technology (DSIT) via UK Research and Innovation; and the Science and Technology Facilities Council [ST/AIRR/I-A-I/1023].

📖 References

CineMA builds upon these open-source projects:

• UK Biobank Cardiac Preprocessing
• Masked Autoencoders
• ConvMAE
• MultiMAE
• MONAI
• PyTorch Vision
• PyTorch Image Models

🤝 Contributing

We welcome contributions! Please create an issue for questions or suggestions.

📧 Contact

For collaborations, reach out to Yunguan Fu (yunguan.fu.18@ucl.ac.uk).

📄 Citation

@article{fu2026development,
title={Development and validation of a versatile foundation model for cine cardiac magnetic resonance image analysis},
author={Fu, Yunguan and Bai, Wenjia and Yi, Weixi and Manisty, Charlotte and Bhuva, Anish N and Treibel, Thomas A and Moon, James C and Clarkson, Matthew J and Davies, Rhodri Huw and Hu, Yipeng},
journal={Communications Medicine},
year={2026},
publisher={Nature Publishing Group UK London}
}