[CellVQ] Illuminating cell states by a comprehensive and interpretable single cell foundation model

CellVQ is a comprehensive and interpretable single-cell foundation model. CellVQ innovatively integrates a Single-Cell Discretization (SCD) module into the single-cell foundation model. This module utilizes a unified cell codebook to transform the cell representation into a cell code and uses an SCD cell embedding derived from the cell codebook to represent the cell corresponding to the code. CellVQ employs an encoder-SCD-decoder architecture, encompassing 511 million parameters and pretrained on over 68 million single-cell data points.

Table of Contents

• Background
• Features
• Installation
• Quick Start
• Usage
  • Command-Line Arguments
  • Running the Example
  • Using Custom Data
• References
• Contact
• License

Background

Advances in high-throughput single-cell sequencing techniques have enabled AI-driven methods to harness extensive datasets, resulting in the emergence of robust single-cell foundation models that demonstrate exceptional representation capabilities across various downstream applications. However, current evidence suggests that the practical utility and generalizability of these foundation models are significantly constrained by the sparsity and heterogeneity of real-world data, as well as their limited interpretability. To truly enhance single-cell research and facilitate drug discovery, a foundation model must exhibit improved generalizability across diverse real-world data sources and provide clear interpretations for novel biological insights.

Features

• Advanced cell representation and Generalizability: CellVQ can be applied to a wide range of single-cell tasks, categorized into prediction and analysis tasks. In the realm of prediction tasks, which encompass cell clustering, annotation, property prediction, gene and drug perturbation prediction, as well as drug response prediction, CellVQ demonstrates significant advancements over other foundation model.
• Effective Quantization: CellVQ effectively represents high-dimensional and multi-modal single-cell data using a single token, minimizing information loss.
• Interprebility: The VQ and CellVQ-graph module provides quantized stastics that help interpret the significance of each gene or property in relation to the cell state. It greatly empowers the analysis tasks. By integrating CellVQ and CellVQ-Graph, the models can be used in four distinct tasks: cell state discovery, gene discovery, gene regulatory network (GRN) analysis, and multimodal analysis.

For more details on the performance and benchmarking, please refer to our paper.

Installation

To get started with CellVQ, follow these steps:

1. Clone the repository:

   git clone https://github.com/A4Bio/CellVQ.git
   cd CellVQ

2. Create and activate the conda environment:

   conda create -n CellVQ python=3.9.17
   conda activate CellVQ
   ./install.sh

3. Download the pretrained Model:

   We provide a pretrained model for CellVQ. Download it here and place it in the pretrained_models directory. We also provide the test data in Download it here.

4. Download the pretraining dataset: To download the pretraining dataset, users can refer to the link.

Quick Start

To quickly try out CellVQ using an example dataset, run the following command:

bash run_example.sh

This script runs the inference.py script with sample data provided in the examples folder.

We also provide an example tutorial in quick_start.ipynb. The instructions for reproducing our results on each downstream task can be found in the corresponding directory.

Usage

Command-line Arguments

The inference.py script supports several command-line arguments:

Argument	Description	Default
--data_path	Path to the dataset.	None
--model_path	Path to the pretrained model checkpoint.	pretrained_model/checkpoint.pt
--save_path	Path to save the output of CellVQ.	example
--device	Device to run the model on (cpu or cuda).	cuda
--verbose	Enable verbose output for debugging.	Disabled
--mode	Model mode.	m1

Running the Example

You can run the example directly from the command line:

python inference.py

Using Custom Data

To use CellVQ with your own data, you need to provide a scRNA-seq or scATAC-seq dataset. For example:

python inference.py --data_path /path/to/your/dataset --save_path /path/to/your/save/path --mode m1 --device cuda

References

For a complete description of the method, see:

TBD

Contact

Please submit any bug reports, feature requests, or general usage feedback as a github issue or discussion.

• Jue Wang (wangjue@westlake.edu.cn)
• Cheng Tan (tancheng@westlake.edu.cn)
• Zhangyang Gao (gaozhangyang@westlake.edu.cn)

License

This project is licensed under the MIT License.