Skip to content
BranchesTags

Repository files navigation

VenusFactory Banner

GitHub starsGitHub forksGitHub issuesGitHub licensePython VersionDocumentationDownloads

Recent News:

  • Welcome to VenusFactory! Please give us a star if you like it!
  • [2025-03-17] Add PETA, ProPrime, ProSST models

✏️ Table of Contents

📑 Features

  • Vaious protein langugae models: ESM2, ESM-b, ESM-1v, ProtBert, ProtT5, Ankh, etc
  • Comprehensive supervised datasets: Localization, Fitness, Solubility, Stability, etc
  • Easy and quick data collector: AlphaFold2 Database, RCSB, InterPro, Uniprot, etc
  • Experiment moitors: Wandb, Local
  • Friendly interface: Gradio UI

🤖 Supported Models

Pre-training Protein Language Models

ESM Series Models: Meta AI's protein language models
ModelSizeParametersGPU MemoryTraining DataTemplate
ESM2-8M8M8M2GB+UR50/Dfacebook/esm2_t6_8M_UR50D
ESM2-35M35M35M4GB+UR50/Dfacebook/esm2_t12_35M_UR50D
ESM2-150M150M150M8GB+UR50/Dfacebook/esm2_t30_150M_UR50D
ESM2-650M650M650M16GB+UR50/Dfacebook/esm2_t33_650M_UR50D
ESM2-3B3B3B24GB+UR50/Dfacebook/esm2_t36_3B_UR50D
ESM2-15B15B15B40GB+UR50/Dfacebook/esm2_t48_15B_UR50D
ESM-1b650M650M16GB+UR50/Sfacebook/esm1b_t33_650M_UR50S
ESM-1v-1650M650M16GB+UR90/Sfacebook/esm1v_t33_650M_UR90S_1
ESM-1v-2650M650M16GB+UR90/Sfacebook/esm1v_t33_650M_UR90S_2
ESM-1v-3650M650M16GB+UR90/Sfacebook/esm1v_t33_650M_UR90S_3
ESM-1v-4650M650M16GB+UR90/Sfacebook/esm1v_t33_650M_UR90S_4
ESM-1v-5650M650M16GB+UR90/Sfacebook/esm1v_t33_650M_UR90S_5

💡 ESM2 models are the latest generation, offering better performance than ESM-1b/1v

BERT-based Models: Transformer encoder architecture
ModelSizeParametersGPU MemoryTraining DataTemplate
ProtBert-Uniref100420M420M12GB+UniRef100Rostlab/prot_bert
ProtBert-BFD420M420M12GB+BFD100Rostlab/prot_bert_bfd
IgBert420M420M12GB+AntibodyExscientia/IgBert
IgBert-unpaired420M420M12GB+AntibodyExscientia/IgBert_unpaired

💡 BFD-trained models generally show better performance on structure-related tasks

T5-based Models: Encoder-decoder architecture
ModelSizeParametersGPU MemoryTraining DataTemplate
ProtT5-XL-UniRef503B3B24GB+UniRef50Rostlab/prot_t5_xl_uniref50
ProtT5-XXL-UniRef5011B11B40GB+UniRef50Rostlab/prot_t5_xxl_uniref50
ProtT5-XL-BFD3B3B24GB+BFD100Rostlab/prot_t5_xl_bfd
ProtT5-XXL-BFD11B11B40GB+BFD100Rostlab/prot_t5_xxl_bfd
IgT53B3B24GB+AntibodyExscientia/IgT5
IgT5-unpaired3B3B24GB+AntibodyExscientia/IgT5_unpaired

💡 T5 models can be used for both encoding and generation tasks

Specialized Models: Task-specific architectures
ModelSizeParametersGPU MemoryFeaturesTemplate
Ankh-base450M450M12GB+Encoder-decoderElnaggarLab/ankh-base
Ankh-large1.2B1.2B20GB+Encoder-decoderElnaggarLab/ankh-large
ProSST-2020110M4GB+MutationAI4Protein/ProSST-20
ProSST-128128110M4GB+MutationAI4Protein/ProSST-128
ProSST-512512110M4GB+MutationAI4Protein/ProSST-512
ProSST-20482048110M4GB+MutationAI4Protein/ProSST-2048
ProSST-40964096110M4GB+MutationAI4Protein/ProSST-4096
ProPrime-690M690M690M16GB+OGT-predictionAI4Protein/Prime_690M

💡 These models often excel in specific tasks or offer unique architectural benefits

PETA Models: Tokenization variants

BPE Tokenization Series

ModelVocab SizeParametersGPU MemoryTemplate
PETA-basebase80M4GB+AI4Protein/deep_base
PETA-bpe-505080M4GB+AI4Protein/deep_bpe_50
PETA-bpe-20020080M4GB+AI4Protein/deep_bpe_200
PETA-bpe-40040080M4GB+AI4Protein/deep_bpe_400
PETA-bpe-80080080M4GB+AI4Protein/deep_bpe_800
PETA-bpe-1600160080M4GB+AI4Protein/deep_bpe_1600
PETA-bpe-3200320080M4GB+AI4Protein/deep_bpe_3200

Unigram Tokenization Series

ModelVocab SizeParametersGPU MemoryTemplate
PETA-unigram-505080M4GB+AI4Protein/deep_unigram_50
PETA-unigram-10010080M4GB+AI4Protein/deep_unigram_100
PETA-unigram-20020080M4GB+AI4Protein/deep_unigram_200
PETA-unigram-40040080M4GB+AI4Protein/deep_unigram_400
PETA-unigram-80080080M4GB+AI4Protein/deep_unigram_800
PETA-unigram-1600160080M4GB+AI4Protein/deep_unigram_1600
PETA-unigram-3200320080M4GB+AI4Protein/deep_unigram_3200

💡 Different tokenization strategies may be better suited for specific tasks

Model Selection Guide

How to choose the right model?

  1. Based on Hardware Constraints:

    • Limited GPU (<8GB): ESM2-8M, ESM2-35M, ProSST
    • Medium GPU (8-16GB): ESM2-150M, ESM2-650M, ProtBert series
    • High-end GPU (24GB+): ESM2-3B, ProtT5-XL, Ankh-large
    • Multiple GPUs: ESM2-15B, ProtT5-XXL

  2. Based on Task Type:

    • Sequence classification: ESM2, ProtBert
    • Structure prediction: ESM2, Ankh
    • Generation tasks: ProtT5
    • Antibody design: IgBert, IgT5
    • Lightweight deployment: ProSST, PETA-base

  3. Based on Training Data:

    • General protein tasks: ESM2, ProtBert
    • Structure-aware tasks: Ankh
    • Antibody-specific: IgBert, IgT5
    • Custom tokenization needs: PETA series

🔍 All models are available through the Hugging Face Hub and can be easily loaded using their templates.

🔬 Supported Training Approaches

ApproachFull-tuningFreeze-tuningSES-AdapterAdaLoRAQLoRALoRADoRAIA3
Supervised Fine-Tuning✅✅✅✅✅✅✅✅

📚 Supported Datasets

Pre-training datasets
Supervised fine-tuning datasets (amino acid sequences/ foldseek sequences/ ss8 sequences)

✨ Only structural sequences are different for the same dataset, for example, DeepLocBinary_ESMFold and DeepLocBinary_AlphaFold2 share the same amino acid sequences, this means if you only want to use the aa_seqs, both are ok!

Supervised fine-tuning datasets (amino acid sequences)

📈 Supported Metrics

NameTorchmetricsProblem Type
accuracyAccuracysingle_label_classification/ multi_label_classification
recallRecallsingle_label_classification/ multi_label_classification
precisionPrecisionsingle_label_classification/ multi_label_classification
f1F1Scoresingle_label_classification/ multi_label_classification
mccMatthewsCorrCoefsingle_label_classification/ multi_label_classification
aucAUROCsingle_label_classification/ multi_label_classification
f1_maxF1ScoreMaxmulti_label_classification
spearman_corrSpearmanCorrCoefregression
mseMeanSquaredErrorregression

✈️ Requirements

Hardware Requirements

  • Recommended: NVIDIA RTX 3090 (24GB) or better
  • Actual requirements depend on your chosen protein language model

Software Requirements

Basic Installation

git clone https://github.com/tyang816/VenusFactory.git
cd VenusFactory
conda create -n venus python=3.10
conda activate venus # For Windows# source activate venus # For Linux
pip install -r requirements.txt

🚀 Quick Start with Venus Web UI

Start Venus Web UI

Get started quickly with our intuitive graphical interface powered by Gradio:

python ./src/webui.py

This will launch the Venus Web UI where you can:

  • Configure and run fine-tuning experiments
  • Monitor training progress
  • Evaluate models
  • Visualize results

Using Each Tab

We provide a detailed guide to help you navigate through each tab of the Venus Web UI.

1. Training Tab: Train your own protein language model

Model_Dataset_Config

Select a protein language model from the dropdown menu. Upload your dataset or select from available datasets and choose metrics appropriate for your problem type.

Training_Parameters Choose a training method (Freeze, SES-Adapter, LoRA, QLoRA etc.) and configure training parameters (batch size, learning rate, etc.).

Preview_CommandTraining_ProgressBest_ModelMonitor_Figs Click "Start Training" and monitor progress in real-time.

Metric_Results

Click "Download CSV" to download the test metrics results.

2. Evaluation Tab: Evaluate your trained model within a benchmark

Model_Dataset_Config

Load your trained model by specifying the model path. Select the same protein language model and model configs used during training. Select a test dataset and configure batch size. Choose evaluation metrics appropriate for your problem type. Finally, click "Start Evaluation" to view performance metrics.

3. Prediction Tab: Use your trained model to predict samples

Predict_Tab

Load your trained model by specifying the model path. Select the same protein language model and model configs used during training.

For single sequence: Enter a protein sequence in the text box.

For batch prediction: Upload a CSV file with sequences.

Batch

Click "Predict" to generate and view results.

4. Download Tab: Collect data from different sources with high efficiency
  • AlphaFold2 Structures: Enter UniProt IDs to download protein structures
  • UniProt: Search for protein information using keywords or IDs
  • InterPro: Retrieve protein family and domain information
  • RCSB PDB: Download experimental protein structures
5. Manual Tab: Detailed documentation and guides

Select a language (English/Chinese).

Navigate through the documentation using the table of contents and find step-by-step guides.

🧬 Code-line Usage

For users who prefer command-line interface, we provide comprehensive script solutions for different scenarios.

Training Methods: Various fine-tuning approaches for different needs

Full Model Fine-tuning

# Freeze-tuning: Train only specific layers while freezing others
bash ./script/train/train_plm_vanilla.sh

Parameter-Efficient Fine-tuning (PEFT)

# SES-Adapter: Selective and Efficient adapter fine-tuning
bash ./script/train/train_plm_ses-adapter.sh
# AdaLoRA: Adaptive Low-Rank Adaptation
bash ./script/train/train_plm_adalora.sh
# QLoRA: Quantized Low-Rank Adaptation
bash ./script/train/train_plm_qlora.sh
# LoRA: Low-Rank Adaptation
bash ./script/train/train_plm_lora.sh
# DoRA: Double Low-Rank Adaptation
bash ./script/train/train_plm_dora.sh
# IA3: Infused Adapter by Inhibiting and Amplifying Inner Activations
bash ./script/train/train_plm_ia3.sh

Training Method Comparison

MethodMemory UsageTraining SpeedPerformance
FreezeLowFastGood
SES-AdapterMediumMediumBetter
AdaLoRALowMediumBetter
QLoRAVery LowSlowerGood
LoRALowFastGood
DoRALowMediumBetter
IA3Very LowFastGood
Model Evaluation: Comprehensive evaluation tools

Basic Evaluation

# Evaluate model performance on test sets
bash ./script/eval/eval.sh

Available Metrics

  • Classification: accuracy, precision, recall, F1, MCC, AUC
  • Regression: MSE, Spearman correlation
  • Multi-label: F1-max

Visualization Tools

  • Training curves
  • Confusion matrices
  • ROC curves
  • Performance comparison plots
Structure Sequence Tools: Process protein structure information

ESM Structure Sequence

# Generate structure sequences using ESM-3
bash ./script/get_get_structure_seq/get_esm3_structure_seq.sh

Secondary Structure

# Predict protein secondary structure
bash ./script/get_get_structure_seq/get_secondary_structure_seq.sh

Features:

  • Support for multiple sequence formats
  • Batch processing capability
  • Integration with popular structure prediction tools
Data Collection Tools: Multi-source protein data acquisition

Format Conversion

# Convert CIF format to PDB
bash ./crawler/convert/maxit.sh

Metadata Collection

# Download metadata from RCSB PDB
bash ./crawler/metadata/download_rcsb.sh

Sequence Data

# Download protein sequences from UniProt
bash ./crawler/sequence/download_uniprot_seq.sh

Structure Data

# Download from AlphaFold2 Database
bash ./crawler/structure/download_alphafold.sh
# Download from RCSB PDB
bash ./crawler/structure/download_rcsb.sh

Features:

  • Automated batch downloading
  • Resume interrupted downloads
  • Data integrity verification
  • Multiple source support
  • Customizable search criteria

Supported Databases

DatabaseData TypeAccess MethodRate Limit
AlphaFold2StructuresREST APIYes
RCSB PDBStructuresFTP/HTTPNo
UniProtSequencesREST APIYes
InterProDomainsREST APIYes
Usage Examples: Common scenarios and solutions

Training Example

# Train a protein solubility predictor using ESM2
bash ./script/train/train_plm_lora.sh \
--model "facebook/esm2_t33_650M_UR50D" \
--dataset "DeepSol" \
--batch_size 32 \
--learning_rate 1e-4

Evaluation Example

# Evaluate the trained model
bash ./script/eval/eval.sh \
--model_path "path/to/your/model" \
--test_dataset "DeepSol_test"

Data Collection Example

# Download structures for a list of UniProt IDs
bash ./crawler/structure/download_alphafold.sh \
--input uniprot_ids.txt \
--output ./structures

💡 All scripts support additional command-line arguments for customization. Use --help with any script to see available options.

🙌 Citation

Please cite our work if you have used our code or data.

@article{tan2025venusfactory,
title={VenusFactory: A Unified Platform for Protein Engineering Data Retrieval and Language Model Fine-Tuning},
author={Tan, Yang and Liu, Chen and Gao, Jingyuan and Wu, Banghao and Li, Mingchen and Wang, Ruilin and Zhang, Lingrong and Yu, Huiqun and Fan, Guisheng and Hong, Liang and Zhou, Bingxin},
journal={arXiv preprint arXiv:2503.15438},
year={2025}
}

🎊 Acknowledgement

Thanks the support of Liang's Lab.

About

Easy data acquisition, benchmark resources, PLM fine-tuning for bio-researchers.

Resources

Readme
License
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages