AAAI2026 paper:Mitigating Error Accumulation in Co-Speech Motion Generation via Global Rotation Diffusion and Multi-Level Constraints

GlobalDiff is developed by Alibaba Cloud and licensed under the Apache License (Version 2.0)

create train data

cd Data/BEAT2 then run following command:

pythoncreate_lmdb.pypath_to_folder/beat_v2.0.0/beat_english_v2.0.0/

it takes a while to finish. then run

pythoncreate_hubert.pytrain_seq_size_60_stride_size_20_global.lmdb

it takes about one hour to finish.

you will get two folders: train_seq_size_60_stride_size_20_global.lmdb and train_seq_size_60_stride_size_20_global_wavlm.lmdb

train

unzip best.zip in Scripts/FM/ckpt/split/SimpleSpeechModel/

cd Scripts/FM
torchrun --nproc_per_node=4 TrainFixedExpressions.py

inference

pythonTest_FixedExpressions_pid_batch.pyckpt/split/DiffusionDITNetPartsFixedExpressions2PostNorm_LL_split_HorizonFlip_MaskedVAE3_W02_BoneDirLoss--data_root=path_to_folder/beat_v2.0.0/beat_english_v2.0.0/

this will inference the all the checkpoints sequentially. the fid is stored in

ckpt/split/DiffusionDITNetPartsFixedExpressions2PostNorm_LL_split_HorizonFlip_MaskedVAE3_W02_BoneDirLoss/record_2.txt

the results with best fid is stored in

ckpt/split/DiffusionDITNetPartsFixedExpressions2PostNorm_LL_split_HorizonFlip_MaskedVAE3_W02_BoneDirLoss/best_pid_2/

Note that this fid is based on our implementations to assess each checkpoint. Please follow the original EMAGE or SemTalkto calculate the fid for a fair comparison.

Visualize

follow EMAGE or SemTalk to use the SMPLX blender addon to visualize the npz results.

results

The results for person-2 are in best_pid_2.zip