How to export a ZeRO checkpoint for PyTorch.

[yanring]

I noticed that ZeRO currently provides apis for save&load checkpoints, but it seems to only be used in scripts that also use ZeRO. So is there a way to export the model as a generic PyTorch .pth file?

Thanks!

[gongjingcs]

I noticed that ZeRO currently provides apis for save&load checkpoints, but it seems to only be used in scripts that also use ZeRO. So is there a way to export the model as a generic PyTorch .pth file?

Thanks!

Hello, I have the same doubts. I used zero to pretrain a model, but did not use zero in finetune, so how do I load this model?