Dataset number and model size

[XiaoqiangZhou]

Thanks for sharing this great work.

A question is that there are just 4k dataset samples. Is it enough to train a Transformer (with 1.3B parameter) on this small dataset. I think it may be easy to overfit. How to avoid this?

Thanks