Replacing LayerNorm with Dynamic Tanh (DyT) in DistilGPT2 + LoRA, evaluated on RE-WILD, Alpaca, and ShareGPT.
deep-learningpytorchtransformerresearch-projectlorapythiafine-tuningpefthuggingfacehpmldistilgpt2dytdynamic-tanhrewild
-
Updated
Sep 3, 2026 - Jupyter Notebook