Code for "Learning to Read Out: Unembedding Dynamics in Language Model Pretraining" — parameter-trajectory crosscoders on the unembedding matrix W_U. Pretrained crosscoders: hf.co/hematteo/parameter-trajectory-crosscoders
language-modelspythiasparse-autoencodersinterpretabilitymechanistic-interpretabilitycrosscoderspretraining-dynamics
-
Updated
Jul 7, 2026 - Python