fast-weights

Official repository for the paper "A Modern Self-Referential Weight Matrix That Learns to Modify Itself" (ICML 2022 & NeurIPS 2021 Deep RL Workshop) and "Accelerating Neural Self-Improvement via Bootstrapping" (ICLR 2023 Workshop)

Implementation of the paper [Using Fast Weights to Attend to the Recent Past](https://arxiv.org/abs/1610.06258)

Official repository for the paper "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers" (NeurIPS 2021)

Implementation of Fast Weight Attention

Official repository for the paper "Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules" (NeurIPS 2022)

PyTorch Language Modeling Toolkit for Fast Weight Programmers

Official repository for the paper "Automating Continual Learning"

Weights that never freeze. A continual-learning architecture, and the pre-registered experiment that falsified its central claim. https://arxiv.org/abs/2607.20792

Official repository for the paper "Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules" (ICLR 2023)

Your language model forgets everything. Sillage gives it a 4 MB memory that persists across sessions, no gradients, no fine-tuning, no growing index. Beats an unbounded kNN-LM at 1/13 the storage (perplexity 31 → 17). CPU-only, fully reproducible, four preprints with Zenodo DOIs.

Usage kit for the Fractale models — LMs whose long-term memory is a self-written fast-weight thought bank

Official repository for the paper "Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions" (EMNLP 2023)

PyTorch implementation of DCT fast weight RNNs

PyTorch Implementation of the paper [Using Fast Weights to Attend to the Recent Past]

A lightweight, Hebbian-based AI engine running natively on WebGPU. Zero-latency, privacy-focused.

Distributed Engrammics — gradient-free transfer of a persistent fast-weight associative state between model instances (DeltaNet, RWKV-7). Paper + reproducible code.

Inference-time drift experiment demonstrating reduced repetition collapse in frozen LLMs.

Naturally fractal, self-modifying language model research with persistent in-weights memory.

Official implementation of Rethinking Expressivity and Efficiency in Test-Time Training

Interactive explainer: BDH's attention has no softmax, so it is exactly a Hebbian synaptic memory — one fixed-size matrix written once per token. Run both forms, watch them agree to 1e-16, then break it with one toggle. DataForge 2026, Pathway track.