ntk

{KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch

[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

Official implementation for the paper: Deep Learning under Continuous Distribution Shift: The Non-Stationary NTK and Spectral Tracking SDE for Quantitative Finance

[Oakland 2024] Exploring the Orthogonality and Linearity of Backdoor Attacks

"Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics" by Wuyang Chen, Xinyu Gong, Yunchao Wei, Humphrey Shi, Zhicheng Yan, Yi Yang, and Zhangyang Wang

Neural Tangent Kernel (NTK) module for the scikit-learn library

Convolutions and more as einsum for PyTorch

[Neurips 2022] "Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis" by Wuyang Chen*, Wei Huang*, Xinyu Gong, Boris Hanin, Zhangyang Wang

Infinite-width neural networks from a practical point of view

Quantum Banking System combines cutting-edge AI technology with modern banking features to provide an unparalleled financial management experience.

Cost-effective Machine Learning and Benchmarking for LLMs

Reproduction of the Neural Tangent Kernel phenomenon on a 1D regression task, with an activation-function comparison testing the prediction that smoother activations linearize better at fixed width.

RoPE Playground – Rotary Positional Embeddings in PyTorch