MicroGrad

A TypeScript implementation of an autograd engine for educational purposes.

Overview

MicroGrad implements backpropagation (reverse-mode autodiff) over a dynamically built Directed Acyclic Graph (DAG). This project demonstrates how to implement automatic differentiation principles in TypeScript.

Key Components

• Value Class: Core autodiff functionality with gradient computation
• Neural Network Primitives: Simple Neuron, Layer, and MLP implementations
• Graph Visualization: Tools to visualize computation graphs

Key improvements over the Python version

• API Design: Both instance and static methods for operations compared to instance-only methods
• Higher Order Gradients: Support for computing higher-order derivatives
• Extended Math: Additional operations including log, exp, tanh, and sigmoid
• Gradient Tools: Methods for gradient health checks and gradient clipping
• Performance: Iterative stack-based topological sort for better efficiency

Usage Example

import{Value}from'@2bad/micrograd';// Create computation graphconsta=newValue(-4.0,'a')constb=newValue(2.0,'b')letc=Value.add(a,b,'c')// a + bletd=Value.add(Value.mul(a,b),Value.pow(b,3),'d')// a * b + b**3// c += c + 1c=Value.add(c,Value.add(c,newValue(1.0)))// c += 1 + c + (-a)c=Value.add(c,Value.add(Value.add(newValue(1.0),c),Value.negate(a)))// d += d * 2 + (b + a).relu()constbPlusA=Value.add(b,a)d=Value.add(d,Value.add(Value.mul(d,2),Value.relu(bPlusA)))// d += 3 * d + (b - a).relu()constbMinusA=Value.sub(b,a)d=Value.add(d,Value.add(Value.mul(3,d),Value.relu(bMinusA)))// e = c - dconste=Value.sub(c,d,'e')// f = e**2constf=Value.pow(e,2,'f')// g = f / 2.0letg=Value.div(f,2.0,'g')// g += 10.0 / fg=Value.add(g,Value.div(10.0,f))// Forward passconsole.log(g.data);// Value of the computation// Backward pass (compute gradients)g.backward();// Access gradientsconsole.log(a.grad);// dg/daconsole.log(b.grad);// dg/db

Building and Testing

# Install dependencies
npm install
# Build the project
npm run build
# Run tests
npm test

Acknowledgements

This project is inspired by micrograd by Andrej Karpathy. The TypeScript implementation extends the core concepts with additional features and type safety.