A reverse-mode automatic differentiation engine with neural networks, built in C++ with Python bindings.

---

AutoNeuroNet is a fully implemented automatic differentiation engine with custom matrices, a full neural network architecture, and a training pipeline. It comes with Python bindings via PyBind11, enabling quick, easy network development in Python, backed by C++ for enhanced speed and performance.

Table of Contents

• Features
• Installation
• Quickstart
• Project Structure
• Building from Source
• Testing
• Demos
• API Overview
• Documentation
• References
• License

Features

• Reverse-Mode Automatic Differentiation - Scalar-level AD with full computation graph and backpropagation
• Custom Matrix Library - 2D differentiable matrices with element-wise and matrix operations
• Neural Network Layers - Linear, ReLU, LeakyReLU, Sigmoid, Tanh, SiLU, ELU, Softmax
• Loss Functions - MSELoss, MAELoss, BCELoss, CrossEntropyLoss, CrossEntropyLossWithLogits
• Optimizers - GradientDescent, SGD (with momentum), Adagrad, RMSProp, Adam, AdamW
• Weight Initialization - Kaiming (He) and Xavier (Glorot) initialization
• Model Persistence - Save and load trained model weights
• NumPy Interop - Convert between NumPy arrays and AutoNeuroNet matrices
• Python Bindings - Full C++ performance accessible from Python via PyBind11
• Cross-Platform - Builds on Linux, macOS, and Windows (Python 3.9 - 3.13)

Installation

Install from PyPI with uv (recommended):

uv add autoneuronet

Or with pip:

pip install autoneuronet

To include dependencies for running the demos:

uv add "autoneuronet[demo]"# or: pip install "autoneuronet[demo]"

Quickstart

Scalar Automatic Differentiation

importautoneuronetasannx=ann.Var(2.0)
y=x**2+x*3.0+1.0# Set the final gradient to 1.0 and perform backpropagationy.setGrad(1.0)
y.backward()
print(f"y: {y.val}") # 11.0 = (2)^2 + 3(2) + 1print(f"dy/dx: {x.grad}") # 7.0 = 2(2) + 3

Matrix Initialization

importautoneuronetasannX=ann.Matrix(10, 1) # shape: (10, 1)y=ann.Matrix(10, 1) # shape: (10, 1)foriinrange(10):
X[i, 0] =ann.Var(i)
y[i, 0] =5.0*i+3.0# y = 5x + 3

Matrix Math

importautoneuronetasannX=ann.Matrix(2, 2)
X[0] = [1.0, 2.0]
X[1] = [3.0, 4.0]
Y=ann.Matrix(2, 2)
Y[0] = [5.0, 6.0]
Y[1] = [7.0, 8.0]
Z=X @ Y# or ann.matmul(X, Y)print(Z)
# Output:# Matrix(2 x 2) =# 19.000000 22.000000# 43.000000 50.000000

NumPy to Matrix

importautoneuronetasannimportnumpyasnpx=np.array([[1.0, 2.0], [3.0, 4.0]])
X=ann.numpy_to_matrix(x)
print(X)
# Output:# Matrix(2 x 2) =# 1.000000 2.000000# 3.000000 4.000000

Neural Networks, Loss Functions, and Optimizers

importautoneuronetasannmodel=ann.NeuralNetwork(
[
ann.Linear(784, 256, init="kaiming"),
ann.ReLU(),
ann.Linear(256, 128, init="kaiming"),
ann.ReLU(),
ann.Linear(128, 10, init="kaiming"),
ann.Softmax(),
]
)
optimizer=ann.SGDOptimizer(
learning_rate=1e-2, model=model, momentum=0.9, weight_decay=1e-4
)
print(model)

Training Loop

loss=ann.MSELoss(labels, logits)
loss.setGrad(1.0)
loss.backward()
optimizer.optimize()
optimizer.resetGrad()
print(f"Loss: {loss.getVal()}")

Project Structure

AutoNeuroNet/
├── include/ # C++ header files
│ ├── Var.hpp # Scalar automatic differentiation
│ ├── Matrix.hpp # 2D differentiable matrix
│ ├── NeuralNetwork.hpp # Layer abstractions and network container
│ ├── Optimizers.hpp # Optimizer algorithms
│ └── LossFunctions.hpp # Loss function implementations
├── src/ # C++ implementation files
│ ├── Var.cpp
│ ├── Matrix.cpp
│ ├── NeuralNetwork.cpp
│ ├── Optimizers.cpp
│ └── LossFunctions.cpp
├── python/autoneuronet/ # Python package
│ ├── __init__.py # Re-exports C++ bindings
│ └── __init__.pyi # Type stubs for IDE support
├── demos/ # Example scripts and notebooks
│ ├── automatic_differentiation.cpp
│ ├── numeric_differentiation.cpp
│ ├── linear_regression.cpp
│ ├── linear_regression_demo.ipynb
│ ├── moons_classification_demo.ipynb
│ ├── mnist_demo.ipynb
│ └── gradient_descent_3d.py
├── docs/ # Documentation source (MkDocs)
│ ├── index.md
│ ├── install.md
│ ├── quickstart.md
│ └── api.md
├── .github/workflows/ # CI/CD
│ └── wheels.yml # Cross-platform wheel builds
├── pybind_wrapper.cpp # PyBind11 binding definitions
├── CMakeLists.txt # CMake build configuration
├── pyproject.toml # Python package metadata
├── mkdocs.yml # Documentation site config
├── uv.lock # uv-managed dependency lockfile
└── LICENSE # Apache License 2.0

Building from Source

Prerequisites

• C++17 compatible compiler
• CMake >= 3.20
• Python >= 3.9
• uv (recommended) or pip
• pybind11 (pulled in automatically by the build backend)

Clone and Build

git clone https://github.com/RishabSA/AutoNeuroNet.git
cd AutoNeuroNet

Install the Python package locally with uv (recommended):

# Creates .venv, builds the C++ extension via scikit-build-core + pybind11,# installs the package and all dependencies from uv.lock.
uv sync --extra demo

Run any command inside the managed environment with uv run:

uv run python -c "import autoneuronet as ann; print(ann.Var(2.0))"

Or install with pip:

pip install ".[demo]"

Build the C++ library directly:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build

Compile C++ Demos

# Automatic differentiation example
g++ demos/automatic_differentiation.cpp src/Var.cpp -I include -o demos/automatic_differentiation
# Linear regression example
g++ demos/linear_regression.cpp src/Var.cpp src/Matrix.cpp src/NeuralNetwork.cpp src/Optimizers.cpp src/LossFunctions.cpp -I include -o demos/linear_regression

Testing

AutoNeuroNet ships with a pytest suite covering the full Python API — scalar Var, Matrix, activations, layers, losses, optimizers, model persistence, and NumPy interop — including numeric grad_check verification of every backward pass.

Install the test extra and run the suite with uv:

uv sync --extra test
uv run pytest

Or with pip:

pip install ".[test]"
pytest

Common invocations:

uv run pytest -v # verbose
uv run pytest tests/test_var_backward.py # a single file
uv run pytest -k "grad"# match by name
uv run pytest --cov=autoneuronet # coverage report

The suite lives under tests/ and is organized by component: test_var_*.py, test_matrix_*.py, test_layer_*.py, test_optimizer_*.py, test_losses.py, test_neural_network.py, test_model_persistence.py, test_numpy_interop.py, and test_operations_module.py. Shared helpers (including grad_check_var / grad_check_matrix) live in tests/conftest.py.

Demos

Demo	Description	Type
MNIST Classification	Handwritten digit recognition on MNIST	Jupyter Notebook
Moons Classification	Binary classification on sklearn moons dataset	Jupyter Notebook
Linear Regression	Simple linear regression walkthrough	Jupyter Notebook
3D Gradient Descent	3D visualization of gradient descent	Python Script
Automatic Differentiation	Scalar AD basics	C++
Numeric Differentiation	Numeric vs automatic differentiation comparison	C++
Linear Regression (C++)	Full training pipeline in C++	C++

API Overview

Core Classes

Class	Description
Var	Differentiable scalar with reverse-mode AD. Supports arithmetic, trig, log, exp, and activation functions.
Matrix	2D container of Var objects. Supports element-wise ops, matrix multiplication (@), and activations.

Layers

Layer	Description
Linear(in, out, init)	Fully connected layer. init: "kaiming" or "xavier".
ReLU	Rectified Linear Unit
LeakyReLU(alpha)	Leaky ReLU with configurable negative slope
Sigmoid	Sigmoid activation
Tanh	Hyperbolic tangent
SiLU	Sigmoid Linear Unit (Swish)
ELU(alpha)	Exponential Linear Unit
Softmax	Softmax normalization

Loss Functions

Function	Use Case
MSELoss	Regression
MAELoss	Regression
BCELoss	Binary classification
CrossEntropyLoss	Multi-class classification (with probabilities)
CrossEntropyLossWithLogits	Multi-class classification (with raw logits)

Optimizers

Optimizer	Key Parameters
GradientDescentOptimizer	learning_rate
SGDOptimizer	learning_rate, momentum, weight_decay
AdagradOptimizer	learning_rate, epsilon
RMSPropOptimizer	learning_rate, decay_rate, epsilon
AdamOptimizer	learning_rate, beta1, beta2, epsilon
AdamWOptimizer	learning_rate, beta1, beta2, epsilon, weight_decay

Utility Functions

Function	Description
matmul(A, B)	Matrix multiplication (also available as A @ B)
numpy_to_matrix(arr)	Convert a NumPy array to an AutoNeuroNet Matrix

For the full API reference, see the documentation.

Documentation

Full documentation is available at rishabsa.github.io/AutoNeuroNet.