tensorgrad

A derivative of micrograd, altered to process multidimensional arrays of scalar values.

Example usage

Below is a slightly contrived example showing a number of possible supported operations:

fromtensorgrad.engineimportValuea=Value([[-4.0,-8.0],[-2.0,-4.0]])
b=Value([[2.0,6.0],[4.0,8.0]])
c=a+bd=a*b+b**3#applies the pow 3 to all values in the tensorc+=c+1#auto-converts the scalar 1 into a gradient-tracked tensor value of [[1,1],[1,1]] to match the shape of the tensor in cc+=1+c+ (-a)
d+=d*2+ (b+a).relu()
d+=3*d+ (b-a).relu()
e=c-df=e**2g=f/2.0g+=10.0/fprint(g.data) # prints the outcome of the forward pass for all four values in the tensorg.backward()
print(a.grad) # prints the numerical value of dg/d for all four values in the tensorprint(b.grad) # prints the numerical value of dg/db for all four values in the tensor

Tracing / visualization

For convenience, the notebook trace_graph.ipynb produces graphviz visualizations. E.g. this one below is of a simple 2D neuron, arrived at by calling draw_dot on the code below, and it shows both the data (left number in each node) and the gradient (right number in each node).

fromtensorgradimportnnn=nn.Neuron(2, [3, 3])
x= [Value([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0]]),
Value([[-2.0, -4.0, -5.0], [-6.0, -7.0, -8.0], [-9.0, -10.0, -11.0]])]
y=n(x)
y.backward()
dot=draw_dot(y)

License

MIT