Synaptic

Important: Synaptic 2.x is in stage of discussion now! Feel free to participate

Synaptic is a javascript neural network library for node.js and the browser, its generalized algorithm is architecture-free, so you can build and train basically any type of first order or even second order neural network architectures.

This library includes a few built-in architectures like multilayer perceptrons, multilayer long-short term memory networks (LSTM), liquid state machines or Hopfield networks, and a trainer capable of training any given network, which includes built-in training tasks/tests like solving an XOR, completing a Distracted Sequence Recall task or an Embedded Reber Grammar test, so you can easily test and compare the performance of different architectures.

The algorithm implemented by this library has been taken from Derek D. Monner's paper:

A generalized LSTM-like training algorithm for second-order recurrent neural networks

There are references to the equations in that paper commented through the source code.

Introduction

If you have no prior knowledge about Neural Networks, you should start by reading this guide.

If you want a practical example on how to feed data to a neural network, then take a look at this article.

You may also want to take a look at this article.

Demos

• Solve an XOR
• Discrete Sequence Recall Task
• Learn Image Filters
• Paint an Image
• Self Organizing Map
• Read from Wikipedia
• Creating a Simple Neural Network (Video)

The source code of these demos can be found in this branch.

Getting started

• Neurons
• Layers
• Networks
• Trainer
• Architect

To try out the examples, checkout the gh-pages branch.

git checkout gh-pages

Overview

Installation

In node

You can install synaptic with npm:

npm install synaptic --save

In the browser

You can install synaptic with bower:

bower install synaptic

Or you can simply use the CDN link, kindly provided by CDNjs

<scriptsrc="https://cdnjs.cloudflare.com/ajax/libs/synaptic/1.1.2/synaptic.js"></script>

Usage

varsynaptic=require('synaptic');// this line is not needed in the browservarNeuron=synaptic.Neuron,Layer=synaptic.Layer,Network=synaptic.Network,Trainer=synaptic.Trainer,Architect=synaptic.Architect;

Now you can start to create networks, train them, or use built-in networks from the Architect.

Examples

Perceptron

This is how you can create a simple perceptron:

.

functionPerceptron(input,hidden,output){// create the layersvarinputLayer=newLayer(input);varhiddenLayer=newLayer(hidden);varoutputLayer=newLayer(output);// connect the layersinputLayer.project(hiddenLayer);hiddenLayer.project(outputLayer);// set the layersthis.set({input: inputLayer,hidden: [hiddenLayer],output: outputLayer});}// extend the prototype chainPerceptron.prototype=newNetwork();Perceptron.prototype.constructor=Perceptron;

Now you can test your new network by creating a trainer and teaching the perceptron to learn an XOR

varmyPerceptron=newPerceptron(2,3,1);varmyTrainer=newTrainer(myPerceptron);myTrainer.XOR();// { error: 0.004998819355993572, iterations: 21871, time: 356 }myPerceptron.activate([0,0]);// 0.0268581547421616myPerceptron.activate([1,0]);// 0.9829673642853368myPerceptron.activate([0,1]);// 0.9831714267395621myPerceptron.activate([1,1]);// 0.02128894618097928

Long Short-Term Memory

This is how you can create a simple long short-term memory network with input gate, forget gate, output gate, and peephole connections:

functionLSTM(input,blocks,output){// create the layersvarinputLayer=newLayer(input);varinputGate=newLayer(blocks);varforgetGate=newLayer(blocks);varmemoryCell=newLayer(blocks);varoutputGate=newLayer(blocks);varoutputLayer=newLayer(output);// connections from input layervarinput=inputLayer.project(memoryCell);inputLayer.project(inputGate);inputLayer.project(forgetGate);inputLayer.project(outputGate);// connections from memory cellvaroutput=memoryCell.project(outputLayer);// self-connectionvarself=memoryCell.project(memoryCell);// peepholesmemoryCell.project(inputGate);memoryCell.project(forgetGate);memoryCell.project(outputGate);// gatesinputGate.gate(input,Layer.gateType.INPUT);forgetGate.gate(self,Layer.gateType.ONE_TO_ONE);outputGate.gate(output,Layer.gateType.OUTPUT);// input to output direct connectioninputLayer.project(outputLayer);// set the layers of the neural networkthis.set({input: inputLayer,hidden: [inputGate,forgetGate,memoryCell,outputGate],output: outputLayer});}// extend the prototype chainLSTM.prototype=newNetwork();LSTM.prototype.constructor=LSTM;

These are examples for explanatory purposes, the Architect already includes Multilayer Perceptrons and Multilayer LSTM network architectures.

Contribute

Synaptic is an Open Source project that started in Buenos Aires, Argentina. Anybody in the world is welcome to contribute to the development of the project.

If you want to contribute feel free to send PR's, just make sure to run npm run test and npm run build before submitting it. This way you'll run all the test specs and build the web distribution files.

<3