This library is a modification of my previous one. Click Here to check my previous library.
$ [sudo] pip3 install nicenet$ git clone https://github.com/Subhash3/Neural_Net_Using_NumPy.git>>>fromnicenetimportNeuralNetworkinputs=2outputs=1network=NeuralNetwork(inputs, outputs, cost="mse")
# Add 2 hidden layers with 16 neurons each and activation function 'tanh'network.add_layer(16, activation_function="tanh") network.add_layer(16, activation_function="tanh")
# Finish the neural network by adding the output layer with sigmoid activation function.network.compile(activation_function="sigmoid")The package contains a Dataset class to create a dataset.
>>>fromnicenetimportDatasetMake sure you have inputs and target values in seperate files in csv format.
input_file="inputs.csv"target_file="targets.csv"# Create a dataset object with the same inputs and outputs defined for the network.dataset_handler=Dataset(inputs, outputs)
dataset_handler.make_dataset(input_file, target_file)
data, size=dataset_handler.get_raw_data()If you want to manually make a dataset, follow these rules:
- Dataset must be a list of data samples.
- A data sample is a tuple containing inputs and target values.
- Input and target values are column vector of size (inputs x 1) and (outputs x 1) respectively.
For eg, a typical XOR data set looks something like :
>>>XOR_data= [
(
np.array([[0], [0]]),
np.array([[0]])
),
(
np.array([[0], [1]]),
np.array([[1]])
),
(
np.array([[1], [0]]),
np.array([[1]])
),
(
np.array([[1], [1]]),
np.array([[0]])
)
]
>>>size=4The library provides a Train function which accepts the dataset, dataset size, and two optional parameters epochs, and logging.
defTrain(self, dataset: T_Dataset, size, epochs=100, logging=False, epoch_logging=True, prediction_evaulator=None):
....
....For Eg: If you want to train your network for 1000 epochs.
>>>network.Train(data, size, epochs=1000)Notice that I didn't change the value of logging as I want the output to be printed for each epoch.
Plot a nice epoch vs error graph
>>>network.epoch_vs_error()Know how well the model performed.
>>>network.evaluate()To take a look at all the layers' info
>>>network.display()Sometimes, learning rate might have to be altered for better convergence.
>>>network.set_learning_rate(0.1)You can export a trained model to a json file which can be loaded and used for predictions in the future.
filename="model.json"network.export_model(filename)To load a model from an exported model (json) file. load_model is a static function, so you must not call this on a NeuralNetwork object!.
filename="model.json"network=NeuralNetwork.load_model(filename)def __init__(self, I, O, cost="mse"):
def__init__(self, I, O, cost="mse"):
""" Creates a Feed Forward Neural Network. Parameters ---------- I : int Number of inputs to the network O : int Number of outputs from the network [cost]: string The cost/loss function used by the neural network. Default value is 'mse' which stands for Mean Squared Error. Available options: mse => Mean Squared Error ce => Cross Entropy Returns ------- Doesn't return anything """def set_learning_rate(self, lr):
defset_learning_rate(self, lr):
""" Modifies the learning rate of the network. Parameters ---------- lr : float New learning rate Returns ------- Doesn't return anything """def add_layer(self, num_nodes, activation_function="sigmoid"):
defadd_layer(self, num_nodes, activation_function="sigmoid"):
""" Adds a layer to the network. Parameters ---------- num_nodes : int Number of nodes in the hidden layer [activation_function] :str It is an optional parameter. Specifies the activation function of the layer. Default value is sigmoid. Available options: sigmoid, tanh, linear, identity, softmax Returns ------- Doesn't return anything """def compile(self, activation_function="sigmoid"):
defcompile(self, activation_function="sigmoid"):
""" Basically, it just adds the output layer to the network. Parameters ---------- [activation_function] :str It is an optional parameter. Specifies the activation function of the layer. Default value is sigmoid. Returns ------- Doesn't return anything """def feedforward(self, input_array: T_Feature_Array):
deffeedforward(self, input_array: T_Feature_Array):
""" Feeds the given input throughout the network Parameters ---------- input_array : T_Feature_Array Input to be fed to the network. It is columnar vector of size Inputs x 1 Returns ------- all_outputs : T_Output_Array An array of all the outputs produced by each layer. """def backpropagate(self, target: T_Target_Array):
defbackpropagate(self, target: T_Target_Array):
""" Backpropagate the error throughout the network This function is called inside the model only. Parameters ---------- target : np.array() It is the ground truth value corresponding to the input. It is columnar vector of size Outputs x 1 Returns ------- Error : float # Returns the Mean Squared Error of the particular output Returns the error using the specified loss function. """def update_weights(self, input_array: T_Feature_Array):
defupdate_weights(self, input_array: T_Feature_Array):
""" Update the weights of the network. This function is called inside the model only. Parameters ---------- input_array : np.array() It is the input fed to the network It is columnar vector of size Inputs x 1 Returns ------- Doesn't return anything """def Train(self, dataset: T_Dataset, size, epochs=100, logging=False, epoch_logging=True, prediction_evaulator=None):
defTrain(self, dataset: T_Dataset, size, epochs=100, logging=False, epoch_logging=True, prediction_evaulator=None):
""" Trains the neural network using the given dataset. Parameters ---------- dataset : T_Dataset size : int Size of the dataset [epochs] : int An optional parameter. Number of epochs to train the network. Default value is 5000 [logging] : bool An optional parameter. If its true, all outputs from the network will be logged out onto STDOUT for each epoch. [epoch_logging] : bool An optional parameter. If it is true, Error in each epoch will be logged to STDOUT. [prediction_evaulator]: (prediction: T_Output_Array, target: T_Output_Array) -> bool An optional parameter. Used to evaluate the fed forward output with the actual target. Default value is 'Utils.judge_prediction' function. Returns ------- Doesn't return anything. """def predict(self, input_array: T_Feature_Array):
defpredict(self, input_array: T_Feature_Array):
""" Predicts the output using a given input. Parameters ---------- input_array : np.array() It is columnar vector of size Inputs x 1 It is the input fed to the network Returns ------- prediction : np.array() Predicted value produced by the network. """def epoch_vs_error(self):
defepoch_vs_error(self):
""" Plot error vs epoch graph Parameters ---------- Doesn't accept any parameters Returns ------- Doesn't return anything """def evaluate(self):
defevaluate(self):
""" Print the basic information about the network. Like accuracy, error ..etc. Parameters ---------- Doesn't accept any parameters Returns ------- Doesn't return anything """def display(self):
defdisplay(self):
""" Print the information of each layer of the network. It can be used to debug the network! Parameters ---------- Doesn't accept any parameters Returns ------- Doesn't return anything """def export_model(self, filename):
defexport_model(self, filename):
""" Export the model to a json file Parameters ---------- filename: str File name to export model Returns ------- Doesn't return anything """def load_model(filename):
defload_model(filename):
""" Load model from an eported (json) model Parameters ---------- filename : str Exported model (json) file Returns ------- brain : NeuralNetwork NeuralNetwork object """def __init__(self, num_nodes, inputs, activation_function, loss_function):
def__init__(self, num_nodes, inputs, activation_function, loss_function):
""" Layer constructor Parameters ---------- num_nodes : int No. of nodes in the layer inputs : int No. of inputs to the layer activation_function Returns ------- None """def feed(self, input_array: T_Feature_Array) -> T_Output_Array:
deffeed(self, input_array: T_Feature_Array) ->T_Output_Array:
""" Feeds the given input array to a particular layer. Parameters ---------- input_array: T_Feature_Array Input array to be fed to the layer Returns ------- output_array: T_Output_Array """def activate(self, x):
defactivate(self, x):
""" Passes the output array to an activation function. Parameters ---------- x Output array from a layer Returns ------- Activated output """def calculate_gradients(self, target_or_weights, layer_type, next_layer_deltas=None):
defcalculate_gradients(self, target_or_weights, layer_type, next_layer_deltas=None):
""" Calculates the gradients for each weight and bias Parameters ---------- target_or_weights This is either targers array of weights matrix. Specifically, it'll be the targets array while computing the gradients for the output layer and weights matrix of the next layer. layer_type This will either be "hidden" or "output" [next_layer_deltas] This is (not exactly) an optional parameter. This will be passed only while computing the gradients of a hidden layer. Returns ------- Doesn't return anything. But stores the gradients as a class attribute. """def update_weights(self, inputs, learning_rate):
defupdate_weights(self, inputs, learning_rate):
""" Tweak the weights of the layer. Parameters ---------- inputs: T_Feature_Array Input to this network learning_rate: float Learning rate of the entire network. Returns ------- Doesn't return anything. """def display(self):
defdisplay(self):
""" Display the metadata of the layer. """def __init__(self, I, O):
def__init__(self, I, O):
""" Dataset Constructor Parameters ---------- I: int No of inputs O: int No of outputs """def make_dataset(self, input_file, target_file):
defmake_dataset(self, input_file, target_file):
""" Creates a dataset Parameters ---------- input_file: str csv file containing the features/inputs. target_file: str csv file containing the targets. Returns ------- Doesn't return anything. """def get_raw_data(self):
defget_raw_data(self):
""" Returns the dataset which was made earlier in make_dataset method Parameters ---------- Doesn't accept anything Returns Tuple[T_Dataset, int] Dataset and its size """def display(self):
defdisplay(self):
""" Prints the dataset """def open_file(filename):
defopen_file(filename):
""" Just a helper function to open a given file and handle errors if any. Parameters ---------- filename: str Filename to be opened Returns ------- fhand A filehandler corresponding to the given file. """def get_min_max_values(self, array: T_Dataset):
defget_min_max_values(self, array: T_Dataset):
""" Computes the min and max of each feature, and each target label of the entire dataset. Parameters ---------- array : List[List[np.array]] List of datasamples datasample = [ column vector of features, column vector of of targets ] Returns ------- Tuple[List[float]] min and max values of features and targets (min_max_of_features, min_max_of_targets) min_max_of_features = List[[min_of_ith_feature, max_of_ith_feature]] min_max_of_targets = List[[min_of_ith_target, max_of_ith_target]] """def scale_data(self, array: T_Dataset, size):
defscale_data(self, array: T_Dataset, size):
""" Scales the data using min-max scaling method. parameters ---------- array: T_Dataset Dataset to be scaled. size: int Size of the given dataset. Returns ------- array: T_Dataset Scaled dataset. """- [x] Generalize the gradient descent algorithm
- [x] Generalise the loss function => Write a separate class for it!
- [x] Implement Cross Entropy Loss
- [ ] Data scaling
- [x] Min Max scaler
- [ ] Data Standardization
- [x] Change the datasample type to a tuple instead of a list.
- [x] Show Progress bar if epoch_logging is False
- [x] Use a function as a parameter to Train method to compare predictions and actual targets.
- [x] convert all camel-cased vars to snake-case.
- [x] API docs
- [x] Add doc strings to all functions.
- [x] Make the class/function declarations' docs collapsable.
- [x] Merge API md files and embed them in Readme.
- [x] Create a section, API, in README to provide documentation for all prototypes.
- [ ] Implement Batch Training
- [ ] Write a separate class for Scalers as the scaling methods increase.
- [ ] Linear and Relu activation functions
- [ ] Ability to perform regression
- [ ] Separate out outputlayer from other layers. => Create a separate class for output layer which inherits Layer.
- [ ] Convolution Nets
- [ ] Recurrent Nets