A deep learning library for training end-to-end Artificial Neural Networks (ANNs), primarily based on numpy and autograd. (This is a small 2-day project for learning purposes.)
Dense_ANN,Dense_Layer
zero_init,uniform_init,xavier_init,he_init,variance_scaling_init,constant_init
sigmoid,tanh,relu,leaky_relu,elu,swish,fixed_point
mse(Mean Squared Error),mae(Mean Absolute Error),binary_cross_entropy,hinge_loss
SGD_Optimizer(Stochastic Gradient Descent) capable of training fully connected ANNs.
L1 (lasso),L2 (ridge),Elastic Net
stop_loss_min,lr_scheduler,checkpoints
loss_plot,mean_validation_plot
- Uses mostly
numpywithautograd, which does not take advantage of GPU and parallelism as modern libraries would. - Only implements end-to-end training, not model-based deep learning.
git clone https://github.com/AFLProjects/deepl.gitcd deeplpython3 setup.pypython3 example.py
fromdeeplimportcorefromdeeplimporttrainingfromdeeplimportvisualizationimportautograd.numpyasnp# Example# Define structurestructure= (3, 3, 3)
nn=core.Dense_ANN(structure, [core.relu, core.fixed_point])
# Training datadata_size=16000train_x= [np.random.rand(3) for_inrange(data_size)]
train_y=100*train_x# Validation datavalidate_size=64validate_x= [np.random.rand(3) for_inrange(validate_size)]
validate_y=100*validate_x# Optimizer and parameterstrainer=training.SGD_Optimizer(nn,
loss=training.mse,
init=core.uniform_init,
init_args=(0, 1),
start_lr=0.1,
callbacks=[training.stop_loss_min,
training.checkpoints],
callback_args=[(10e-3,),
(250,)],
validate_x=validate_x,
validate_y=validate_y,
reg=training.lasso,
reg_params=(0.001,))
# Trainweights_tensor, loss_values=trainer.train(train_x, train_y, data_size)
# Plotsvisualization.loss_plot(trainer, 0.5, 'MSE')
visualization.mean_validation_plot(trainer, 0.5, 'MSE')
Performance for this specific case( (3,3,3) ) : init took 8.833e-06 seconds uniform_init took 2.813e-05 seconds init took 6.354e-05 seconds train took 9.781e-01 seconds