Deep Learning from Scratch by Seth Weidman

Author:Seth Weidman
Language: eng
Format: epub
Publisher: O'Reilly Media
Published: 2019-09-12T16:00:00+00:00

mnist_soft = NeuralNetwork( layers=[Dense(neurons=89, activation=Tanh(), weight_init="glorot", dropout=0.8), Dense(neurons=10, activation=Linear(), weight_init="glorot")], loss = SoftmaxCrossEntropy(), seed=20190119)

and training the model with the same hyperparameters as before (exponential weight decay from an initial learning rate of 0.2 to a final learning rate of 0.05) results in:

Validation loss after 10 epochs is 0.285 Validation loss after 20 epochs is 0.232 Validation loss after 30 epochs is 0.199 Validation loss after 40 epochs is 0.196 Loss increased after epoch 50, final loss was 0.196, using the model from epoch 40 The model validation accuracy is: 96.95%

This is another significant decrease in loss over what we saw previously: the model achieves a minimum loss of 0.196, compared to 0.244 before.

The real power of dropout comes when we add more layers. Let’s change the model we’ve been using throughout this chapter to be a deep learning model, defining the first hidden layer to have twice as many neurons as did our hidden layer before (178) and our second hidden layer to have half as many (46). Our model looks like:

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.