Variational Methods for Machine Learning with Applications to Deep Networks by unknow

Author:unknow
Language: eng
Format: epub
ISBN: 9783030706791
Publisher: Springer International Publishing

4.4.3.1 The Normalizing Factor

We consider a network with L layers and V l units on each layer l, taking in vector-shaped inputs x i. Thus, the output z l of each layer can be arranged into a vector, and the weights between two consecutive layers into a weight matrix W l with dimensions V l × (V l−1 + 1), where the + 1 stems from the inclusion of a bias term. The pre-activation of a layer l is given by , and for all except the last layer, this gets transformed according to the non-linear mapping , known as ReLU [38].

We make the simplifying assumption that the output z L of the network at the last layer L is distributed as a Gaussian and proceed to compute the normalizing constant K of the associated shifted distribution as

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.