Probabilistic Machine Learning for Civil Engineers by James-A. Goulet

Author:James-A. Goulet
Language: eng
Format: epub
Publisher: MIT Press

where is the weight defining the dependence between the jth variable of the 0th layer (i.e., the first covariate itself) and the ith hidden variable of the 1st layer. Following the same notation, describes the plane’s intercept for the ith hidden variable of the 1st layer. The only hidden variable on the second layer is defined as a linear combination of the two hidden variables on layer 1, so

The observation model is now defined by , so each observation of the system response is defined as . Note that implicitly depends on the covariates x, so that there are covariates implicitly associated with each observation yi. The complete data set is D = {Dx, Dy} = {(xi, yi), ∀i ∈ {1 : D}}.

In this second case, adding hidden variables did not help us to generalize our model to nonlinear system responses because linear functions of linear functions are themselves still linear. Therefore, no matter how many layers of hidden variables we add, the final model remains a hyperplane. In order to describe nonlinear functions, we need to introduce nonlinearities using activation functions.

Activation functions An activation function describes a nonlinear transformation of a hidden variable z. Common activation functions are the logistic, the hyperbolic tangent (tanh), and the rectified linear unit (ReLU),

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.