Machine Learning with R by Abhijit Ghatak

Author:Abhijit Ghatak
Language: eng
Format: epub, pdf
Publisher: Springer Singapore, Singapore

In practice, we stop when the Euclidean norm, is less than a defined tolerance value. The gradient of is - (Refer Eqn 1.​7.​2).

Gradient descent tries to minimize the function f(w) by solving a sequence of simple quadratic approximations to f(w). It makes use of Taylor’s theorem, which states that if a function is differentiable at a point x, than there exists a linear approximation at the point x, defined by a function such that-

(3.12.2)

3.13 Building a Machine Learning Algorithm

We have discussed the constructs that go in to make a machine learning algorithm. Let us concisely go through the recipe for building machine learning algorithms.

As you may appreciate now, most learning algorithms start with a data set which is i.i.d., from which the model is constructed. The model needs to be evaluated for overfitting/underfitting. This is directly related to the capacity of the model.

The loss function of the model is a combination of its fit and its capacity. We therefore need to regularize our model to find an optimal balance between the fit and the model capacity by an optimization process. If the model is linear it has a closed form solution3 and we can use appropriate optimization algorithms to optimize the loss function; however, if the model is nonlinear the loss function may not be optimized in closed form and would require an iterative numerical optimization procedure like gradient descent or coordinate descent. The model capacity defines the structural parameters of the model.

The hyperparameters of the model are inherent to the model itself which controls its behavior. We have to search out the best parameter values of the model by a process known as model tuning. This is achieved by CV.

Finally, after arriving at the “best” algorithm, we evaluate the model on the unseen data and compare the results with other algorithms.

This recipe is applicable to both supervised and unsupervised learning algorithms.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.