Mastering Predictive Analytics with R by Rui Miguel Forte

Author:Rui Miguel Forte [Forte, Rui Miguel]
Language: eng
Format: azw3, pdf
Publisher: Packt Publishing
Published: 2015-06-17T04:00:00+00:00

Cross-validation

We've seen that many times in the real world, we come across a situation where we don't have an available test data set that we can use in order to measure the performance of our model on unseen data. The most typical reason is that we have very few data overall and want to use all of it to train our model. Another situation is that we want to keep a sample of the data as a validation set to tune some model meta parameters such as cost and gamma for SVMs with radial kernels, and as a result, we've already reduced our starting data and don't want to reduce it further.

Whatever the reason for the lack of a test data set, we already know that we should never use our training data as a measure of model performance and generalization because of the problem of overfitting. This is especially relevant for powerful and expressive models such as the nonlinear models of neural networks and SVMs with radial kernels that are often capable of approximating the training data very closely but may end up failing to generalize well on unseen data. In this section, we introduce the notion of cross-validation, which is perhaps best explained by a diagram:

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.