An Introduction to Machine Learning by Miroslav Kubat

Author:Miroslav Kubat
Language: eng
Format: epub, pdf
Publisher: Springer International Publishing, Cham

How are the individual classifiers induced in the bagging approach?

What is meant by bootstrapping?

9.2 Schapire’s Boosting

Although the bagging approach often achieves impressive results, it suffers from a serious shortcoming: the voting classifiers have all been induced independently of each other from randomly selected data. One would surmise that a smarter—and perhaps more successful—approach should rely on a mechanism that makes the classifiers complement each other. For instance, this can be done by inducing each of them from training examples that are perceived as difficult by the other classifiers. Schapire’s boosting was invented with this idea in mind.

Induction of Three Mutually Complementing Classifiers Suppose that a random subset, T 1 ∈ T, of m training examples has been created. These are used to induce the first classifier, C 1. When testing this classifier on the entire training set, T, we will observe that it misclassifies a certain number of examples.

Suppose we now create another training subset, T 2 ∈ T. Let it consist of m examples selected in a manner that ensures that the previously induced C 1 classifies correctly 50% of them, misclassifying the remaining 50%. This means that T 2 is so difficult for C 1 that the classifier will not outperform a flipped coin. From the training subset thus created, the second classifier, C 2, is induced.

The two classifiers, C 1 and C 2, having been induced each from different examples, will inevitably differ in how they label certain instances. A tie-breaker is therefore needed. To this end, a third training subset, T 3, is created, consisting only of examples on which C 1 and C 2 differ. From this third subset, T 3, the third classifier, C 3, is induced.

The principle is summarized by the pseudocode in Table 9.2. When an example is presented, the master classifier collects the labels recommended by the three classifiers, and then returns the class that has received more votes.Table 9.2The algorithm of Schapire’s boosting

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.