Statistical Pattern Recognition Booklet by Jimmy Azar

Author:Jimmy Azar [Azar, Jimmy]
Language: eng
Format: azw3
Published: 2015-09-13T16:00:00+00:00

Figure 2.35. A trained combiner. Stage 1 consists of base classifiers, while stage 2 consists of a combining classifier. The initial dataset may be split between a dataset used for training the base classifiers and another for training the combiner. Of course, the dataset used for training the combiner would still need to pass through the base classifiers to yield the outputs needed as features for the combiner input/feature space. Using an independent training set for the combiner is generally a good idea, however it also decreases the performance of the base classifiers which are not then benefitting from using the entire initial dataset. When addressing both fixed rules and trained combiners: if the base classifiers are undertrained (i.e. weakly trained), and the outputs are made comparable, then fixed rules may be suitable to use; also in case of a trained combiner, there may not be a need to use a separate training set for the combiner. However, if the base classifiers are overtrained, then the outputs would be biased and fixed rules are expected to perform poorly; in case of a trained combiner, an independent training set would be required to train the combiner.

Note that in this form of two-stage combining, if the combiner is a decision tree, then the parallel cascade of base classifiers is equivalent to sequential combining, whereby different objects may be handled by different base classifiers as determined by the combiner.

In summary, if a dataset is limited where a single training set is available, then fixed rules may be suitable provided overtraining of the base classifiers is avoided. If a trained combiner is employed, then the base classifiers also must not be overtrained. On the other hand, if two training sets are available, one for the base classifiers and one for the combiner, then some overtraining of the base classifiers is not as much a problem. An equal split of the training set however may not be optimal since the base classifiers and the combiner have different feature space dimensionalities: for the first, the dimensionality is equal to the number of features in the problem, whereas for the second, the dimensionality is equal to the number of classes multiplied by the number of base classifiers used. The subject of how the training set should be split is still inconclusive and requires further study.

In what follows, we introduce some methods that were devised for improving classification performance. The first such method is bootstrap aggregating. The method employs bootstrap sampling, i.e., sampling with replacement, in order to obtain from a set of samples, many sets also consisting of samples each. Thus it may happen that some samples from the original dataset are replicated whereas others may not be represented at all. With each object having an equal probability of being selected, , the expected fraction of unique samples appearing in a bootstrapped set can be shown to be , i.e., approximately 63.2%, whereas the remainder objects are expected to be replicates. The same classifier is then trained on each of the bootstrapped sets, and a voting scheme gives the final classification.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.