Machine Learning for Data Streams by Albert Bifet

Author:Albert Bifet
Language: eng
Format: epub
Tags: data mining; stream; data; mining; statistics; techniques; analysis; learning; extract; algorithm; data stream; mining; MOA; massive online analysis; software; implementation; applications; approximation; big data
Publisher: MIT Press

7.2 Weighted Majority

The Weighted Majority Algorithm, proposed by Littlestone and Warmuth [160], combines N existing predictors called “experts” (in this context), and learns to adjust their weights over time. It is similar to the perceptron algorithm, but its update rule changes the weights multiplicatively rather than additively. Unlike the perceptron, it can be shown to converge to almost the error rate of the best expert, plus a small term depending on N. It is thus useful when the number of experts is large and many of them perform poorly.

The algorithm appears in figure 7.1. The input to the algorithm is a stream of items requiring prediction, each followed by its correct label. Upon receiving item number t, xt, the algorithm emits a prediction Å·t for the label of xt. Then the algorithm receives the correct label yt for xt. In the algorithm the sign function returns 0 for negative numbers and 1 for nonnegative ones.

Figure 7.1

The Weighted Majority algorithm.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.