Lossless Online Bayesian Bagging
Herbert K. H. Lee, Merlise A. Clyde; 5(Feb):143--151, 2004.
Abstract
Bagging frequently improves the predictive performance of a model.
An online version has recently been introduced, which attempts to
gain the benefits of an online algorithm while approximating regular
bagging. However, regular online bagging is an approximation to its
batch counterpart and so is not lossless with respect to the bagging
operation. By operating under the Bayesian paradigm, we introduce
an online Bayesian version of bagging which is exactly equivalent to
the batch Bayesian version, and thus when combined with a lossless
learning algorithm gives a completely lossless online bagging
algorithm. We also note that the Bayesian formulation resolves a
theoretical problem with bagging, produces less variability in
its estimates, and can improve predictive performance for smaller
data sets.
[abs]
[pdf][ps.gz][ps]