Combining Bias and Variance Reduction Techniques for Regression Trees
Abstract
Gradient Boosting and bagging applied to regressors can reduce the error due to bias and variance respectively. Alternatively, Stochastic Gradient Boosting (SGB) and Iterated Bagging (IB) attempt to simultaneously reduce the contribution of both bias and variance to error. We provide an extensive empirical analysis of these methods, along with two alternate bias-variance reduction approaches — bagging Gradient Boosting (BagGB) and bagging Stochastic Gradient Boosting (BagSGB). Experimental results demonstrate that SGB does not perform as well as IB or the alternate approaches. Furthermore, results show that, while BagGB and BagSGB perform competitively for low-bias learners, in general, Iterated Bagging is the most effective of these methods.
Cite
Text
Suen et al. "Combining Bias and Variance Reduction Techniques for Regression Trees." European Conference on Machine Learning, 2005. doi:10.1007/11564096_76Markdown
[Suen et al. "Combining Bias and Variance Reduction Techniques for Regression Trees." European Conference on Machine Learning, 2005.](https://mlanthology.org/ecmlpkdd/2005/suen2005ecml-combining/) doi:10.1007/11564096_76BibTeX
@inproceedings{suen2005ecml-combining,
title = {{Combining Bias and Variance Reduction Techniques for Regression Trees}},
author = {Suen, Yuk Lai and Melville, Prem and Mooney, Raymond J.},
booktitle = {European Conference on Machine Learning},
year = {2005},
pages = {741-749},
doi = {10.1007/11564096_76},
url = {https://mlanthology.org/ecmlpkdd/2005/suen2005ecml-combining/}
}