Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution
Abstract
This paper examines a novel gradient boosting framework for regression. We regularize gradient boosted trees by introducing subsampling and employ a modified shrinkage algorithm so that at every boosting stage the estimate is given by an average of trees. The resulting algorithm, titled "Boulevard'", is shown to converge as the number of trees grows. This construction allows us to demonstrate a central limit theorem for this limit, providing a characterization of uncertainty for predictions. A simulation study and real world examples provide support for both the predictive accuracy of the model and its limiting behavior.
Cite
Text
Zhou and Hooker. "Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution." Journal of Machine Learning Research, 2022.Markdown
[Zhou and Hooker. "Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution." Journal of Machine Learning Research, 2022.](https://mlanthology.org/jmlr/2022/zhou2022jmlr-boulevard/)BibTeX
@article{zhou2022jmlr-boulevard,
title = {{Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution}},
author = {Zhou, Yichen and Hooker, Giles},
journal = {Journal of Machine Learning Research},
year = {2022},
pages = {1-44},
volume = {23},
url = {https://mlanthology.org/jmlr/2022/zhou2022jmlr-boulevard/}
}