AR-Boost: Reducing Overfitting by a Robust Data-Driven Regularization Strategy
Abstract
We introduce a novel, robust data-driven regularization strategy called Adaptive Regularized Boosting (AR-Boost), motivated by a desire to reduce overfitting. We replace AdaBoost’s hard margin with a regularized soft margin that trades-off between a larger margin, at the expense of misclassification errors. Minimizing this regularized exponential loss results in a boosting algorithm that relaxes the weak learning assumption further : it can use classifiers with error greater than $\frac{1}{2}$ . This enables a natural extension to multiclass boosting, and further reduces overfitting in both the binary and multiclass cases. We derive bounds for training and generalization errors, and relate them to AdaBoost. Finally, we show empirical results on benchmark data that establish the robustness of our approach and improved performance overall.
Cite
Text
Saha et al. "AR-Boost: Reducing Overfitting by a Robust Data-Driven Regularization Strategy." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2013. doi:10.1007/978-3-642-40994-3_1Markdown
[Saha et al. "AR-Boost: Reducing Overfitting by a Robust Data-Driven Regularization Strategy." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2013.](https://mlanthology.org/ecmlpkdd/2013/saha2013ecmlpkdd-arboost/) doi:10.1007/978-3-642-40994-3_1BibTeX
@inproceedings{saha2013ecmlpkdd-arboost,
title = {{AR-Boost: Reducing Overfitting by a Robust Data-Driven Regularization Strategy}},
author = {Saha, Baidya Nath and Kunapuli, Gautam and Ray, Nilanjan and Maldjian, Joseph A. and Natarajan, Sriraam},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2013},
pages = {1-16},
doi = {10.1007/978-3-642-40994-3_1},
url = {https://mlanthology.org/ecmlpkdd/2013/saha2013ecmlpkdd-arboost/}
}