Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound
Abstract
We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective.The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives -- both with uninformed (data-independent) and informed (data-dependent) priors.
Cite
Text
Zantedeschi et al. "Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound." Neural Information Processing Systems, 2021.Markdown
[Zantedeschi et al. "Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound." Neural Information Processing Systems, 2021.](https://mlanthology.org/neurips/2021/zantedeschi2021neurips-learning/)BibTeX
@inproceedings{zantedeschi2021neurips-learning,
title = {{Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound}},
author = {Zantedeschi, Valentina and Viallard, Paul and Morvant, Emilie and Emonet, Rémi and Habrard, Amaury and Germain, Pascal and Guedj, Benjamin},
booktitle = {Neural Information Processing Systems},
year = {2021},
url = {https://mlanthology.org/neurips/2021/zantedeschi2021neurips-learning/}
}