Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks
Abstract
We present a comprehensive study of multilayer neural networks with binary activation, relying on the PAC-Bayesian theory. Our contributions are twofold: (i) we develop an end-to-end framework to train a binary activated deep neural network, (ii) we provide nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks. Our results are obtained by minimizing the expected loss of an architecture-dependent aggregation of binary activated deep neural networks. Our analysis inherently overcomes the fact that binary activation function is non-differentiable. The performance of our approach is assessed on a thorough numerical experiment protocol on real-life datasets.
Cite
Text
Letarte et al. "Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks." Neural Information Processing Systems, 2019.Markdown
[Letarte et al. "Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks." Neural Information Processing Systems, 2019.](https://mlanthology.org/neurips/2019/letarte2019neurips-dichotomize/)BibTeX
@inproceedings{letarte2019neurips-dichotomize,
title = {{Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks}},
author = {Letarte, Gaël and Germain, Pascal and Guedj, Benjamin and Laviolette, Francois},
booktitle = {Neural Information Processing Systems},
year = {2019},
pages = {6872-6882},
url = {https://mlanthology.org/neurips/2019/letarte2019neurips-dichotomize/}
}