Flexible Learning of Sparse Neural Networks via Constrained $l_0$ Regularization
Abstract
We propose to approach the problem of learning $L_0$-sparse networks using a constrained formulation of the optimization problem. This is in contrast to commonly used penalized approaches, which combine the regularization terms additively with the (surrogate) empirical risk. Our experiments demonstrate that we can obtain approximate solutions to the constrained optimization problem with comparable performance to state-of-the art methods for $L_0$-sparse training. Finally, we discuss how this constrained approach provides greater (hyper-)parameter interpretability and accountability from a practitioner's point of view.
Cite
Text
Gallego-Posada et al. "Flexible Learning of Sparse Neural Networks via Constrained $l_0$ Regularization." NeurIPS 2021 Workshops: LatinX_in_AI, 2021.Markdown
[Gallego-Posada et al. "Flexible Learning of Sparse Neural Networks via Constrained $l_0$ Regularization." NeurIPS 2021 Workshops: LatinX_in_AI, 2021.](https://mlanthology.org/neuripsw/2021/gallegoposada2021neuripsw-flexible/)BibTeX
@inproceedings{gallegoposada2021neuripsw-flexible,
title = {{Flexible Learning of Sparse Neural Networks via Constrained $l_0$ Regularization}},
author = {Gallego-Posada, Jose and Ramirez, Juan and Erraqabi, Akram},
booktitle = {NeurIPS 2021 Workshops: LatinX_in_AI},
year = {2021},
url = {https://mlanthology.org/neuripsw/2021/gallegoposada2021neuripsw-flexible/}
}