Convex Neural Networks
Abstract
Convexity has recently received a lot of attention in the machine learning community, and the lack of convexity has been seen as a major disadvantage of many learning algorithms, such as multi-layer artificial neural networks. We show that training multi-layer neural networks in which the number of hidden units is learned can be viewed as a convex optimization problem. This problem involves an infinite number of variables, but can be solved by incrementally inserting a hidden unit at a time, each time finding a linear classifier that minimizes a weighted sum of errors.
Cite
Text
Bengio et al. "Convex Neural Networks." Neural Information Processing Systems, 2005.Markdown
[Bengio et al. "Convex Neural Networks." Neural Information Processing Systems, 2005.](https://mlanthology.org/neurips/2005/bengio2005neurips-convex/)BibTeX
@inproceedings{bengio2005neurips-convex,
title = {{Convex Neural Networks}},
author = {Bengio, Yoshua and Roux, Nicolas L. and Vincent, Pascal and Delalleau, Olivier and Marcotte, Patrice},
booktitle = {Neural Information Processing Systems},
year = {2005},
pages = {123-130},
url = {https://mlanthology.org/neurips/2005/bengio2005neurips-convex/}
}