Failures of Gradient-Based Deep Learning
Abstract
In recent years, Deep Learning has become the go-to solution for a broad range of applications, often outperforming state-of-the-art. However, it is important, for both theoreticians and practitioners, to gain a deeper understanding of the difficulties and limitations associated with common approaches and algorithms. We describe four types of simple problems, for which the gradient-based algorithms commonly used in deep learning either fail or suffer from significant difficulties. We illustrate the failures through practical experiments, and provide theoretical insights explaining their source, and how they might be remedied.
Cite
Text
Shalev-Shwartz et al. "Failures of Gradient-Based Deep Learning." International Conference on Machine Learning, 2017.Markdown
[Shalev-Shwartz et al. "Failures of Gradient-Based Deep Learning." International Conference on Machine Learning, 2017.](https://mlanthology.org/icml/2017/shalevshwartz2017icml-failures/)BibTeX
@inproceedings{shalevshwartz2017icml-failures,
title = {{Failures of Gradient-Based Deep Learning}},
author = {Shalev-Shwartz, Shai and Shamir, Ohad and Shammah, Shaked},
booktitle = {International Conference on Machine Learning},
year = {2017},
pages = {3067-3075},
volume = {70},
url = {https://mlanthology.org/icml/2017/shalevshwartz2017icml-failures/}
}