Learning Step Size Controllers for Robust Neural Network Training

Christian Daniel, Jonathan Taylor, Sebastian Nowozin

AAAI 2016 pp. 1519-1525

doi:10.1609/AAAI.V30I1.10187 /aaai/2016/daniel2016aaai-learning/

Abstract

This paper investigates algorithms to automatically adapt the learning rate of neural networks (NNs). Starting with stochastic gradient descent, a large variety of learning methods has been proposed for the NN setting. However, these methods are usually sensitive to the initial learning rate which has to be chosen by the experimenter. We investigate several features and show how an adaptive controller can adjust the learning rate without prior knowledge of the learning problem at hand.

PDF AAAI Semantic Scholar

Cite

Text

Daniel et al. "Learning Step Size Controllers for Robust Neural Network Training." AAAI Conference on Artificial Intelligence, 2016. doi:10.1609/AAAI.V30I1.10187

Markdown

[Daniel et al. "Learning Step Size Controllers for Robust Neural Network Training." AAAI Conference on Artificial Intelligence, 2016.](https://mlanthology.org/aaai/2016/daniel2016aaai-learning/) doi:10.1609/AAAI.V30I1.10187

BibTeX

@inproceedings{daniel2016aaai-learning,
  title     = {{Learning Step Size Controllers for Robust Neural Network Training}},
  author    = {Daniel, Christian and Taylor, Jonathan and Nowozin, Sebastian},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2016},
  pages     = {1519-1525},
  doi       = {10.1609/AAAI.V30I1.10187},
  url       = {https://mlanthology.org/aaai/2016/daniel2016aaai-learning/}
}