How Chaotic Are Recurrent Neural Networks?
Abstract
Recurrent neural networks (RNNs) are non-linear dynamic systems. Previous work believes that RNN may suffer from the phenomenon of chaos, where the system is sensitive to initial states and unpredictable in the long run. In this paper, however, we perform a systematic empirical analysis, showing that a vanilla or long short term memory (LSTM) RNN does not exhibit chaotic behavior along the training process in real applications such as text generation. Our findings suggest that future work in this direction should address the other side of non-linear dynamics for RNN.
Cite
Text
Vakilipourtakalou and Mou. "How Chaotic Are Recurrent Neural Networks?." ICLR 2020 Workshops: DeepDiffEq, 2020.Markdown
[Vakilipourtakalou and Mou. "How Chaotic Are Recurrent Neural Networks?." ICLR 2020 Workshops: DeepDiffEq, 2020.](https://mlanthology.org/iclrw/2020/vakilipourtakalou2020iclrw-chaotic/)BibTeX
@inproceedings{vakilipourtakalou2020iclrw-chaotic,
title = {{How Chaotic Are Recurrent Neural Networks?}},
author = {Vakilipourtakalou, Pourya and Mou, Lili},
booktitle = {ICLR 2020 Workshops: DeepDiffEq},
year = {2020},
url = {https://mlanthology.org/iclrw/2020/vakilipourtakalou2020iclrw-chaotic/}
}