The Dormant Neuron Phenomenon in Deep Reinforcement Learning
Abstract
In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agent’s network suffers from an increasing number of inactive neurons, thereby affecting network expressivity. We demonstrate the presence of this phenomenon across a variety of algorithms and environments, and highlight its effect on learning. To address this issue, we propose a simple and effective method (ReDo) that Recycles Dormant neurons throughout training. Our experiments demonstrate that ReDo maintains the expressive power of networks by reducing the number of dormant neurons and results in improved performance.
Cite
Text
Sokar et al. "The Dormant Neuron Phenomenon in Deep Reinforcement Learning." International Conference on Machine Learning, 2023.Markdown
[Sokar et al. "The Dormant Neuron Phenomenon in Deep Reinforcement Learning." International Conference on Machine Learning, 2023.](https://mlanthology.org/icml/2023/sokar2023icml-dormant/)BibTeX
@inproceedings{sokar2023icml-dormant,
title = {{The Dormant Neuron Phenomenon in Deep Reinforcement Learning}},
author = {Sokar, Ghada and Agarwal, Rishabh and Castro, Pablo Samuel and Evci, Utku},
booktitle = {International Conference on Machine Learning},
year = {2023},
pages = {32145-32168},
volume = {202},
url = {https://mlanthology.org/icml/2023/sokar2023icml-dormant/}
}