Are Skip Connections Necessary for Biologically Plausible Learning Rules?
Abstract
Recognizing that backpropagation has been the workhorse of deep learning, it is time to explore other alternative learning methods. Several biologically motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. However, none of these methods have produced competitive performance against backpropagation. In this paper, we show that biologically motivated learning rules with skip connections between intermediate layers can perform as well as backpropagation on the MNIST dataset and is robust to various sets of hyper-parameters.
Cite
Text
Im et al. "Are Skip Connections Necessary for Biologically Plausible Learning Rules?." NeurIPS 2019 Workshops: Neuro_AI, 2019.Markdown
[Im et al. "Are Skip Connections Necessary for Biologically Plausible Learning Rules?." NeurIPS 2019 Workshops: Neuro_AI, 2019.](https://mlanthology.org/neuripsw/2019/im2019neuripsw-skip/)BibTeX
@inproceedings{im2019neuripsw-skip,
title = {{Are Skip Connections Necessary for Biologically Plausible Learning Rules?}},
author = {Im, Daniel Jiwoong and Patil, Rutuja and Branson, Kristin},
booktitle = {NeurIPS 2019 Workshops: Neuro_AI},
year = {2019},
url = {https://mlanthology.org/neuripsw/2019/im2019neuripsw-skip/}
}