Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks
Abstract
Learning deeper convolutional neural networks has become a tendency in recent years. However, many empirical evidences suggest that performance improvement cannot be attained by simply stacking more layers. In this paper, we consider the issue from an information theoretical perspective, and propose a novel method Relay Backpropagation, which encourages the propagation of effective information through the network in training stage. By virtue of the method, we achieved the first place in ILSVRC 2015 Scene Classification Challenge. Extensive experiments on two large scale challenging datasets demonstrate the effectiveness of our method is not restricted to a specific dataset or network architecture.
Cite
Text
Shen et al. "Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks." European Conference on Computer Vision, 2016. doi:10.1007/978-3-319-46478-7_29Markdown
[Shen et al. "Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks." European Conference on Computer Vision, 2016.](https://mlanthology.org/eccv/2016/shen2016eccv-relay/) doi:10.1007/978-3-319-46478-7_29BibTeX
@inproceedings{shen2016eccv-relay,
title = {{Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks}},
author = {Shen, Li and Lin, Zhouchen and Huang, Qingming},
booktitle = {European Conference on Computer Vision},
year = {2016},
pages = {467-482},
doi = {10.1007/978-3-319-46478-7_29},
url = {https://mlanthology.org/eccv/2016/shen2016eccv-relay/}
}