The Singular Values of Convolutional Layers
Abstract
We characterize the singular values of the linear transformation associated with a standard 2D multi-channel convolutional layer, enabling their efficient computation. This characterization also leads to an algorithm for projecting a convolutional layer onto an operator-norm ball. We show that this is an effective regularizer; for example, it improves the test error of a deep residual network using batch normalization on CIFAR-10 from 6.2% to 5.3%.
Cite
Text
Sedghi et al. "The Singular Values of Convolutional Layers." International Conference on Learning Representations, 2019.Markdown
[Sedghi et al. "The Singular Values of Convolutional Layers." International Conference on Learning Representations, 2019.](https://mlanthology.org/iclr/2019/sedghi2019iclr-singular/)BibTeX
@inproceedings{sedghi2019iclr-singular,
title = {{The Singular Values of Convolutional Layers}},
author = {Sedghi, Hanie and Gupta, Vineet and Long, Philip M.},
booktitle = {International Conference on Learning Representations},
year = {2019},
url = {https://mlanthology.org/iclr/2019/sedghi2019iclr-singular/}
}