Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression
Abstract
In this paper, we analyze two popular network compression techniques, i.e. filter pruning and low-rank decomposition, in a unified sense. By simply changing the way the sparsity regularization is enforced, filter pruning and low-rank decomposition can be derived accordingly. This provides another flexible choice for network compression because the techniques complement each other. For example, in popular network architectures with shortcut connections (e.g. ResNet), filter pruning cannot deal with the last convolutional layer in a ResBlock while the low-rank decomposition methods can. In addition, we propose to compress the whole network jointly instead of in a layer-wise manner. Our approach proves its potential as it compares favorably to the state-of-the-art on several benchmarks. Code is available at https://github.com/ofsoundof/group_sparsity.
Cite
Text
Li et al. "Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. doi:10.1109/CVPR42600.2020.00804Markdown
[Li et al. "Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.](https://mlanthology.org/cvpr/2020/li2020cvpr-group/) doi:10.1109/CVPR42600.2020.00804BibTeX
@inproceedings{li2020cvpr-group,
title = {{Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression}},
author = {Li, Yawei and Gu, Shuhang and Mayer, Christoph and Van Gool, Luc and Timofte, Radu},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2020},
doi = {10.1109/CVPR42600.2020.00804},
url = {https://mlanthology.org/cvpr/2020/li2020cvpr-group/}
}