Thanks for Nothing: Predicting Zero-Valued Activations with Lightweight Convolutional Neural Networks
Abstract
Convolutional neural networks (CNNs) introduce state-of-the-art results for various tasks with the price of high computational demands. Inspired by the observation that spatial correlation exists in CNN output feature maps (ofms), we propose a method to dynamically predict whether ofm activations are zero-valued or not according to their neighboring activation values, thereby avoiding zero-valued activations and reducing the number of convolution operations. We implement the zero activation predictor (ZAP) with a lightweight CNN, which imposes negligible overheads and is easy to deploy on existing models. ZAPs are trained by mimicking hidden layer ouputs; thereby, enabling a parallel and label-free training. Furthermore, without retraining, each ZAP can be tuned to a different operating point trading accuracy for MAC reduction.
Cite
Text
Shomron et al. "Thanks for Nothing: Predicting Zero-Valued Activations with Lightweight Convolutional Neural Networks." Proceedings of the European Conference on Computer Vision (ECCV), 2020. doi:10.1007/978-3-030-58607-2_14Markdown
[Shomron et al. "Thanks for Nothing: Predicting Zero-Valued Activations with Lightweight Convolutional Neural Networks." Proceedings of the European Conference on Computer Vision (ECCV), 2020.](https://mlanthology.org/eccv/2020/shomron2020eccv-thanks/) doi:10.1007/978-3-030-58607-2_14BibTeX
@inproceedings{shomron2020eccv-thanks,
title = {{Thanks for Nothing: Predicting Zero-Valued Activations with Lightweight Convolutional Neural Networks}},
author = {Shomron, Gil and Banner, Ron and Shkolnik, Moran and Weiser, Uri},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2020},
doi = {10.1007/978-3-030-58607-2_14},
url = {https://mlanthology.org/eccv/2020/shomron2020eccv-thanks/}
}