Deterministic Binary Filters for Convolutional Neural Networks
Abstract
We propose Deterministic Binary Filters, an approach to Convolutional Neural Networks that learns weighting coefficients of predefined orthogonal binary basis instead of the conventional approach of learning directly the convolutional filters. This approach results in model architectures with significantly fewer parameters (4x to 16x) and smaller model sizes (32x due to the use of binary rather than floating point precision). We show our deterministic filter design can be integrated into well-known network architectures (such as ResNet and SqueezeNet) with as little as 2% loss of accuracy (under datasets like CIFAR-10). Under ImageNet, they result in 3x model size reduction compared to sub-megabyte binary networks while reaching comparable accuracy levels.
Cite
Text
Tseng et al. "Deterministic Binary Filters for Convolutional Neural Networks." International Joint Conference on Artificial Intelligence, 2018. doi:10.24963/IJCAI.2018/380Markdown
[Tseng et al. "Deterministic Binary Filters for Convolutional Neural Networks." International Joint Conference on Artificial Intelligence, 2018.](https://mlanthology.org/ijcai/2018/tseng2018ijcai-deterministic/) doi:10.24963/IJCAI.2018/380BibTeX
@inproceedings{tseng2018ijcai-deterministic,
title = {{Deterministic Binary Filters for Convolutional Neural Networks}},
author = {Tseng, Vincent W. S. and Bhattacharya, Sourav and Fernández-Marqués, Javier and Alizadeh, Milad and Tong, Catherine and Lane, Nicholas D.},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2018},
pages = {2739-2747},
doi = {10.24963/IJCAI.2018/380},
url = {https://mlanthology.org/ijcai/2018/tseng2018ijcai-deterministic/}
}