Differentiable Patch Selection for Image Recognition

Jean-Baptiste Cordonnier, Aravindh Mahendran, Alexey Dosovitskiy, Dirk Weissenborn, Jakob Uszkoreit, Thomas Unterthiner

CVPR 2021 pp. 2351-2360

doi:10.1109/CVPR46437.2021.00238 /cvpr/2021/cordonnier2021cvpr-differentiable/

Abstract

Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand. We propose a method based on a differentiable Top-K operator to select the most relevant parts of the input to efficiently process high resolution images. Our method may be interfaced with any downstream neural network, is able to aggregate information from different patches in a flexible way, and allows the whole model to be trained end-to-end using backpropagation. We show results for traffic sign recognition, inter-patch relationship reasoning, and fine-grained recognition without using object/part bounding box annotations during training.

PDF CVPR Semantic Scholar

Cite

Text

Cordonnier et al. "Differentiable Patch Selection for Image Recognition." Conference on Computer Vision and Pattern Recognition, 2021. doi:10.1109/CVPR46437.2021.00238

Markdown

[Cordonnier et al. "Differentiable Patch Selection for Image Recognition." Conference on Computer Vision and Pattern Recognition, 2021.](https://mlanthology.org/cvpr/2021/cordonnier2021cvpr-differentiable/) doi:10.1109/CVPR46437.2021.00238

BibTeX

@inproceedings{cordonnier2021cvpr-differentiable,
  title     = {{Differentiable Patch Selection for Image Recognition}},
  author    = {Cordonnier, Jean-Baptiste and Mahendran, Aravindh and Dosovitskiy, Alexey and Weissenborn, Dirk and Uszkoreit, Jakob and Unterthiner, Thomas},
  booktitle = {Conference on Computer Vision and Pattern Recognition},
  year      = {2021},
  pages     = {2351-2360},
  doi       = {10.1109/CVPR46437.2021.00238},
  url       = {https://mlanthology.org/cvpr/2021/cordonnier2021cvpr-differentiable/}
}