Sanity Checks for Patch Visualisation in Prototype-Based Image Classification

Romain Xu-Darme, Georges Quénot, Zakaria Chihani, Marie-Christine Rousset

CVPRW 2023 pp. 3691-3696

doi:10.1109/CVPRW59228.2023.00377 /cvprw/2023/xudarme2023cvprw-sanity/

Abstract

In this work, we perform an analysis of the visualisation methods implemented in ProtoPNet and ProtoTree, two self-explaining visual classifiers based on prototypes. We show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour, which can create a false sense of bias in the model. We also demonstrate quantitatively that this issue can be mitigated by using other saliency methods that provide more faithful image patches.

PDF CVPRW Semantic Scholar

Cite

Text

Xu-Darme et al. "Sanity Checks for Patch Visualisation in Prototype-Based Image Classification." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023. doi:10.1109/CVPRW59228.2023.00377

Markdown

[Xu-Darme et al. "Sanity Checks for Patch Visualisation in Prototype-Based Image Classification." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023.](https://mlanthology.org/cvprw/2023/xudarme2023cvprw-sanity/) doi:10.1109/CVPRW59228.2023.00377

BibTeX

@inproceedings{xudarme2023cvprw-sanity,
  title     = {{Sanity Checks for Patch Visualisation in Prototype-Based Image Classification}},
  author    = {Xu-Darme, Romain and Quénot, Georges and Chihani, Zakaria and Rousset, Marie-Christine},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2023},
  pages     = {3691-3696},
  doi       = {10.1109/CVPRW59228.2023.00377},
  url       = {https://mlanthology.org/cvprw/2023/xudarme2023cvprw-sanity/}
}