SI-Score: An Image Dataset for Fine-Grained Analysis of Robustness to Object Location, Rotation and Size
Abstract
Before deploying machine learning models it is critical to assess their robustness. In the context of deep neural networks for image understanding, changing the object location, rotation and size may affect the predictions in non-trivial ways. In this work we perform a fine-grained analysis of robustness with respect to these factors of variation using SI-Score, a synthetic dataset. In particular, we investigate ResNets, Vision Transformers and CLIP, and identify interesting qualitative differences between these.
Cite
Text
Yung et al. "SI-Score: An Image Dataset for Fine-Grained Analysis of Robustness to Object Location, Rotation and Size." ICML 2022 Workshops: Shift_Happens, 2022.Markdown
[Yung et al. "SI-Score: An Image Dataset for Fine-Grained Analysis of Robustness to Object Location, Rotation and Size." ICML 2022 Workshops: Shift_Happens, 2022.](https://mlanthology.org/icmlw/2022/yung2022icmlw-siscore/)BibTeX
@inproceedings{yung2022icmlw-siscore,
title = {{SI-Score: An Image Dataset for Fine-Grained Analysis of Robustness to Object Location, Rotation and Size}},
author = {Yung, Jessica and Romijnders, Rob and Kolesnikov, Alexander and Beyer, Lucas and Djolonga, Josip and Houlsby, Neil and Gelly, Sylvain and Lucic, Mario and Zhai, Xiaohua},
booktitle = {ICML 2022 Workshops: Shift_Happens},
year = {2022},
url = {https://mlanthology.org/icmlw/2022/yung2022icmlw-siscore/}
}