Robustness Tokens: Towards Adversarial Robustness of Transformers
Abstract
Recently, large pre-trained foundation models have become widely adopted by machine learning practitioners for a multitude of tasks. Given that such models are publicly available, relying on their use as backbone models for downstream tasks might result in high vulnerability to adversarial attacks crafted with the same public model. In this work, we propose Robustness Tokens, a novel approach specific to the transformer architecture that fine-tunes a few additional private tokens with low computational requirements instead of tuning model parameters as done in traditional adversarial training. We show that Robustness Tokens make Vision Transformer models significantly more robust to white-box adversarial attacks while also retaining the original downstream performances.
Cite
Text
Pulfer et al. "Robustness Tokens: Towards Adversarial Robustness of Transformers." Proceedings of the European Conference on Computer Vision (ECCV), 2024. doi:10.1007/978-3-031-73202-7_7Markdown
[Pulfer et al. "Robustness Tokens: Towards Adversarial Robustness of Transformers." Proceedings of the European Conference on Computer Vision (ECCV), 2024.](https://mlanthology.org/eccv/2024/pulfer2024eccv-robustness/) doi:10.1007/978-3-031-73202-7_7BibTeX
@inproceedings{pulfer2024eccv-robustness,
title = {{Robustness Tokens: Towards Adversarial Robustness of Transformers}},
author = {Pulfer, Brian and Belousov, Yury and Voloshynovskiy, Slava},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2024},
doi = {10.1007/978-3-031-73202-7_7},
url = {https://mlanthology.org/eccv/2024/pulfer2024eccv-robustness/}
}