Batch Normalization Is a Cause of Adversarial Vulnerability
Abstract
Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets. Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension. Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.
Cite
Text
Galloway et al. "Batch Normalization Is a Cause of Adversarial Vulnerability." ICML 2019 Workshops: Deep_Phenomena, 2019.Markdown
[Galloway et al. "Batch Normalization Is a Cause of Adversarial Vulnerability." ICML 2019 Workshops: Deep_Phenomena, 2019.](https://mlanthology.org/icmlw/2019/galloway2019icmlw-batch/)BibTeX
@inproceedings{galloway2019icmlw-batch,
title = {{Batch Normalization Is a Cause of Adversarial Vulnerability}},
author = {Galloway, Angus and Golubeva, Anna and Tanay, Thomas and Moussa, Medhat and Taylor, Graham W.},
booktitle = {ICML 2019 Workshops: Deep_Phenomena},
year = {2019},
url = {https://mlanthology.org/icmlw/2019/galloway2019icmlw-batch/}
}