First-Order Adversarial Vulnerability of Neural Networks and Input Dimension
Abstract
Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisingly, vulnerability does not depend on network topology: for many standard network architectures, we prove that at initialization, the L1-norm of these gradients grows as the square root of the input dimension, leaving the networks increasingly vulnerable with growing image size. We empirically show that this dimension-dependence persists after either usual or robust training, but gets attenuated with higher regularization.
Cite
Text
Simon-Gabriel et al. "First-Order Adversarial Vulnerability of Neural Networks and Input Dimension." International Conference on Machine Learning, 2019.Markdown
[Simon-Gabriel et al. "First-Order Adversarial Vulnerability of Neural Networks and Input Dimension." International Conference on Machine Learning, 2019.](https://mlanthology.org/icml/2019/simongabriel2019icml-firstorder/)BibTeX
@inproceedings{simongabriel2019icml-firstorder,
title = {{First-Order Adversarial Vulnerability of Neural Networks and Input Dimension}},
author = {Simon-Gabriel, Carl-Johann and Ollivier, Yann and Bottou, Leon and Schölkopf, Bernhard and Lopez-Paz, David},
booktitle = {International Conference on Machine Learning},
year = {2019},
pages = {5809-5817},
volume = {97},
url = {https://mlanthology.org/icml/2019/simongabriel2019icml-firstorder/}
}