Adversarially Robust Panoptic Segmentation (ARPaS) Benchmark

Abstract

We propose the Adversarially Robust Panoptic Segmentation (ARPaS) benchmark to assess the general robustness of panoptic segmentation techniques. To account for the differences between instance and semantic segmentation, we propose to treat each segment as an independent target to optimise pixel-level adversaries. Additionally, we include common corruptions to quantify the effect of naturally occurring image perturbations in this task. We deploy the ARPaS benchmark to evaluate the robustness of state-of-the-art representatives from families of panoptic segmentation methods on standard datasets, showing their fragility in the face of attacks. To gain further insights into the effects of attacking the models, we introduce a diagnostic tool to decompose the error analysis. Finally, we empirically demonstrate that a baseline adversarial training strategy can significantly improve the robustness of these methods.