Benchmarking 3D Perception Robustness to Common Corruptions and Sensor Failure

Abstract

The robustness of the 3D perception system under common corruptions and sensor failure is pivotal for safety-critical applications. Existing large-scale 3D perception datasets often contain data that are meticulously cleaned. Such configurations, however, cannot reflect the reliability of perception models during the deployment stage. In this work, we contribute Robo3D, the first test suite heading toward probing the robustness of 3D detectors and segmentors under out-of-distribution scenarios against natural corruptions that occur in the real-world environment. Specifically, we consider eight corruption types (each with three severity levels) that are likely to happen under 1) adverse weather conditions, such as fog, rain, and snow; 2) external disturbances that are caused by motions or result in the missing of LiDAR beams; and 3) internal sensor failure, including crosstalk, possible incomplete echo, and cross-sensor scenarios. We reveal that, although promising results have been progressively achieved on standard benchmarks, the state-of-the-art 3D perception models are at risk of being vulnerable to data corruptions. Based on our observations, we further draw suggestions on aspects including LiDAR representation, training strategies, and augmentation. We hope this work could inspire follow-up research in designing more robust and reliable 3D perception models. Our robustness evaluation toolkit is publicly available at https://github.com/ldkong1205/Robo3D.