CDAda: A Curriculum Domain Adaptation for Nighttime Semantic Segmentation

Abstract

Autonomous driving needs to ensure all-weather safety, especially in unfavorable environments such as night and rain. However, the current daytime-trained semantic segmentation networks face significant performance degradation at night because of the huge domain divergence. In this paper, we propose a novel Curriculum Domain Adaptation method (CDAda) to realize the smooth semantic knowledge transfer from daytime to nighttime. Specifically, it consists of two steps: 1) inter-domain style adaptation: fine-tune the daytime-trained model on the labeled synthetic nighttime images through the proposed frequency-based style transformation method (replace the low-frequency components of daytime images with those of nighttime images); 2) intra-domain gradual self-training: separate the nighttime domain into the easy split nighttime domain and hard split nighttime domain based on the "entropy + illumination" ranking principle, then gradually adapt the model to the two sub-domains through pseudo supervision on easy split data and entropy minimization on hard split data. To the best of our knowledge, we first extend the idea of intra-domain adaptation to self-training and prove different treatments on two parts can reduce the distribution divergence in the nighttime domain itself. In particular, aimed at the adopted unlabeled day-night image pairs, the prediction of the daytime images can guide the segmentation on the nighttime images by ensuring patch-level consistency. Extensive experiments on Nighttime Driving, Dark Zurich, and BDD100K-night dataset highlight the effectiveness of our approach with the more favorable performance 50.9%, 45.0%, and 33.8% Mean IoU against existing state-of-the-art approaches.