Semi-Stereo: A Universal Stereo Matching Framework for Imperfect Data via Semi-Supervised Learning

Abstract

Data matters in deep-learning-based binocular stereo matching. Obtaining a perfect dataset for stereo matching is hard and thus imperfect data is common in existing benchmark datasets, such as KITTI, ETH3D and Middlebury. The imperfectness typically has two forms: sparse-labeled data or even unlabeled data. Current stereo matching networks ignore the supervision from these imperfect data itself, even the semi-supervised networks often suffer from confirmation bias in the predictions. Besides, current methods lack a unified solution to utilize the supervision signal from those imperfect data. To mitigate this research gap, we propose Semi-Stereo, the first unified stereo matching framework empowered by the teacher-student paradigm where the teacher and the student networks are trained in a mutual-beneficial manner. To explore the rich knowledge in imperfect data, we propose a consistency regularization module with weak-strong augmentation strategies. Further, in order for the teacher to provide more reliable pseudo labels, we design a confidence module, powered by left-right consistency (LRC) check and disparity distribution entropy (DDE). Extensive experiments demonstrate Semi-Stereo produces accurate and consistent predictions in untrained semantic regions and improves the performance of baseline networks in multiple tasks, including domain adaptation and domain generalization.