Depth Completion Using Laplacian Pyramid-Based Depth Residuals

Abstract

In this paper, we propose a robust and efficient depth completion network based on residuals. Unlike previous methods that directly predict a depth residual, we reconstruct high-frequency information in complex scenes by exploiting the efficiency of the Laplacian pyramid in representing multi-scale content. Specifically, the framework can be divided into two stages: sparse-to-coarse and coarse-to-fine. In the sparse-to-coarse stage, we only recover depth from the sparse depth map without using any additional color image, and downsample the result to filter out unreliable high-frequency information from the sparse depth measurement. In the coarse-to-fine stage, we use features extracted from both data modalities to model high-frequency components as a series of multi-scale depth residuals via a Laplacian pyramid. Considering the wide distribution of high-frequency information in the frequency domain, we propose a Global-Local Refinement Network (GLRN) to estimate depth residuals separately at each scale. Furthermore, to compensate for the structural information lost by coarse depth map downsampling and further optimize the results with the color image, we propose a novel and efficient Affinity decay spatial propagation network (AD-SPN), which is used to refine the depth estimation results at each scale. Extensive experiments on indoor and outdoor datasets demonstrate that our approach achieves state-of-the-art performance.