Context-Consistent Stereo Matching

Abstract

Although our two eyes view the world from different perspectives, our brain can effortlessly associate items seen by one eye with those by the other - leading to the binocular depth sensation. We would like computers to make this match as well as humans, so that an intelligent system can perceive 3-D using binocular inputs from cameras. Current methods still have difficulty when two cameras are widely separated; more so when the images are of poor quality or contain repetitive patterns, because they can no longer distinguish features by examining only the local patches they occupy. Here we propose to improve the feature-matching by further involving global image information. We proposed a topological graph, called the Salient Feature Graph (SFG), to describe intrinsic structure of a scene based on its image. We then used the SFG to compare semi-local structures extracted from different perspectives. This semi-local comparison enabled our new algorithm, Context-Consistent Assignment (or CCA), to establish feature correspondences by dynamically involving both local appearance and global structures. We ran our algorithm and conventional methods on images of 3D urban scenes and counted how many correct matches they made. Our approach consistently outperformed competitors on difficult images such as low resolution inputs and noisy images.