Depth-Based Patch Scaling for Content-Aware Stereo Image Completion

Abstract

A number of recent algorithms have been proposed for working with stereo image pairs in ways that are already familiar to users of single-image editing tools. In particular, Morse, et al. (2012) have proposed a method for performing image completion in stereo images so as to maintain stereoscopic consistency. Like prior work in stereo completion, this method drew source texture only from regions at the same depth as the target region, which while helping the result can sometimes overly limit the pool of suitable source textures. Other methods such as the Generalized PatchMatch approach of Barnes, et al. (2010) have used scaled (and otherwise transformed) source texture to improve the quality of the completed target region, but these methods rely on randomly sampling the scale (or transformation) space without knowledge of scene geometry. This paper extends stereo image completion to include source textures scaled according to the relative differences in depth between image regions. Limited random sampling is used to make the method robust to minor errors in the stereo disparities and to provide for non-uniform aspect ratios, but with far fewer random samples than prior unrestrained sampling of scale. A preference for unscaled or downsampled source textures rather than upsampled ones is incorporated into the objective function and avoids an inherent matching bias towards low-frequency regions. Results demonstrate that using scene geometry to drive scale selection results in improved image completion compared to either single-image completion or prior methods for stereo completion.