Exploring the Limits: Applying State-of-the-Art Stereo Matching Algorithms to Rectified Ultra-Wide Stereo

Abstract

Stereo cameras leveraging two-view geometry have predominantly focused on narrow field-of-view rectified stereo using the pinhole camera model. This research trend overlooks the complexities and potential of wide-angle stereo systems, which necessitate the use of wide-angle fisheye optics that can not be well approximated by pinhole camera model. Consequently, a lack of standardized form leads researchers to explore various strategies. Currently, a dichotomy exists between utilizing raw images directly or rectifying them. Wide-angle stereo rectification opens the potential to reuse the latest state-of-the-art (SOTA) algorithms designed for pinhole rectified stereo as a black box. However, rectification comes at the cost of severe distortions throughout the image and non-linear triangulation of 3D structure. The literature currently lacks a thorough examination of the implications of these distortions and the impact of applying the latest SOTA algorithms to stereo-rectified wide-angle images. Our work addresses this gap by conducting an exhaustive analysis of the wide-angle rectified stereo framework, delivering concrete recommendations for developing accurate wide-angle stereo systems.