Feature Matching in the Dark: Homography-Based RGB-IR Feature Transformation for Low-Light Vision

Abstract

This paper presents a new approach that leverages the complementary information from RGB and infrared (IR) images to create a unified feature set in the RGB image space, enhancing the performance of downstream deep learning tasks in low light conditions. We first utilize the MINIMA framework for cross-modal feature matching, generating a homography matrix to transform features from the IR to the RGB image space. We then leverage this unified set to develop a downstream application: a dual-stream deep learning network for accurate surface normal estimation that remains consistent in low-light conditions. Our experiments demonstrate a significant increase in usable image features in both standard and challenging lighting conditions in indoor and outdoor scenes. Additionally, our dual stream model outperforms a state-of-the-art RGB-only surface normal prediction model at low light levels. This proposed system maintains real-time performance while providing a new technique for improved understanding of low-light scenes.