A Global-to-Local Framework for Infrared and Visible Image Sequence Registration

Abstract

Based on the development of image registration, sequence registration can be done by computing the transformations between consecutive frames. To take into account the accumulated error, global registration method is usually employed as a global error minimizing approach. However, in real surveillance applications, the visible sequence and infrared sequence may be taken at different times, or from different viewpoints, and may have different dynamic contents. Therefore, global registration is only an approximate estimation for two sequences, resulting in inferior local contents. In this paper we present a novel integrated global-to-local framework that addresses the problems of dynamic infrared and visible image sequence registration. We propose to maximize the sum of the mutual information of two sequences for the global homography estimation. Then, frame-to-frame registration is performed to estimate the per-frame local homography. Finally, a smoothing strategy is adopted to smooth the local homographies in the temporal domain to enforce temporal consistency. We evaluate our proposed framework by comparing it to the state-of-the art sequence registration algorithm. Our method achieves improved performance on the public benchmark dataset.