Are Deep Learning Models Pre-Trained on RGB Data Good Enough for RGB-Thermal Image Retrieval?

Abstract

RGB-Thermal (RGB-T) image retrieval is crucial in scenarios where RGB data alone is insufficient for reliable decision-making. These include all-day, all-weather surveillance and security operations, search and rescue operations and autonomous navigation systems. However, RGB-T image retrieval remains underexplored due to the nature of the currently available datasets. Specifically, these datasets do not lend themselves to training models in the standard RGB visual place recognition (VPR) setting. Therefore, we explore and analyse the effectiveness of existing RGB pre-trained models in addressing the RGB-T image retrieval problem. In particular, we evaluate the performance of numerous pre-trained models on the RGB-T image retrieval task. The efficacy of the models is evaluated on eight RGB-T datasets. Quantitatively, recall rates, Central Kernel Alignment (CKA), and the proposed centroid condition are used for evaluation. Qualitative analysis uses distance plots, t-SNE plots and heatmaps like Saliency Based Similarity Maps (SBSM). Interestingly, and surprisingly, some of the pre-trained models deliver good crossdomain retrieval performance. To the best of our knowledge, this analysis is the first of its kind in RGB-T image retrieval with the available RGB-T datasets. We believe this will serve as a baseline for future work in this area of research.