Multi-Scale Feature Fusion Using Channel Transformers for Guided Thermal Image Super Resolution

Abstract

Thermal imaging, leveraging the infrared spectrum, offers a compelling alternative to visible spectrum (VIS) imagery in challenging environmental conditions like low- light, occlusions, and adverse weather. However, its widespread adoption in computer vision tasks is hampered by lower spatial resolution. We address this challenge by proposing a novel framework titled Multi-Scale Feature Fusion using Channel Transformers (MSFFCT) for Guided Thermal Image Super-Resolution (GTISR).GTISR tackles the resolution limitations of thermal imagery. It leverages high-resolution RGB information as a guide to reconstruct high-resolution thermal imagery from low-resolution thermal inputs. At the core of MSFFCT lies a novel deep learning architecture that combines the strengths of two powerful approaches: channel-based transformers and multi-scale fusion.MSFFCT overcomes inherent limitations of Convolutional Neural Networks (CNNs) typically used in super-resolution tasks. CNNs often suffer from restricted receptive fields, limiting their ability to capture long-range dependencies within the image. Additionally, computational cost grows significantly with larger inputs. MSFFCT addresses these shortcomings by enabling efficient processing of global information and offering superior scalability. MSFFCT achieved state-of-the-art results on the ×8 and × 16 GTISR tasks of the 2024 Perception Beyond Visual Spectrum (PBVS) challenge, winning 2nd place in both tasks and demonstrating its effectiveness in real-world scenarios.