Flexible Window-Based Self-Attention Transformer in Thermal Image Super-Resolution

Abstract

The aim of this paper is to improve the resolution of low-quality thermal images obtained from downsampled images afflicted with noise and blur, alongside high-resolution visible images, to achieve high-resolution thermal imagery. Our proposed method, named Flexible Window-based Self-attention Transformer (FW-SAT), operates across global, regional, and local scales to effectively enhance the fine details in the thermal domain. FW-SAT integrates various attention mechanisms such as channel and spatial attention, window-based self-attention, and flexible window-based self-attention. Notably, flexible window-based self-attention aggregates regional window features based on window-based self-attention, while channel and spatial attention mechanisms capture global information. Additionally, window-based self-attention is employed to explore local features within the image. We assess the performance of FW-SAT in the PBVS-2024 Thermal Image Super-Resolution Challenge (GTISR) - Track2. Our extensive experiments demonstrate that our proposed approach surpasses state-of-the-art techniques in both qualitative and quantitative evaluations. Code will be available at https://github.com/jianghongcheng/FW-SAT.