A Deep Wavelet Network for High-Resolution Microscopy Hyperspectral Image Reconstruction

Abstract

Microscopy hyperspectral imaging (MHSI) integrates conventional imaging with spectroscopy to capture images through numbers of narrow spectral bands, and has attracted much attention in histopathology image analysis. However, due to the current hardware limitation, the observed microscopy hyperspectral images (MHSIs) has much lower spatial resolution than the coupled RGB images. High-resolution MHSI (HR-MHSI) reconstruction by fusing low-resolution MHSIs (LR-MHSIs) with high-resolution (HR) RGB images can compensate for the information loss. Unfortunately, radiometrical inconsistency between the two modalities often causes spectral and spatial distortions which degrade quality of the reconstructed images. To address these issues, we propose a Deep Wavelet Network (WaveNet) based on spectral grouping and feature adaptation for HR-MHSI reconstruction. For preservation of spectral information, the band-group-wise adaptation framework exploits intrinsic spectral correlation and decompose MHSI reconstruction in separate spectral groups. In spatial domain, we design the mutual adaptation block to jointly fuse the wavelet and convolution features by wavelet-attention for high-frequency information extraction. Specifically, to effectively inject spatial details from HR-RGB to LR-MHSI, radiometry-aware alignment is conducted between the two modalities. We compare WaveNet with several state-of-the-art methods by their performance on a public pathological MHSI dataset. Experimental results demonstrate its efficacy and reliability under different settings.