Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging

Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Henghui Ding, Yulun Zhang, Radu Timofte, Luc V Gool

NeurIPS 2022

/neurips/2022/cai2022neurips-degradationaware/

Abstract

In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement. Among these algorithms, deep unfolding methods demonstrate promising performance but suffer from two issues. Firstly, they do not estimate the degradation patterns and ill-posedness degree from CASSI to guide the iterative learning. Secondly, they are mainly CNN-based, showing limitations in capturing long-range dependencies. In this paper, we propose a principled Degradation-Aware Unfolding Framework (DAUF) that estimates parameters from the compressed image and physical mask, and then uses these parameters to control each iteration. Moreover, we customize a novel Half-Shuffle Transformer (HST) that simultaneously captures local contents and non-local dependencies. By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST), for HSI reconstruction. Experiments show that DAUHST surpasses state-of-the-art methods while requiring cheaper computational and memory costs. Code and models are publicly available at https://github.com/caiyuanhao1998/MST

PDF NeurIPS OpenReview Semantic Scholar

Cite

Text

Cai et al. "Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging." Neural Information Processing Systems, 2022.

Markdown

[Cai et al. "Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging." Neural Information Processing Systems, 2022.](https://mlanthology.org/neurips/2022/cai2022neurips-degradationaware/)

BibTeX

@inproceedings{cai2022neurips-degradationaware,
  title     = {{Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging}},
  author    = {Cai, Yuanhao and Lin, Jing and Wang, Haoqian and Yuan, Xin and Ding, Henghui and Zhang, Yulun and Timofte, Radu and Gool, Luc V},
  booktitle = {Neural Information Processing Systems},
  year      = {2022},
  url       = {https://mlanthology.org/neurips/2022/cai2022neurips-degradationaware/}
}