M2T: Masking Transformers Twice for Faster Decoding
Abstract
We show how bidirectional transformers trained for masked token prediction can be applied to neural image compression to achieve state-of-the-art results. Such models were previously used for image_generation_ by progressive sampling groups of masked tokens according to uncertainty-adaptive schedules. Unlike these works, we demonstrate that predefined, deterministic schedules perform as well or better for image compression. This insight allows us to use masked attention during training in addition to masked inputs, and activation caching during inference, to significantly speed up our models (4x higher inference speed) at a small increase in bitrate.
Cite
Text
Mentzer et al. "M2T: Masking Transformers Twice for Faster Decoding." International Conference on Computer Vision, 2023. doi:10.1109/ICCV51070.2023.00492Markdown
[Mentzer et al. "M2T: Masking Transformers Twice for Faster Decoding." International Conference on Computer Vision, 2023.](https://mlanthology.org/iccv/2023/mentzer2023iccv-m2t/) doi:10.1109/ICCV51070.2023.00492BibTeX
@inproceedings{mentzer2023iccv-m2t,
title = {{M2T: Masking Transformers Twice for Faster Decoding}},
author = {Mentzer, Fabian and Agustson, Eirikur and Tschannen, Michael},
booktitle = {International Conference on Computer Vision},
year = {2023},
pages = {5340-5349},
doi = {10.1109/ICCV51070.2023.00492},
url = {https://mlanthology.org/iccv/2023/mentzer2023iccv-m2t/}
}