RTTLC: Video Colorization with Restored Transformer and Test-Time Local Converter

Jinjing Li, Qirong Liang, Qipei Li, Ruipeng Gang, Ji Fang, Chi-Chen Lin, Shuang Feng, Xiaofeng Liu

CVPRW 2023 pp. 1722-1730

doi:10.1109/CVPRW59228.2023.00173 /cvprw/2023/li2023cvprw-rttlc/

Abstract

Video colorization is a highly challenging and ill-posed problem that suffers from severe flickering artifacts and color distribution inconsistency. To resolve these issues, we propose a Restored Transformer and Test-time Local Converter network(RTTLC). Firstly, we introduce a Bidirectional Recurrent Block and a Learnable Guided Mask to our network. This leverages hidden knowledge from adjacent frames that include rich information about occlusion, resulting in significant enhancements in visual quality. Secondly, we integrate a Restored Transformer that enables the network to utilize more spatial contextual information and capture multi-scale information more accurately. Thirdly, during inference, we utilize the Test-time Local Converter(TLC) strategy to alleviate distribution shift and enhance the performance of the model. Experimental results show good performance of FID and CDC. Notably, RTTLC achieves second prize in both tracks of the NTIRE23 video colorization challenges.

CVPRW Semantic Scholar

Cite

Text

Li et al. "RTTLC: Video Colorization with Restored Transformer and Test-Time Local Converter." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023. doi:10.1109/CVPRW59228.2023.00173

Markdown

[Li et al. "RTTLC: Video Colorization with Restored Transformer and Test-Time Local Converter." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023.](https://mlanthology.org/cvprw/2023/li2023cvprw-rttlc/) doi:10.1109/CVPRW59228.2023.00173

BibTeX

@inproceedings{li2023cvprw-rttlc,
  title     = {{RTTLC: Video Colorization with Restored Transformer and Test-Time Local Converter}},
  author    = {Li, Jinjing and Liang, Qirong and Li, Qipei and Gang, Ruipeng and Fang, Ji and Lin, Chi-Chen and Feng, Shuang and Liu, Xiaofeng},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2023},
  pages     = {1722-1730},
  doi       = {10.1109/CVPRW59228.2023.00173},
  url       = {https://mlanthology.org/cvprw/2023/li2023cvprw-rttlc/}
}