Fully Automatic Video Colorization with Self-Regularization and Diversity
Abstract
We present a fully automatic approach to video colorization with self-regularization and diversity. Our model contains a colorization network for video frame colorization and a refinement network for spatiotemporal color refinement. Without any labeled data, both networks can be trained with self-regularized losses defined in bilateral and temporal space. The bilateral loss enforces color consistency between neighboring pixels in a bilateral space and the temporal loss imposes constraints between corresponding pixels in two nearby frames. While video colorization is a multi-modal problem, our method uses a perceptual loss with diversity to differentiate various modes in the solution space. Perceptual experiments demonstrate that our approach outperforms state-of-the-art approaches on fully automatic video colorization.
Cite
Text
Lei and Chen. "Fully Automatic Video Colorization with Self-Regularization and Diversity." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. doi:10.1109/CVPR.2019.00387Markdown
[Lei and Chen. "Fully Automatic Video Colorization with Self-Regularization and Diversity." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.](https://mlanthology.org/cvpr/2019/lei2019cvpr-fully/) doi:10.1109/CVPR.2019.00387BibTeX
@inproceedings{lei2019cvpr-fully,
title = {{Fully Automatic Video Colorization with Self-Regularization and Diversity}},
author = {Lei, Chenyang and Chen, Qifeng},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2019},
doi = {10.1109/CVPR.2019.00387},
url = {https://mlanthology.org/cvpr/2019/lei2019cvpr-fully/}
}