Unsupervised Training for 3D Morphable Model Regression
Abstract
We present a method for training a regression network from image pixels to 3D morphable model coordinates using only unlabeled photographs. The training loss is based on features from a facial recognition network, computed on-the-fly by rendering the predicted faces with a differentiable renderer. To make training from features feasible and avoid network fooling effects, we introduce three objectives: a batch distribution loss that encourages the output distribution to match the distribution of the morphable model, a loopback loss that ensures the network can correctly reinterpret its own output, and a multi-view identity loss that compares the features of the predicted 3D face and the input photograph from multiple viewing angles. We train a regression network using these objectives, a set of unlabeled photographs, and the morphable model itself, and demonstrate state-of-the-art results.
Cite
Text
Genova et al. "Unsupervised Training for 3D Morphable Model Regression." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. doi:10.1109/CVPR.2018.00874Markdown
[Genova et al. "Unsupervised Training for 3D Morphable Model Regression." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.](https://mlanthology.org/cvpr/2018/genova2018cvpr-unsupervised/) doi:10.1109/CVPR.2018.00874BibTeX
@inproceedings{genova2018cvpr-unsupervised,
title = {{Unsupervised Training for 3D Morphable Model Regression}},
author = {Genova, Kyle and Cole, Forrester and Maschinot, Aaron and Sarna, Aaron and Vlasic, Daniel and Freeman, William T.},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2018},
doi = {10.1109/CVPR.2018.00874},
url = {https://mlanthology.org/cvpr/2018/genova2018cvpr-unsupervised/}
}