3D Morphable Models as Spatial Transformer Networks

Anil Bas, Patrik Huber, William A. P. Smith, Muhammad Awais, Josef Kittler

ICCVW 2017 pp. 895-903

doi:10.1109/ICCVW.2017.110 /iccvw/2017/bas2017iccvw-3d/

Abstract

In this paper, we show how a 3D Morphable Model (i.e. a statistical model of the 3D shape of a class of objects such as faces) can be used to spatially transform input data as a module (a 3DMM-STN) within a convolutional neural network. This is an extension of the original spatial transformer network in that we are able to interpret and normalise 3D pose changes and self-occlusions. The trained localisation part of the network is independently useful since it learns to fit a 3D morphable model to a single image. We show that the localiser can be trained using only simple geometric loss functions on a relatively small dataset yet is able to perform robust normalisation on highly uncontrolled images including occlusion, self-occlusion and large pose changes.

PDF ICCVW Semantic Scholar

Cite

Text

Bas et al. "3D Morphable Models as Spatial Transformer Networks." IEEE/CVF International Conference on Computer Vision Workshops, 2017. doi:10.1109/ICCVW.2017.110

Markdown

[Bas et al. "3D Morphable Models as Spatial Transformer Networks." IEEE/CVF International Conference on Computer Vision Workshops, 2017.](https://mlanthology.org/iccvw/2017/bas2017iccvw-3d/) doi:10.1109/ICCVW.2017.110

BibTeX

@inproceedings{bas2017iccvw-3d,
  title     = {{3D Morphable Models as Spatial Transformer Networks}},
  author    = {Bas, Anil and Huber, Patrik and Smith, William A. P. and Awais, Muhammad and Kittler, Josef},
  booktitle = {IEEE/CVF International Conference on Computer Vision Workshops},
  year      = {2017},
  pages     = {895-903},
  doi       = {10.1109/ICCVW.2017.110},
  url       = {https://mlanthology.org/iccvw/2017/bas2017iccvw-3d/}
}