Semi-Supervised 3D Face Representation Learning from Unconstrained Photo Collections

Abstract

Recovering 3D geometry shape, albedo, and lighting from a single image is a typical ill-posed problem. To address this challenging problem, we propose to utilize the joint constraints from unconstrained photo collections of one person to recover his or her identity shape and albedo. Unconstrained photo collections include one’s photos captured under different times, backgrounds, and expressions, e.g., photos posted on Instagram. We train our model in a semi-supervised manner with adversarial loss to exploit large amounts of unconstrained facial images. A novel center loss is introduced to make sure that facial images from the same subject have the same identity shape and albedo. Besides, our proposed model disentangles identity, expression, pose, and lighting representations, which improves the overall reconstruction performance and facilitates facial editing applications, e.g., expression transfer. Comprehensive experiments demonstrate that our model produces high-quality reconstruction compared to state-of-the-art methods and is robust to various expression, pose, and lighting conditions.