3D Face Shape Regression from 2D Videos with Multi-Reconstruction and Mesh Retrieval

Abstract

This paper introduces our submission to the 2nd 3DFAW Challenge. To get a high-accuracy 3D dense face shape based on 2D videos or multiple images, a framework which is consist of multi-reconstruction branches and a mesh retrieval module, is proposed to effectively utilize the information of all frames and the results predicted by all branches. The recent state-of-the-art methods based on single-view and multi-view are introduced to form an ensemble of independent regression networks. The candidate 3D shape of each branch is synthesized by weighted linear combination of the results on all frames to boost the depth estimation and invisible regions reconstruction. Finally, the best fitting mesh is retrieved according to the distance between the synthesized texture and the ground truth texture. Experiment results show that our approach obtains competitive results near the accuracy of "pseudo" ground truths, and achieves superior performance over most of submissions by other teams in the testing phases.