Is Pose Really Solved? a Frontalization Study on Off-Angle Face Matching

Abstract

Recently, impressive results have been achieved on many large-scale face recognition benchmarks, such as IJB-A Janus and Janus CS3. These datasets were designed to test robustness to nuisance transformations simultaneously such as pose, illumination, expression etc. We present a study paper, where we find that despite this goal in evaluation, there exists a significant frontal bias in yaw pose in these datasets. Therefore, high-performance on these recent datasets is misleading and does not reflect robustness to extreme pose in yaw. Moreover, many real-world applications only allow for a single frontal enrollment in a gallery (law enforcement, immigration etc.). As we show in our study, face recognition in this highly constrained setting with extreme pose variation in the probe images remains a highly challenging problem. Traditional approaches, performing well on datasets such as IJB-A Janus, perform much worse on older but highly controlled datasets such as CMU MPIE. To aid our study, we present a simple and practical method to handle pose variation in face recognition pipelines designed to deal with extremely off-angle faces. Our approach is to ignore the half of the face with any self-occlusion. This method allows our models to be highly robust to pose, and helps us achieve state-of-the-art results on several protocols using the CMU MPIE dataset as well as very accurate results on the CFP dataset, outperforming recent efforts using the same training data.