Joint Face Alignment: Rescue Bad Alignments with Good Ones by Regularized Re-Fitting

Abstract

Nowadays, more and more applications need to jointly align a set of facial images from one specific person, which forms the so-called joint face alignment problem. To address this problem, in this paper, starting from an initial face alignment results, we propose to enhance the alignments by a fundamentally novel idea: rescuing the bad alignments with their well-aligned neighbors . In our method, a discriminative alignment evaluator is well designed to assess the initial face alignments and separate the well-aligned images from the badly-aligned ones. To correct the bad ones, a robust regularized re-fitting algorithm is proposed by exploiting the appearance consistency between the badly-aligned image and its k well-aligned nearest neighbors. Experiments conducted on faces in the wild demonstrate that our method greatly improves the initial face alignment results of an off-the-shelf facial landmark locator. In addition, the effectiveness of our method is validated through comparing with other state-of-the-art methods in joint face alignment under complex conditions.