Illumination-Invariant Face Recognition with Deep Relit Face Images

Abstract

Uncontrolled illumination is one of the most significant challenges in face recognition. The performance of state-of-the-art face recognition algorithms drops drastically when measured on datasets with large illumination variations. In this paper, we propose a deep face relighting algorithm and employ it as a data augmentation method to enrich training data with illumination variations. For an input image, the proposed face relighting as data augmentation (FRADA) approach first estimates its 3D morphable model coefficients and spherical harmonic lighting coefficients. Then, it extracts the face normals, face mask, face shading, and face albedo, and renders new face images under random lighting conditions following physically-based image formation theory. Qualitative results demonstrate that FRADA produces more realistic images than the state-of-the-art face relighting algorithm. Quantitative experiments confirm the effectiveness of our relighting approach for face recognition. We successfully enhance the robustness of face templates to illumination variations simply by training face recognition algorithms with our relit images.