2D-3D Heterogeneous Face Recognition Based on Deep Coupled Spectral Regression

Abstract

As one of the major branches in Face Recognition (FR), 2D-3D Heterogeneous FR (HFR), where face comparison is achieved across the texture and shape modalities, has become more important. This paper proposes a novel deep learning based end-to-end approach, namely Deep Coupled Spectral Regression (DCSR), for such an issue. It jointly makes use of both the advantages of CNN based deep features and CSR based common subspace. Specifically, from 2D texture and 3D depth face maps, DCSR extracts more powerful features by a deep network with the cross-modality triplet loss, which show much better uniqueness and robustness than the hand-crafted ones. Further, DCSR learns the shared space between different modalities with the constraints of sample labels, and is thereby more discriminative than the widely used unsupervised methods. More importantly, the two steps above are integrated through a couple layer to explicitly optimize the weights of deep features and projection directions rather than a simple combination. Experiments are carried out on the FRGC v2.0 database, and the results reported clearly demonstrate the competency of our proposed method. Its generalization ability is also validated by additional experiments conducted on the CASIA NIR-VIS 2.0 database.