Multilinear Projection for Appearance-Based Recognition in the Tensor Framework

Abstract

Numerical multilinear (tensor) algebra is a principled mathematical approach to disentangling and explicitly and parsimoniously representing the essential factors or modes of image formation, among them illumination, scene geometry, and imaging, thereby dramatically improving the performance of appearance-based recognition. Generalizing concepts from linear (matrix) algebra, we define the identity tensor and the pseudo-inverse tensor and we employ them to develop a multilinear projection algorithm, which is natural for performing recognition in the tensor algebraic framework. Our multilinear projection algorithm simultaneously projects an unlabeled test image into multiple constituent mode spaces spanned by learned, mode-specific basis sets in order to infer its mode labels. Multilinear projection is applied to unconstrained facial image recognition, where the mode labels are person identity, viewpoint, illumination, etc.