Face Recognition: The Problem of Compensating for Changes in Illumination Direction

Abstract

Recognizing faces is a difficult problem due to the generally similar shape of faces combined with the considerable variability in images of the same face under different viewing conditions. In this paper we consider image variation due mainly to illumination conditions. We study several image representations that are often considered insensitive to changes of illumination conditions, such as edge maps, derivatives of the grey-level image, and the image convolved with Gabor filters. For each of these representations, we compare the differences between images of the same face under different imaging conditions, with differences between images of distinct faces. The comparison is performed using a controlled database of faces, in which each of the imaging parameters (illumination, viewing position, and expression) is controlled separately. The main result of these studies is that the variations between the images of the same face due to illumination and viewing directions are almost always larger than image variations due to a change in face identity. For illumination changes, this reversal is almost complete except for representations that emphasized the horizontal features. However, even for these representations, systems based only on comparing such representations will fail to recognize up to 30% of the faces in our database. We conclude that these representations are insufficient by themselves to overcome the variation between images due to changes in illumination direction as well as changes due to viewing position and expression.