Feature Extraction via Generalized Uncorrelated Linear Discriminant Analysis

Abstract

Feature extraction is important in many applications, such as text and imageretrieval, because of high dimensionality. Uncorrelated Linear Discriminant Analysis(ULDA) was recently proposed for feature extraction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for manyapplications. In this paper, we will first propose the ULDA/QR algorithm to simplify theprevious implementation of ULDA. Then we propose the ULDA/GSVD algorithm, based on a novel optimization criterion, to address the singularity problem. It isapplicable for undersampled problem, where the data dimension is much larger than the datasize, such as text and image retrieval. The novel criterion used in ULDA/GSVD is the perturbed version of the one from ULDA/QR, while surprisingly, the solution to ULDA/GSVD is shown to be independent of the amount of perturbation applied. We did extensive experiments on text and face image data to show theeffectiveness of ULDA/GSVD and compare with other popular feature extraction algorithms.