Nonlinear Discriminant Analysis Using Kernel Functions

Abstract

Fishers linear discriminant analysis (LDA) is a classical multivari(cid:173) ate technique both for dimension reduction and classification. The data vectors are transformed into a low dimensional subspace such that the class centroids are spread out as much as possible. In this subspace LDA works as a simple prototype classifier with lin(cid:173) ear decision boundaries. However, in many applications the linear boundaries do not adequately separate the classes. We present a nonlinear generalization of discriminant analysis that uses the ker(cid:173) nel trick of representing dot products by kernel functions. The pre(cid:173) sented algorithm allows a simple formulation of the EM-algorithm in terms of kernel functions which leads to a unique concept for un(cid:173) supervised mixture analysis, supervised discriminant analysis and semi-supervised discriminant analysis with partially unlabelled ob(cid:173) servations in feature spaces.