A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis
Abstract
Fisher linear discriminant analysis (LDA) and its kernel extension—kernel discriminant analysis (KDA)—are well known methods that consider dimensionality reduction and classification jointly. While widely deployed in practical problems, there are still unresolved issues surrounding their efficient implementation and their relationship with least mean squared error procedures. In this paper we address these issues within the framework of regularized estimation. Our approach leads to a flexible and efficient implementation of LDA as well as KDA. We also uncover a general relationship between regularized discriminant analysis and ridge regression. This relationship yields variations on conventional LDA based on the pseudoinverse and a direct equivalence to an ordinary least squares estimator. Experimental results on a collection of benchmark data sets demonstrate the effectiveness of our approach.
Cite
Text
Zhang et al. "A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2009. doi:10.1007/978-3-642-04174-7_41Markdown
[Zhang et al. "A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2009.](https://mlanthology.org/ecmlpkdd/2009/zhang2009ecmlpkdd-flexible/) doi:10.1007/978-3-642-04174-7_41BibTeX
@inproceedings{zhang2009ecmlpkdd-flexible,
title = {{A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis}},
author = {Zhang, Zhihua and Dai, Guang and Jordan, Michael I.},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2009},
pages = {632-647},
doi = {10.1007/978-3-642-04174-7_41},
url = {https://mlanthology.org/ecmlpkdd/2009/zhang2009ecmlpkdd-flexible/}
}