Linear Discriminant Analysis: New Formulations and Overfit Analysis

AAAI 2011 pp. 417-422

doi:10.1609/AAAI.V25I1.7926 /aaai/2011/luo2011aaai-linear/

Abstract

In this paper, we will present a unified view for LDA. We will (1) emphasize that standard LDA solutions are not unique, (2) propose several new LDA formulations: St-orthonormal LDA, Sw-orthonormal LDA and orthogonal LDA which have unique solutions, and (3) show that with St-orthonormal LDA and Sw-orthonormal LDA formulations, solutions to all four major LDA objective functions are identical. Furthermore, we perform an indepth analysis to show that the LDA sometimes performs poorly due to over-fitting, i.e., it picks up PCA dimensions with small eigenvalues. From this analysis, we propose a stable LDA which uses PCA first to reduce to a small PCA subspace and do LDA in the subspace.

PDF AAAI Semantic Scholar

Cite

Text

Luo et al. "Linear Discriminant Analysis: New Formulations and Overfit Analysis." AAAI Conference on Artificial Intelligence, 2011. doi:10.1609/AAAI.V25I1.7926

Markdown

[Luo et al. "Linear Discriminant Analysis: New Formulations and Overfit Analysis." AAAI Conference on Artificial Intelligence, 2011.](https://mlanthology.org/aaai/2011/luo2011aaai-linear/) doi:10.1609/AAAI.V25I1.7926

BibTeX

@inproceedings{luo2011aaai-linear,
  title     = {{Linear Discriminant Analysis: New Formulations and Overfit Analysis}},
  author    = {Luo, Dijun and Ding, Chris H. Q. and Huang, Heng},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2011},
  pages     = {417-422},
  doi       = {10.1609/AAAI.V25I1.7926},
  url       = {https://mlanthology.org/aaai/2011/luo2011aaai-linear/}
}