On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes
Abstract
We compare discriminative and generative learning as typified by logistic regression and naive Bayes. We show, contrary to a widely(cid:173) held belief that discriminative classifiers are almost always to be preferred, that there can often be two distinct regimes of per(cid:173) formance as the training set size is increased, one in which each algorithm does better. This stems from the observation- which is borne out in repeated experiments- that while discriminative learning has lower asymptotic error, a generative classifier may also approach its (higher) asymptotic error much faster.
Cite
Text
Ng and Jordan. "On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes." Neural Information Processing Systems, 2001.Markdown
[Ng and Jordan. "On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes." Neural Information Processing Systems, 2001.](https://mlanthology.org/neurips/2001/ng2001neurips-discriminative/)BibTeX
@inproceedings{ng2001neurips-discriminative,
title = {{On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes}},
author = {Ng, Andrew Y. and Jordan, Michael I.},
booktitle = {Neural Information Processing Systems},
year = {2001},
pages = {841-848},
url = {https://mlanthology.org/neurips/2001/ng2001neurips-discriminative/}
}