ML Anthology

An Information Theoretic Analysis of Maximum Likelihood Mixture Estimation for Exponential Families

Arindam Banerjee, Inderjit S. Dhillon, Joydeep Ghosh, Srujana Merugu
ICML 2004
doi:10.1145/1015330.1015431 /icml/2004/banerjee2004icml-information/

Abstract

An important task in unsupervised learning is maximum likelihood mixture estimation (MLME) for exponential families. In this paper, we prove a mathematical equivalence between this MLME problem and the rate distortion problem for Bregman divergences. We also present new theoretical results in rate distortion theory for Bregman divergences. Further, an analysis of the problems as a trade-off between compression and preservation of information is presented that yields the information bottleneck method as an interesting special case.

PDF ICML Semantic Scholar

Cite

Text

Banerjee et al. "An Information Theoretic Analysis of Maximum Likelihood Mixture Estimation for Exponential Families." International Conference on Machine Learning, 2004. doi:10.1145/1015330.1015431

Markdown

[Banerjee et al. "An Information Theoretic Analysis of Maximum Likelihood Mixture Estimation for Exponential Families." International Conference on Machine Learning, 2004.](https://mlanthology.org/icml/2004/banerjee2004icml-information/) doi:10.1145/1015330.1015431

BibTeX

@inproceedings{banerjee2004icml-information,
  title     = {{An Information Theoretic Analysis of Maximum Likelihood Mixture Estimation for Exponential Families}},
  author    = {Banerjee, Arindam and Dhillon, Inderjit S. and Ghosh, Joydeep and Merugu, Srujana},
  booktitle = {International Conference on Machine Learning},
  year      = {2004},
  doi       = {10.1145/1015330.1015431},
  url       = {https://mlanthology.org/icml/2004/banerjee2004icml-information/}
}