Admixture of Poisson MRFs: A Topic Model with Word Dependencies
Abstract
This paper introduces a new topic model based on an admixture of Poisson Markov Random Fields (APM), which can model dependencies between words as opposed to previous independent topic models such as PLSA (Hofmann, 1999), LDA (Blei et al., 2003) or SAM (Reisinger et al., 2010). We propose a class of admixture models that generalizes previous topic models and show an equivalence between the conditional distribution of LDA and independent Poissons—suggesting that APM subsumes the modeling power of LDA. We present a tractable method for estimating the parameters of an APM based on the pseudo log-likelihood and demonstrate the benefits of APM over previous models by preliminary qualitative and quantitative experiments.
Cite
Text
Inouye et al. "Admixture of Poisson MRFs: A Topic Model with Word Dependencies." International Conference on Machine Learning, 2014.Markdown
[Inouye et al. "Admixture of Poisson MRFs: A Topic Model with Word Dependencies." International Conference on Machine Learning, 2014.](https://mlanthology.org/icml/2014/inouye2014icml-admixture/)BibTeX
@inproceedings{inouye2014icml-admixture,
title = {{Admixture of Poisson MRFs: A Topic Model with Word Dependencies}},
author = {Inouye, David and Ravikumar, Pradeep and Dhillon, Inderjit},
booktitle = {International Conference on Machine Learning},
year = {2014},
pages = {683-691},
volume = {32},
url = {https://mlanthology.org/icml/2014/inouye2014icml-admixture/}
}