Learning Continuous Latent Variable Models with Bregman Divergences
Abstract
We present a class of unsupervised statistical learning algorithms that are formulated in terms of minimizing Bregman divergences— a family of generalized entropy measures defined by convex functions. We obtain novel training algorithms that extract hidden latent structure by minimizing a Bregman divergence on training data, subject to a set of non-linear constraints which consider hidden variables. An alternating minimization procedure with nested iterative scaling is proposed to find feasible solutions for the resulting constrained optimization problem. The convergence of this algorithm along with its information geometric properties are characterized. Index Terms — statistical machine learning, unsupervised learning, Bregman divergence, information geometry, alternating minimization, forward projection, backward projection, iterative scaling.
Cite
Text
Wang and Schuurmans. "Learning Continuous Latent Variable Models with Bregman Divergences." International Conference on Algorithmic Learning Theory, 2003. doi:10.1007/978-3-540-39624-6_16Markdown
[Wang and Schuurmans. "Learning Continuous Latent Variable Models with Bregman Divergences." International Conference on Algorithmic Learning Theory, 2003.](https://mlanthology.org/alt/2003/wang2003alt-learning/) doi:10.1007/978-3-540-39624-6_16BibTeX
@inproceedings{wang2003alt-learning,
title = {{Learning Continuous Latent Variable Models with Bregman Divergences}},
author = {Wang, Shaojun and Schuurmans, Dale},
booktitle = {International Conference on Algorithmic Learning Theory},
year = {2003},
pages = {190-204},
doi = {10.1007/978-3-540-39624-6_16},
url = {https://mlanthology.org/alt/2003/wang2003alt-learning/}
}