SpectralLeader: Online Spectral Learning for Single Topic Models
Abstract
We study the problem of learning a latent variable model online from a stream of data. Latent variable models are popular because they can explain observed data through unobserved concepts. These models have traditionally been studied in the offline setting. In the online setting, online expectation maximization (EM) is arguably the most popular approach for learning latent variable models. Although online EM is computationally efficient, it typically converges to a local optimum. In this work, we develop a new online learning algorithm for latent variable models, which we call $\mathtt{SpectralLeader}$ . $\mathtt{SpectralLeader}$ converges to the global optimum, and we derive a sublinear upper bound on its n -step regret in a single topic model. In both synthetic and real-world experiments, we show that $\mathtt{SpectralLeader}$ performs similarly to or better than online EM with tuned hyper-parameters.
Cite
Text
Yu et al. "SpectralLeader: Online Spectral Learning for Single Topic Models." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2018. doi:10.1007/978-3-030-10928-8_23Markdown
[Yu et al. "SpectralLeader: Online Spectral Learning for Single Topic Models." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2018.](https://mlanthology.org/ecmlpkdd/2018/yu2018ecmlpkdd-spectralleader/) doi:10.1007/978-3-030-10928-8_23BibTeX
@inproceedings{yu2018ecmlpkdd-spectralleader,
title = {{SpectralLeader: Online Spectral Learning for Single Topic Models}},
author = {Yu, Tong and Kveton, Branislav and Wen, Zheng and Bui, Hung and Mengshoel, Ole J.},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2018},
pages = {379-395},
doi = {10.1007/978-3-030-10928-8_23},
url = {https://mlanthology.org/ecmlpkdd/2018/yu2018ecmlpkdd-spectralleader/}
}