Exponential Stochastic Cellular Automata for Massively Parallel Inference
Abstract
We propose an embarrassingly parallel, memory efficient inference algorithm for latent variable models in which the complete data likelihood is in the exponential family. The algorithm is a stochastic cellular automaton and converges to a valid maximum a posteriori fixed point. Applied to latent Dirichlet allocation we find that our algorithm is over an order or magnitude faster than the fastest current approaches. A simple C++/MPI implementation on a 20-node Amazon EC2 cluster samples at more than 1 billion tokens per second. We process 3 billion documents and achieve predictive power competitive with collapsed Gibbs sampling and variational inference.
Cite
Text
Zaheer et al. "Exponential Stochastic Cellular Automata for Massively Parallel Inference." International Conference on Artificial Intelligence and Statistics, 2016.Markdown
[Zaheer et al. "Exponential Stochastic Cellular Automata for Massively Parallel Inference." International Conference on Artificial Intelligence and Statistics, 2016.](https://mlanthology.org/aistats/2016/zaheer2016aistats-exponential/)BibTeX
@inproceedings{zaheer2016aistats-exponential,
title = {{Exponential Stochastic Cellular Automata for Massively Parallel Inference}},
author = {Zaheer, Manzil and Wick, Michael L. and Tristan, Jean-Baptiste and Smola, Alexander J. and Jr., Guy L. Steele},
booktitle = {International Conference on Artificial Intelligence and Statistics},
year = {2016},
pages = {966-975},
url = {https://mlanthology.org/aistats/2016/zaheer2016aistats-exponential/}
}