From Monte Carlo to Las Vegas: Improving Restricted Boltzmann Machine Training Through Stopping Sets

Pedro H. P. Savarese, Mayank Kakodkar, Bruno Ribeiro

AAAI 2018 pp. 4016-4025

doi:10.1609/AAAI.V32I1.11821 /aaai/2018/savarese2018aaai-monte/

Abstract

We propose a Las Vegas transformation of Markov Chain Monte Carlo (MCMC) estimators of Restricted Boltzmann Machines (RBMs). We denote our approach Markov Chain Las Vegas (MCLV). MCLV gives statistical guarantees in exchange for random running times. MCLV uses a stopping set built from the training data and has maximum number of Markov chain steps K (referred as MCLV-K). We present a MCLV-K gradient estimator (LVS-K) for RBMs and explore the correspondence and differences between LVS-K and Contrastive Divergence (CD-K), with LVS-K significantly outperforming CD-K training RBMs over the MNIST dataset, indicating MCLV to be a promising direction in learning generative models.

PDF AAAI Semantic Scholar

Cite

Text

Savarese et al. "From Monte Carlo to Las Vegas: Improving Restricted Boltzmann Machine Training Through Stopping Sets." AAAI Conference on Artificial Intelligence, 2018. doi:10.1609/AAAI.V32I1.11821

Markdown

[Savarese et al. "From Monte Carlo to Las Vegas: Improving Restricted Boltzmann Machine Training Through Stopping Sets." AAAI Conference on Artificial Intelligence, 2018.](https://mlanthology.org/aaai/2018/savarese2018aaai-monte/) doi:10.1609/AAAI.V32I1.11821

BibTeX

@inproceedings{savarese2018aaai-monte,
  title     = {{From Monte Carlo to Las Vegas: Improving Restricted Boltzmann Machine Training Through Stopping Sets}},
  author    = {Savarese, Pedro H. P. and Kakodkar, Mayank and Ribeiro, Bruno},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2018},
  pages     = {4016-4025},
  doi       = {10.1609/AAAI.V32I1.11821},
  url       = {https://mlanthology.org/aaai/2018/savarese2018aaai-monte/}
}