A Better Way to Pretrain Deep Boltzmann Machines
Abstract
We describe how the pre-training algorithm for Deep Boltzmann Machines (DBMs) is related to the pre-training algorithm for Deep Belief Networks and we show that under certain conditions, the pre-training procedure improves the variational lower bound of a two-hidden-layer DBM. Based on this analysis, we develop a different method of pre-training DBMs that distributes the modelling work more evenly over the hidden layers. Our results on the MNIST and NORB datasets demonstrate that the new pre-training algorithm allows us to learn better generative models.
Cite
Text
Hinton and Salakhutdinov. "A Better Way to Pretrain Deep Boltzmann Machines." Neural Information Processing Systems, 2012.Markdown
[Hinton and Salakhutdinov. "A Better Way to Pretrain Deep Boltzmann Machines." Neural Information Processing Systems, 2012.](https://mlanthology.org/neurips/2012/hinton2012neurips-better/)BibTeX
@inproceedings{hinton2012neurips-better,
title = {{A Better Way to Pretrain Deep Boltzmann Machines}},
author = {Hinton, Geoffrey E. and Salakhutdinov, Ruslan},
booktitle = {Neural Information Processing Systems},
year = {2012},
pages = {2447-2455},
url = {https://mlanthology.org/neurips/2012/hinton2012neurips-better/}
}