Convergence of the Wake-Sleep Algorithm
Abstract
The W-S (Wake-Sleep) algorithm is a simple learning rule for the models with hidden variables. It is shown that this algorithm can be applied to a factor analysis model which is a linear version of the Helmholtz ma(cid:173) chine. But even for a factor analysis model, the general convergence is not proved theoretically. In this article, we describe the geometrical un(cid:173) derstanding of the W-S algorithm in contrast with the EM (Expectation(cid:173) Maximization) algorithm and the em algorithm. As the result, we prove the convergence of the W-S algorithm for the factor analysis model. We also show the condition for the convergence in general models.
Cite
Text
Ikeda et al. "Convergence of the Wake-Sleep Algorithm." Neural Information Processing Systems, 1998.Markdown
[Ikeda et al. "Convergence of the Wake-Sleep Algorithm." Neural Information Processing Systems, 1998.](https://mlanthology.org/neurips/1998/ikeda1998neurips-convergence/)BibTeX
@inproceedings{ikeda1998neurips-convergence,
title = {{Convergence of the Wake-Sleep Algorithm}},
author = {Ikeda, Shiro and Amari, Shun-ichi and Nakahara, Hiroyuki},
booktitle = {Neural Information Processing Systems},
year = {1998},
pages = {239-245},
url = {https://mlanthology.org/neurips/1998/ikeda1998neurips-convergence/}
}