Mixture Density Estimation
Abstract
Gaussian mixtures (or so-called radial basis function networks) for density estimation provide a natural counterpart to sigmoidal neu(cid:173) ral networks for function fitting and approximation. In both cases, it is possible to give simple expressions for the iterative improve(cid:173) ment of performance as components of the network are introduced one at a time. In particular, for mixture density estimation we show that a k-component mixture estimated by maximum likelihood (or by an iterative likelihood improvement that we introduce) achieves log-likelihood within order 1/k of the log-likelihood achievable by any convex combination. Consequences for approximation and es(cid:173) timation using Kullback-Leibler risk are also given. A Minimum Description Length principle selects the optimal number of compo(cid:173) nents k that minimizes the risk bound.
Cite
Text
Li and Barron. "Mixture Density Estimation." Neural Information Processing Systems, 1999.Markdown
[Li and Barron. "Mixture Density Estimation." Neural Information Processing Systems, 1999.](https://mlanthology.org/neurips/1999/li1999neurips-mixture/)BibTeX
@inproceedings{li1999neurips-mixture,
title = {{Mixture Density Estimation}},
author = {Li, Jonathan Q. and Barron, Andrew R.},
booktitle = {Neural Information Processing Systems},
year = {1999},
pages = {279-285},
url = {https://mlanthology.org/neurips/1999/li1999neurips-mixture/}
}