Newton Diagram and Stochastic Complexity in Mixture of Binomial Distributions
Abstract
Many singular learning machines such as neural networks and mixture models are used in the information engineering field. In spite of their wide range applications, their mathematical foundation of analysis is not yet constructed because of the singularities in the parameter space. In recent years, we developed the algebraic geometrical method that shows the relation between the efficiency in Bayesian estimation and the singularities. We also constructed an algorithm in which the Newton diagram is used to search of desingularization maps. Using the Newton diagram, we are able to reveal the exact value of the asymptotic stochastic complexity, which is a criterion of the model selection. In this paper, we apply the method and the algorithm to a mixture of binomial distributions and clarify its stochastic complexity. Since our result is given by the mathematically rigorous way, it can contribute to the evaluation of the conventional approximations for calculating the stochastic complexity, such as the Markov Chain Monte Carlo and Variational Bayes methods.
Cite
Text
Yamazaki and Watanabe. "Newton Diagram and Stochastic Complexity in Mixture of Binomial Distributions." International Conference on Algorithmic Learning Theory, 2004. doi:10.1007/978-3-540-30215-5_27Markdown
[Yamazaki and Watanabe. "Newton Diagram and Stochastic Complexity in Mixture of Binomial Distributions." International Conference on Algorithmic Learning Theory, 2004.](https://mlanthology.org/alt/2004/yamazaki2004alt-newton/) doi:10.1007/978-3-540-30215-5_27BibTeX
@inproceedings{yamazaki2004alt-newton,
title = {{Newton Diagram and Stochastic Complexity in Mixture of Binomial Distributions}},
author = {Yamazaki, Keisuke and Watanabe, Sumio},
booktitle = {International Conference on Algorithmic Learning Theory},
year = {2004},
pages = {350-364},
doi = {10.1007/978-3-540-30215-5_27},
url = {https://mlanthology.org/alt/2004/yamazaki2004alt-newton/}
}