On Stochastic Complexity and Admissible Models for Neural Network Classifiers
Abstract
Given some training data how should we choose a particular network clas(cid:173) sifier from a family of networks of different complexities? In this paper we discuss how the application of stochastic complexity theory to classifier design problems can provide some insights into this problem. In particular we introduce the notion of admissible models whereby the complexity of models under consideration is affected by (among other factors) the class entropy, the amount of training data, and our prior belief. In particular we discuss the implications of these results with respect to neural architec(cid:173) tures and demonstrate the approach on real data from a medical diagnosis task.
Cite
Text
Smyth. "On Stochastic Complexity and Admissible Models for Neural Network Classifiers." Neural Information Processing Systems, 1990.Markdown
[Smyth. "On Stochastic Complexity and Admissible Models for Neural Network Classifiers." Neural Information Processing Systems, 1990.](https://mlanthology.org/neurips/1990/smyth1990neurips-stochastic/)BibTeX
@inproceedings{smyth1990neurips-stochastic,
title = {{On Stochastic Complexity and Admissible Models for Neural Network Classifiers}},
author = {Smyth, Padhraic},
booktitle = {Neural Information Processing Systems},
year = {1990},
pages = {818-824},
url = {https://mlanthology.org/neurips/1990/smyth1990neurips-stochastic/}
}