Information Measure Based Skeletonisation

NeurIPS 1991 pp. 1080-1087

/neurips/1991/ramachandran1991neurips-information/

Abstract

Automatic determination of proper neural network topology by trimming over-sized networks is an important area of study, which has previously been addressed using a variety of techniques. In this paper, we present Information Measure Based Skeletonisation (IMBS), a new approach to this problem where superfluous hidden units are removed based on their information measure (1M). This measure, borrowed from decision tree in(cid:173) duction techniques, reflects the degree to which the hyperplane formed by a hidden unit discriminates between training data classes. We show the results of applying IMBS to three classification tasks and demonstrate that it removes a substantial number of hidden units without significantly affecting network performance.

PDF NeurIPS Semantic Scholar

Cite

Text

Ramachandran and Pratt. "Information Measure Based Skeletonisation." Neural Information Processing Systems, 1991.

Markdown

[Ramachandran and Pratt. "Information Measure Based Skeletonisation." Neural Information Processing Systems, 1991.](https://mlanthology.org/neurips/1991/ramachandran1991neurips-information/)

BibTeX

@inproceedings{ramachandran1991neurips-information,
  title     = {{Information Measure Based Skeletonisation}},
  author    = {Ramachandran, Sowmya and Pratt, Lorien Y.},
  booktitle = {Neural Information Processing Systems},
  year      = {1991},
  pages     = {1080-1087},
  url       = {https://mlanthology.org/neurips/1991/ramachandran1991neurips-information/}
}