Entropy Inference and the James-Stein Estimator, with Application to Nonlinear Gene Association Networks
Abstract
We present a procedure for effective estimation of entropy and mutual information from small-sample data, and apply it to the problem of inferring high-dimensional gene association networks. Specifically, we develop a James-Stein-type shrinkage estimator, resulting in a procedure that is highly efficient statistically as well as computationally. Despite its simplicity, we show that it outperforms eight other entropy estimation procedures across a diverse range of sampling scenarios and data-generating models, even in cases of severe undersampling. We illustrate the approach by analyzing E. coli gene expression data and computing an entropy-based gene-association network from gene expression data. A computer program is available that implements the proposed shrinkage estimator.
Cite
Text
Hausser and Strimmer. "Entropy Inference and the James-Stein Estimator, with Application to Nonlinear Gene Association Networks." Journal of Machine Learning Research, 2009.Markdown
[Hausser and Strimmer. "Entropy Inference and the James-Stein Estimator, with Application to Nonlinear Gene Association Networks." Journal of Machine Learning Research, 2009.](https://mlanthology.org/jmlr/2009/hausser2009jmlr-entropy/)BibTeX
@article{hausser2009jmlr-entropy,
title = {{Entropy Inference and the James-Stein Estimator, with Application to Nonlinear Gene Association Networks}},
author = {Hausser, Jean and Strimmer, Korbinian},
journal = {Journal of Machine Learning Research},
year = {2009},
pages = {1469-1484},
volume = {10},
url = {https://mlanthology.org/jmlr/2009/hausser2009jmlr-entropy/}
}