Nonparametric Estimation of Renyi Divergence and Friends
Abstract
We consider nonparametric estimation of L_2, Renyi-αand Tsallis-αdivergences between continuous distributions. Our approach is to construct estimators for particular integral functionals of two densities and translate them into divergence estimators. For the integral functionals, our estimators are based on corrections of a preliminary plug-in estimator. We show that these estimators achieve the parametric convergence rate of n^-1/2 when the densities’ smoothness, s, are both at least d/4 where d is the dimension. We also derive minimax lower bounds for this problem which confirm that s > d/4 is necessary to achieve the n^-1/2 rate of convergence. We validate our theoretical guarantees with a number of simulations.
Cite
Text
Krishnamurthy et al. "Nonparametric Estimation of Renyi Divergence and Friends." International Conference on Machine Learning, 2014.Markdown
[Krishnamurthy et al. "Nonparametric Estimation of Renyi Divergence and Friends." International Conference on Machine Learning, 2014.](https://mlanthology.org/icml/2014/krishnamurthy2014icml-nonparametric/)BibTeX
@inproceedings{krishnamurthy2014icml-nonparametric,
title = {{Nonparametric Estimation of Renyi Divergence and Friends}},
author = {Krishnamurthy, Akshay and Kandasamy, Kirthevasan and Poczos, Barnabas and Wasserman, Larry},
booktitle = {International Conference on Machine Learning},
year = {2014},
pages = {919-927},
volume = {32},
url = {https://mlanthology.org/icml/2014/krishnamurthy2014icml-nonparametric/}
}