Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation
Abstract
We introduce several novel change of measure inequalities for two families of divergences: $f$-divergences and $\alpha$-divergences. We show how the variational representation for $f$-divergences leads to novel change of measure inequalities. We also present a multiplicative change of measure inequality for $\alpha$-divergences and a generalized version of Hammersley-Chapman-Robbins inequality. Finally, we present several applications of our change of measure inequalities, including PAC-Bayesian bounds for various classes of losses and non-asymptotic intervals for Monte Carlo estimates.
Cite
Text
Ohnishi and Honorio. "Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation." Artificial Intelligence and Statistics, 2021.Markdown
[Ohnishi and Honorio. "Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation." Artificial Intelligence and Statistics, 2021.](https://mlanthology.org/aistats/2021/ohnishi2021aistats-novel/)BibTeX
@inproceedings{ohnishi2021aistats-novel,
title = {{Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation}},
author = {Ohnishi, Yuki and Honorio, Jean},
booktitle = {Artificial Intelligence and Statistics},
year = {2021},
pages = {1711-1719},
volume = {130},
url = {https://mlanthology.org/aistats/2021/ohnishi2021aistats-novel/}
}