PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-Clustering
Abstract
We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian generalization bound for classification, the result has the appealingly simple form of a tradeoff between empirical performance and the KL-divergence of the posterior from the prior. Moreover, the PAC-Bayesian generalization bound for classification can be derived as a special case of the bound for density estimation. To illustrate a possible application of our bound we derive a generalization bound for co-clustering. The bound provides a criterion to evaluate the ability of co-clustering to predict new co-occurrences, thus introducing a supervised flavor to this traditionally unsupervised task.
Cite
Text
Seldin and Tishby. "PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-Clustering." Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, 2009.Markdown
[Seldin and Tishby. "PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-Clustering." Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, 2009.](https://mlanthology.org/aistats/2009/seldin2009aistats-pacbayesian/)BibTeX
@inproceedings{seldin2009aistats-pacbayesian,
title = {{PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-Clustering}},
author = {Seldin, Yevgeny and Tishby, Naftali},
booktitle = {Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics},
year = {2009},
pages = {472-479},
volume = {5},
url = {https://mlanthology.org/aistats/2009/seldin2009aistats-pacbayesian/}
}