PAC-Bayes Analysis of Maximum Entropy Classification
Abstract
We extend and apply the PAC-Bayes theorem to the analysis of maximum entropy learning by considering maximum entropy classification. The theory introduces a multiple sampling technique that controls an effective margin of the bound. We further develop a dual implementation of the convex optimisation that optimises the bound. This algorithm is tested on some simple datasets and the value of the bound compared with the test error.
Cite
Text
Shawe-Taylor and Hardoon. "PAC-Bayes Analysis of Maximum Entropy Classification." Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, 2009.Markdown
[Shawe-Taylor and Hardoon. "PAC-Bayes Analysis of Maximum Entropy Classification." Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, 2009.](https://mlanthology.org/aistats/2009/shawetaylor2009aistats-pacbayes/)BibTeX
@inproceedings{shawetaylor2009aistats-pacbayes,
title = {{PAC-Bayes Analysis of Maximum Entropy Classification}},
author = {Shawe-Taylor, John and Hardoon, David},
booktitle = {Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics},
year = {2009},
pages = {480-487},
volume = {5},
url = {https://mlanthology.org/aistats/2009/shawetaylor2009aistats-pacbayes/}
}