A Comparison of Tight Generalization Error Bounds
Abstract
We investigate the empirical applicability of several bounds (a number of which are new) on the true error rate of learned classifiers which hold whenever the examples are chosen independently at random from a fixed distribution. The collection of tricks we use includes:1. A technique using unlabeled data for a tight derandomization of randomized bounds.2. A tight form of the progressive validation bound.3. The exact form of the test set bound. The bounds are implemented in the semibound package and are freely available.
Cite
Text
Kääriäinen and Langford. "A Comparison of Tight Generalization Error Bounds." International Conference on Machine Learning, 2005. doi:10.1145/1102351.1102403Markdown
[Kääriäinen and Langford. "A Comparison of Tight Generalization Error Bounds." International Conference on Machine Learning, 2005.](https://mlanthology.org/icml/2005/kaariainen2005icml-comparison/) doi:10.1145/1102351.1102403BibTeX
@inproceedings{kaariainen2005icml-comparison,
title = {{A Comparison of Tight Generalization Error Bounds}},
author = {Kääriäinen, Matti and Langford, John},
booktitle = {International Conference on Machine Learning},
year = {2005},
pages = {409-416},
doi = {10.1145/1102351.1102403},
url = {https://mlanthology.org/icml/2005/kaariainen2005icml-comparison/}
}