Estimating Accuracy from Unlabeled Data: A Bayesian Approach
Abstract
We consider the question of how unlabeled data can be used to estimate the true accuracy of learned classifiers, and the related question of how outputs from several classifiers performing the same task can be combined based on their estimated accuracies. To answer these questions, we first present a simple graphical model that performs well in practice. We then provide two nonparametric extensions to it that improve its performance. Experiments on two real-world data sets produce accuracy estimates within a few percent of the true accuracy, using solely unlabeled data. Our models also outperform existing state-of-the-art solutions in both estimating accuracies, and combining multiple classifier outputs.
Cite
Text
Platanios et al. "Estimating Accuracy from Unlabeled Data: A Bayesian Approach." International Conference on Machine Learning, 2016.Markdown
[Platanios et al. "Estimating Accuracy from Unlabeled Data: A Bayesian Approach." International Conference on Machine Learning, 2016.](https://mlanthology.org/icml/2016/platanios2016icml-estimating/)BibTeX
@inproceedings{platanios2016icml-estimating,
title = {{Estimating Accuracy from Unlabeled Data: A Bayesian Approach}},
author = {Platanios, Emmanouil Antonios and Dubey, Avinava and Mitchell, Tom},
booktitle = {International Conference on Machine Learning},
year = {2016},
pages = {1416-1425},
volume = {48},
url = {https://mlanthology.org/icml/2016/platanios2016icml-estimating/}
}