Shift-Pessimistic Active Learning Using Robust Bias-Aware Prediction
Abstract
Existing approaches to active learning are generally optimistic about their certainty with respect to data shift between labeled and unlabeled data. They assume that unknown datapoint labels follow the inductive biases of the active learner. As a result, the most useful datapoint labels—ones that refute current inductive biases—are rarely solicited. We propose a shift-pessimistic approach to active learning that assumes the worst-case about the unknown conditional label distribution. This closely aligns model uncertainty with generalization error, enabling more useful label solicitation. We investigate the theoretical benefits of this approach and demonstrate its empirical advantages on probabilistic binary classification tasks.
Cite
Text
Liu et al. "Shift-Pessimistic Active Learning Using Robust Bias-Aware Prediction." AAAI Conference on Artificial Intelligence, 2015. doi:10.1609/AAAI.V29I1.9609Markdown
[Liu et al. "Shift-Pessimistic Active Learning Using Robust Bias-Aware Prediction." AAAI Conference on Artificial Intelligence, 2015.](https://mlanthology.org/aaai/2015/liu2015aaai-shift/) doi:10.1609/AAAI.V29I1.9609BibTeX
@inproceedings{liu2015aaai-shift,
title = {{Shift-Pessimistic Active Learning Using Robust Bias-Aware Prediction}},
author = {Liu, Anqi and Reyzin, Lev and Ziebart, Brian D.},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2015},
pages = {2764-2770},
doi = {10.1609/AAAI.V29I1.9609},
url = {https://mlanthology.org/aaai/2015/liu2015aaai-shift/}
}