Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation
Abstract
We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation. Evaluating model performance is a challenging and important problem when labels are expensive. ASEs address this active testing problem using a surrogate-based estimation approach that interpolates the errors of points with unknown labels, rather than forming a Monte Carlo estimator. ASEs actively learn the underlying surrogate, and we propose a novel acquisition strategy, XWED, that tailors this learning to the final estimation task. We find that ASEs offer greater label-efficiency than the current state-of-the-art when applied to challenging model evaluation problems for deep neural networks.
Cite
Text
Kossen et al. "Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation." Neural Information Processing Systems, 2022.Markdown
[Kossen et al. "Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation." Neural Information Processing Systems, 2022.](https://mlanthology.org/neurips/2022/kossen2022neurips-active/)BibTeX
@inproceedings{kossen2022neurips-active,
title = {{Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation}},
author = {Kossen, Jannik and Farquhar, Sebastian and Gal, Yarin and Rainforth, Thomas},
booktitle = {Neural Information Processing Systems},
year = {2022},
url = {https://mlanthology.org/neurips/2022/kossen2022neurips-active/}
}