Unsupervised and Semi-Supervised Bias Benchmarking in Face Recognition
Abstract
We introduce Semi-supervised Performance Evaluation for Face Recognition (SPE-FR). SPE-FR is a statistical method for evaluating the performance and algorithmic bias of face verification systems when identity labels are unavailable or incomplete. The method is based on parametric Bayesian modeling of the face embedding similarity scores. SPE-FR produces point estimates, performance curves, and confidence bands that reflect uncertainty in the estimation procedure. Focusing on the unsupervised setting wherein no identity labels are available, we validate our method through experiments on a wide range of face embedding models and two publicly available evaluation datasets. Experiments show that SPE-FR can accurately assess performance on data with no identity labels, and confidently reveal demographic biases in system performance.
Cite
Text
Chouldechova et al. "Unsupervised and Semi-Supervised Bias Benchmarking in Face Recognition." Proceedings of the European Conference on Computer Vision (ECCV), 2022. doi:10.1007/978-3-031-19778-9_17Markdown
[Chouldechova et al. "Unsupervised and Semi-Supervised Bias Benchmarking in Face Recognition." Proceedings of the European Conference on Computer Vision (ECCV), 2022.](https://mlanthology.org/eccv/2022/chouldechova2022eccv-unsupervised/) doi:10.1007/978-3-031-19778-9_17BibTeX
@inproceedings{chouldechova2022eccv-unsupervised,
title = {{Unsupervised and Semi-Supervised Bias Benchmarking in Face Recognition}},
author = {Chouldechova, Alexandra and Deng, Siqi and Wang, Yongxin and Xia, Wei and Perona, Pietro},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2022},
doi = {10.1007/978-3-031-19778-9_17},
url = {https://mlanthology.org/eccv/2022/chouldechova2022eccv-unsupervised/}
}