Learning to Reject with a Fixed Predictor: Application to Decontextualization
Abstract
We study the problem of classification with a reject option for a fixed predictor, crucial to natural language processing. We introduce a new problem formulation for this scenario, and an algorithm minimizing a new surrogate loss function. We provide a complete theoretical analysis of the surrogate loss function with a strong $H$-consistency guarantee. For evaluation, we choose the \textit{decontextualization} task, and provide a manually-labelled dataset of $2\mathord,000$ examples. Our algorithm significantly outperforms the baselines considered, with a $\sim 25$% improvement in coverage when halving the error rate, which is only $\sim 3$% away from the theoretical limit.
Cite
Text
Mohri et al. "Learning to Reject with a Fixed Predictor: Application to Decontextualization." International Conference on Learning Representations, 2024.Markdown
[Mohri et al. "Learning to Reject with a Fixed Predictor: Application to Decontextualization." International Conference on Learning Representations, 2024.](https://mlanthology.org/iclr/2024/mohri2024iclr-learning/)BibTeX
@inproceedings{mohri2024iclr-learning,
title = {{Learning to Reject with a Fixed Predictor: Application to Decontextualization}},
author = {Mohri, Christopher and Andor, Daniel and Choi, Eunsol and Collins, Michael and Mao, Anqi and Zhong, Yutao},
booktitle = {International Conference on Learning Representations},
year = {2024},
url = {https://mlanthology.org/iclr/2024/mohri2024iclr-learning/}
}