Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism
Abstract
Neural Information Retrieval (NIR) has significantly improved upon heuristic-based Information Retrieval (IR) systems. Yet, failures remain frequent, the models used often being unable to retrieve documents relevant to the user's query. We address this challenge by proposing a lightweight abstention mechanism tailored for real-world constraints, with particular emphasis placed on the reranking phase. We introduce a protocol for evaluating abstention strategies in black-box scenarios (typically encountered when relying on API services), demonstrating their efficacy, and propose a simple yet effective data-driven mechanism. We provide open-source code for experiment replication and abstention implementation, fostering wider adoption and application in diverse contexts.
Cite
Text
Gisserot-Boukhlef et al. "Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism." Transactions on Machine Learning Research, 2024.Markdown
[Gisserot-Boukhlef et al. "Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism." Transactions on Machine Learning Research, 2024.](https://mlanthology.org/tmlr/2024/gisserotboukhlef2024tmlr-trustworthy/)BibTeX
@article{gisserotboukhlef2024tmlr-trustworthy,
title = {{Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism}},
author = {Gisserot-Boukhlef, Hippolyte and Faysse, Manuel and Malherbe, Emmanuel and Hudelot, Celine and Colombo, Pierre},
journal = {Transactions on Machine Learning Research},
year = {2024},
url = {https://mlanthology.org/tmlr/2024/gisserotboukhlef2024tmlr-trustworthy/}
}