Feedback Detection for Live Predictors
Abstract
A predictor that is deployed in a live production system may perturb the features it uses to make predictions. Such a feedback loop can occur, for example, when a model that predicts a certain type of behavior ends up causing the behavior it predicts, thus creating a self-fulfilling prophecy. In this paper we analyze predictor feedback detection as a causal inference problem, and introduce a local randomization scheme that can be used to detect non-linear feedback in real-world problems. We conduct a pilot study for our proposed methodology using a predictive system currently deployed as a part of a search engine.
Cite
Text
Wager et al. "Feedback Detection for Live Predictors." Neural Information Processing Systems, 2014.Markdown
[Wager et al. "Feedback Detection for Live Predictors." Neural Information Processing Systems, 2014.](https://mlanthology.org/neurips/2014/wager2014neurips-feedback/)BibTeX
@inproceedings{wager2014neurips-feedback,
title = {{Feedback Detection for Live Predictors}},
author = {Wager, Stefan and Chamandy, Nick and Muralidharan, Omkar and Najmi, Amir},
booktitle = {Neural Information Processing Systems},
year = {2014},
pages = {3428-3436},
url = {https://mlanthology.org/neurips/2014/wager2014neurips-feedback/}
}