Quantifying the Confidence of Anomaly Detectors in Their Example-Wise Predictions
Abstract
Anomaly detection focuses on identifying examples in the data that somehow deviate from what is expected or typical. Algorithms for this task usually assign a score to each example that represents how anomalous the example is. Then, a threshold on the scores turns them into concrete predictions. However, each algorithm uses a different approach to assign the scores, which makes them difficult to interpret and can quickly erode a user’s trust in the predictions. This paper introduces an approach for assessing the reliability of any anomaly detector’s example-wise predictions. To do so, we propose a Bayesian approach for converting anomaly scores to probability estimates. This enables the anomaly detector to assign a confidence score to each prediction which captures its uncertainty in that prediction. We theoretically analyze the convergence behaviour of our confidence estimate. Empirically, we demonstrate the effectiveness of the framework in quantifying a detector’s confidence in its predictions on a large benchmark of datasets.
Cite
Text
Perini et al. "Quantifying the Confidence of Anomaly Detectors in Their Example-Wise Predictions." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020. doi:10.1007/978-3-030-67664-3_14Markdown
[Perini et al. "Quantifying the Confidence of Anomaly Detectors in Their Example-Wise Predictions." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020.](https://mlanthology.org/ecmlpkdd/2020/perini2020ecmlpkdd-quantifying/) doi:10.1007/978-3-030-67664-3_14BibTeX
@inproceedings{perini2020ecmlpkdd-quantifying,
title = {{Quantifying the Confidence of Anomaly Detectors in Their Example-Wise Predictions}},
author = {Perini, Lorenzo and Vercruyssen, Vincent and Davis, Jesse},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2020},
pages = {227-243},
doi = {10.1007/978-3-030-67664-3_14},
url = {https://mlanthology.org/ecmlpkdd/2020/perini2020ecmlpkdd-quantifying/}
}