Obtaining Well Calibrated Probabilities Using Bayesian Binning

Mahdi Pakdaman Naeini, Gregory F. Cooper, Milos Hauskrecht

AAAI 2015 pp. 2901-2907

doi:10.1609/AAAI.V29I1.9602 /aaai/2015/naeini2015aaai-obtaining/

Abstract

Learning probabilistic predictive models that are well calibrated is critical for many prediction and decision-making tasks in artificial intelligence. In this paper we present a new non-parametric calibration method called Bayesian Binning into Quantiles (BBQ) which addresses key limitations of existing calibration methods. The method post processes the output of a binary classification algorithm; thus, it can be readily combined with many existing classification algorithms. The method is computationally tractable, and empirically accurate, as evidenced by the set of experiments reported here on both real and simulated datasets.

PDF AAAI Semantic Scholar

Cite

Text

Naeini et al. "Obtaining Well Calibrated Probabilities Using Bayesian Binning." AAAI Conference on Artificial Intelligence, 2015. doi:10.1609/AAAI.V29I1.9602

Markdown

[Naeini et al. "Obtaining Well Calibrated Probabilities Using Bayesian Binning." AAAI Conference on Artificial Intelligence, 2015.](https://mlanthology.org/aaai/2015/naeini2015aaai-obtaining/) doi:10.1609/AAAI.V29I1.9602

BibTeX

@inproceedings{naeini2015aaai-obtaining,
  title     = {{Obtaining Well Calibrated Probabilities Using Bayesian Binning}},
  author    = {Naeini, Mahdi Pakdaman and Cooper, Gregory F. and Hauskrecht, Milos},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2015},
  pages     = {2901-2907},
  doi       = {10.1609/AAAI.V29I1.9602},
  url       = {https://mlanthology.org/aaai/2015/naeini2015aaai-obtaining/}
}