Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data
Abstract
Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO-CV) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO sub-sampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orders of magnitude more efficient than previous approaches, as well as being better suited to model comparison.
Cite
Text
Magnusson et al. "Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data." Artificial Intelligence and Statistics, 2020.Markdown
[Magnusson et al. "Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data." Artificial Intelligence and Statistics, 2020.](https://mlanthology.org/aistats/2020/magnusson2020aistats-leaveoneout/)BibTeX
@inproceedings{magnusson2020aistats-leaveoneout,
title = {{Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data}},
author = {Magnusson, Måns and Vehtari, Aki and Jonasson, Johan and Andersen, Michael},
booktitle = {Artificial Intelligence and Statistics},
year = {2020},
pages = {341-351},
volume = {108},
url = {https://mlanthology.org/aistats/2020/magnusson2020aistats-leaveoneout/}
}