Scoring Rule Nets: Beyond Mean Target Prediction in Multivariate Regression
Abstract
Probabilistic regression models trained with maximum likelihood estimation (MLE), can sometimes overestimate variance to an unacceptable degree. This is mostly problematic in the multivariate domain. While univariate models often optimize the popular Continuous Ranked Probability Score (CRPS), in the multivariate domain, no such alternative to MLE has yet been widely accepted. The Energy Score - the most investigated alternative - notoriously lacks closed-form expressions and sensitivity to the correlation between target variables. In this paper, we propose Conditional CRPS: a multivariate strictly proper scoring rule that extends CRPS. We show that closed-form expressions exist for popular distributions and illustrate their sensitivity to correlation. We then show in a variety of experiments on both synthetic and real data, that Conditional CRPS often outperforms MLE, and produces results comparable to state-of-the-art non-parametric models, such as Distributional Random Forest (DRF).
Cite
Text
Roordink and Hess. "Scoring Rule Nets: Beyond Mean Target Prediction in Multivariate Regression." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023. doi:10.1007/978-3-031-43415-0_12Markdown
[Roordink and Hess. "Scoring Rule Nets: Beyond Mean Target Prediction in Multivariate Regression." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023.](https://mlanthology.org/ecmlpkdd/2023/roordink2023ecmlpkdd-scoring/) doi:10.1007/978-3-031-43415-0_12BibTeX
@inproceedings{roordink2023ecmlpkdd-scoring,
title = {{Scoring Rule Nets: Beyond Mean Target Prediction in Multivariate Regression}},
author = {Roordink, Daan and Hess, Sibylle},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2023},
pages = {190-205},
doi = {10.1007/978-3-031-43415-0_12},
url = {https://mlanthology.org/ecmlpkdd/2023/roordink2023ecmlpkdd-scoring/}
}