Ensembling Shift Detectors: An Extensive Empirical Evaluation
Abstract
The term dataset shift refers to the situation where the data used to train a machine learning model is different from where the model operates. While several types of shifts naturally occur, existing shift detectors are usually designed to address only a specific type of shift. We propose a simple yet powerful technique to ensemble complementary shift detectors, while tuning the significance level of each detector's statistical test to the dataset. This enables a more robust shift detection, capable of addressing all different types of shift, which is essential in real-life settings where the precise shift type is often unknown. This approach is validated by a large-scale statistically sound benchmark study over various synthetic shifts applied to real-world structured datasets.
Cite
Text
Maggio and Dreyfus-Schmidt. "Ensembling Shift Detectors: An Extensive Empirical Evaluation." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2021. doi:10.1007/978-3-030-86523-8_22Markdown
[Maggio and Dreyfus-Schmidt. "Ensembling Shift Detectors: An Extensive Empirical Evaluation." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2021.](https://mlanthology.org/ecmlpkdd/2021/maggio2021ecmlpkdd-ensembling/) doi:10.1007/978-3-030-86523-8_22BibTeX
@inproceedings{maggio2021ecmlpkdd-ensembling,
title = {{Ensembling Shift Detectors: An Extensive Empirical Evaluation}},
author = {Maggio, Simona and Dreyfus-Schmidt, Léo},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2021},
pages = {362-377},
doi = {10.1007/978-3-030-86523-8_22},
url = {https://mlanthology.org/ecmlpkdd/2021/maggio2021ecmlpkdd-ensembling/}
}