Evaluating Robustness to Dataset Shift via Parametric Robustness Sets
Abstract
We give a method for proactively identifying small, plausible shifts in distribution which lead to large differences in model performance. To ensure that these shifts are plausible, we parameterize them in terms of interpretable changes in causal mechanisms of observed variables. This defines a parametric robustness set of plausible distributions and a corresponding worst-case loss. We construct a local approximation to the loss under shift, and show that problem of finding worst-case shifts can be efficiently solved.
Cite
Text
Oberst et al. "Evaluating Robustness to Dataset Shift via Parametric Robustness Sets." ICML 2022 Workshops: SCIS, 2022.Markdown
[Oberst et al. "Evaluating Robustness to Dataset Shift via Parametric Robustness Sets." ICML 2022 Workshops: SCIS, 2022.](https://mlanthology.org/icmlw/2022/oberst2022icmlw-evaluating/)BibTeX
@inproceedings{oberst2022icmlw-evaluating,
title = {{Evaluating Robustness to Dataset Shift via Parametric Robustness Sets}},
author = {Oberst, Michael and Thams, Nikolaj and Sontag, David},
booktitle = {ICML 2022 Workshops: SCIS},
year = {2022},
url = {https://mlanthology.org/icmlw/2022/oberst2022icmlw-evaluating/}
}