Control+Shift: Generating Controllable Distribution Shifts
Abstract
We propose a new method for generating realistic datasets with distribution shifts using any decoder-based generative model. Our approach systematically creates datasets with varying intensities of distribution shifts, facilitating a comprehensive analysis of model performance degradation. We then use these generated datasets to evaluate the performance of various commonly used networks and observe a consistent decline in performance with increasing shift intensity, even when the effect is almost perceptually unnoticeable to the human eye. We see this degradation even when using data augmentations. We also find that enlarging the training dataset beyond a certain point has no effect on the robustness and that stronger inductive biases increase robustness.
Cite
Text
Friedman and Chowers. "Control+Shift: Generating Controllable Distribution Shifts." European Conference on Computer Vision Workshops, 2024. doi:10.1007/978-3-031-91907-7_19Markdown
[Friedman and Chowers. "Control+Shift: Generating Controllable Distribution Shifts." European Conference on Computer Vision Workshops, 2024.](https://mlanthology.org/eccvw/2024/friedman2024eccvw-control/) doi:10.1007/978-3-031-91907-7_19BibTeX
@inproceedings{friedman2024eccvw-control,
title = {{Control+Shift: Generating Controllable Distribution Shifts}},
author = {Friedman, Roy and Chowers, Rhea},
booktitle = {European Conference on Computer Vision Workshops},
year = {2024},
pages = {321-334},
doi = {10.1007/978-3-031-91907-7_19},
url = {https://mlanthology.org/eccvw/2024/friedman2024eccvw-control/}
}