Robustness to Corruption in Pre-Trained Bayesian Neural Networks
Abstract
We develop ShiftMatch, a new training-data-dependent likelihood for robustness to corruption in Bayesian neural networks (BNNs). ShiftMatch is inspired by the training-data-dependent “EmpCov” priors from Izmailov et al. (2021a), and efficiently matches test-time spatial correlations to those at training time. Critically, ShiftMatch is designed to leave the neural network’s training time likelihood unchanged, allowing it to use publicly available samples from pre-trained BNNs. Using pre-trained HMC samples, ShiftMatch gives strong performance improvements on CIFAR-10-C, outperforms EmpCov priors (though ShiftMatch uses extra information from a minibatch of corrupted test points), and is perhaps the first Bayesian method capable of convincingly outperforming plain deep ensembles.
Cite
Text
Wang and Aitchison. "Robustness to Corruption in Pre-Trained Bayesian Neural Networks." International Conference on Learning Representations, 2023.Markdown
[Wang and Aitchison. "Robustness to Corruption in Pre-Trained Bayesian Neural Networks." International Conference on Learning Representations, 2023.](https://mlanthology.org/iclr/2023/wang2023iclr-robustness/)BibTeX
@inproceedings{wang2023iclr-robustness,
title = {{Robustness to Corruption in Pre-Trained Bayesian Neural Networks}},
author = {Wang, Xi and Aitchison, Laurence},
booktitle = {International Conference on Learning Representations},
year = {2023},
url = {https://mlanthology.org/iclr/2023/wang2023iclr-robustness/}
}