Cold Posterior Effect Towards Adversarial Robustness

Abstract

In this study, we delve into the application of Bayesian Neural Networks (BNN) as a prominent strategy for addressing adversarial attacks, elucidating their enhanced robustness. Specifically, our investigation centers on the cold posterior effect within BNNs and its role in fortifying the models against adversarial perturbations. Our findings underscore that harnessing the cold posterior effect markedly augments the models' resilience to adversarial manipulations when compared to warm counterparts, thereby elevating the overall security and reliability of the model. To substantiate these observations, we conduct rigorous experiments involving popular white-box and black-box attacks, on both fully connected networks and ResNet-20 architectures. Our empirical results unequivocally demonstrate the superior performance of cold models over warm models with multiple training methods including SGMCMC, SGHMC, and VI, against adversarial threats in diverse scenarios. This study not only contributes empirical evidence but also offers theoretical insights elucidating the efficacy of the cold posterior effect in bolstering the adversarial robustness of BNNs.