Radial and Directional Posteriors for Bayesian Deep Learning
Abstract
We propose a new variational family for Bayesian neural networks. We decompose the variational posterior into two components, where the radial component captures the strength of each neuron in terms of its magnitude; while the directional component captures the statistical dependencies among the weight parameters. The dependencies learned via the directional density provide better modeling performance compared to the widely-used Gaussian mean-field-type variational family. In addition, the strength of input and output neurons learned via our posterior provides a structured way to compress neural networks. Indeed, experiments show that our variational family improves predictive performance and yields compressed networks simultaneously.
Cite
Text
Oh et al. "Radial and Directional Posteriors for Bayesian Deep Learning." AAAI Conference on Artificial Intelligence, 2020. doi:10.1609/AAAI.V34I04.5976Markdown
[Oh et al. "Radial and Directional Posteriors for Bayesian Deep Learning." AAAI Conference on Artificial Intelligence, 2020.](https://mlanthology.org/aaai/2020/oh2020aaai-radial/) doi:10.1609/AAAI.V34I04.5976BibTeX
@inproceedings{oh2020aaai-radial,
title = {{Radial and Directional Posteriors for Bayesian Deep Learning}},
author = {Oh, ChangYong and Adamczewski, Kamil and Park, Mijung},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2020},
pages = {5298-5305},
doi = {10.1609/AAAI.V34I04.5976},
url = {https://mlanthology.org/aaai/2020/oh2020aaai-radial/}
}