A Stein Variational Newton Method
Abstract
Stein variational gradient descent (SVGD) was recently proposed as a general purpose nonparametric variational inference algorithm: it minimizes the Kullback–Leibler divergence between the target distribution and its approximation by implementing a form of functional gradient descent on a reproducing kernel Hilbert space [Liu & Wang, NIPS 2016]. In this paper, we accelerate and generalize the SVGD algorithm by including second-order information, thereby approximating a Newton-like iteration in function space. We also show how second-order information can lead to more effective choices of kernel. We observe significant computational gains over the original SVGD algorithm in multiple test cases.
Cite
Text
Detommaso et al. "A Stein Variational Newton Method." Neural Information Processing Systems, 2018.Markdown
[Detommaso et al. "A Stein Variational Newton Method." Neural Information Processing Systems, 2018.](https://mlanthology.org/neurips/2018/detommaso2018neurips-stein/)BibTeX
@inproceedings{detommaso2018neurips-stein,
title = {{A Stein Variational Newton Method}},
author = {Detommaso, Gianluca and Cui, Tiangang and Marzouk, Youssef and Spantini, Alessio and Scheichl, Robert},
booktitle = {Neural Information Processing Systems},
year = {2018},
pages = {9169-9179},
url = {https://mlanthology.org/neurips/2018/detommaso2018neurips-stein/}
}