Disentangling Impact of Capacity, Objective, Batchsize, Estimators, and Step-Size on Flow VI
Abstract
Normalizing flow-based variational inference (flow VI) is a promising approximate inference approach, but its performance remains inconsistent across studies. Numerous algorithmic choices influence flow VI’s performance. We conduct a step-by-step analysis to disentangle the impact of some of the key factors: capacity, objectives, gradient estimators, number of gradient estimates (batchsize), and step-sizes. Each step examines one factor while neutralizing others using insights from the previous steps and/or using extensive parallel computation. To facilitate high-fidelity evaluation, we curate a benchmark of synthetic targets that represent common posterior pathologies and allow for exact sampling. We provide specific recommendations for different factors and propose a flow VI recipe that matches or surpasses leading turnkey Hamiltonian Monte Carlo (HMC) methods.
Cite
Text
Agrawal and Domke. "Disentangling Impact of Capacity, Objective, Batchsize, Estimators, and Step-Size on Flow VI." Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, 2025.Markdown
[Agrawal and Domke. "Disentangling Impact of Capacity, Objective, Batchsize, Estimators, and Step-Size on Flow VI." Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, 2025.](https://mlanthology.org/aistats/2025/agrawal2025aistats-disentangling/)BibTeX
@inproceedings{agrawal2025aistats-disentangling,
title = {{Disentangling Impact of Capacity, Objective, Batchsize, Estimators, and Step-Size on Flow VI}},
author = {Agrawal, Abhinav and Domke, Justin},
booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics},
year = {2025},
pages = {325-333},
volume = {258},
url = {https://mlanthology.org/aistats/2025/agrawal2025aistats-disentangling/}
}