Solving Bayesian Inverse Problems with Diffusion Priors and Off-Policy RL
Abstract
This paper presents a practical application of Relative Trajectory Balance (RTB), a recently introduced off-policy reinforcement learning (RL) objective that can asymptotically solve Bayesian inverse problems optimally. We extend the original work by using RTB to train conditional diffusion model posteriors from pretrained unconditional priors for challenging linear and non-linear inverse problems in vision, and science. We use the objective alongside techniques such as off-policy backtracking exploration to improve training. Importantly, our results show that existing training-free diffusion posterior methods struggle to perform effective posterior inference in latent space due to inherent biases.
Cite
Text
Scimeca et al. "Solving Bayesian Inverse Problems with Diffusion Priors and Off-Policy RL." ICLR 2025 Workshops: DeLTa, 2025.Markdown
[Scimeca et al. "Solving Bayesian Inverse Problems with Diffusion Priors and Off-Policy RL." ICLR 2025 Workshops: DeLTa, 2025.](https://mlanthology.org/iclrw/2025/scimeca2025iclrw-solving/)BibTeX
@inproceedings{scimeca2025iclrw-solving,
title = {{Solving Bayesian Inverse Problems with Diffusion Priors and Off-Policy RL}},
author = {Scimeca, Luca and Venkatraman, Siddarth and Jain, Moksh and Kim, Minsu and Sendera, Marcin and Hasan, Mohsin and Adam, Alexandre and Hezaveh, Yashar and Perreault-Levasseur, Laurence and Bengio, Yoshua and Berseth, Glen and Malkin, Nikolay},
booktitle = {ICLR 2025 Workshops: DeLTa},
year = {2025},
url = {https://mlanthology.org/iclrw/2025/scimeca2025iclrw-solving/}
}