Algorithms for Optimal Adaptation ofDiffusion Models to Reward Functions
Abstract
We develop algorithms for adapting pretrained diffusion models to optimize reward functions while retaining fidelity to the pretrained model. We propose a general framework for this adaptation that trades off fidelity to a pretrained diffusion model and achieving high reward. Our algorithms take advantage of the continuous nature of diffusion processes to pose reward-based learning either as a trajectory optimization or continuous state reinforcement learning problem. We demonstrate the efficacy of our approach across several application domains, including the generation of time series of household power consumption and images satisfying specific constraints like the absence of memorized images or corruptions.
Cite
Text
Dvijotham et al. "Algorithms for Optimal Adaptation ofDiffusion Models to Reward Functions." ICML 2023 Workshops: Frontiers4LCD, 2023.Markdown
[Dvijotham et al. "Algorithms for Optimal Adaptation ofDiffusion Models to Reward Functions." ICML 2023 Workshops: Frontiers4LCD, 2023.](https://mlanthology.org/icmlw/2023/dvijotham2023icmlw-algorithms/)BibTeX
@inproceedings{dvijotham2023icmlw-algorithms,
title = {{Algorithms for Optimal Adaptation ofDiffusion Models to Reward Functions}},
author = {Dvijotham, Krishnamurthy Dj and Omidshafiei, Shayegan and Lee, Kimin and Collins, Katherine M. and Ramachandran, Deepak and Weller, Adrian and Ghavamzadeh, Mohammad and Nasr, Milad and Fan, Ying and Liu, Jeremiah Zhe},
booktitle = {ICML 2023 Workshops: Frontiers4LCD},
year = {2023},
url = {https://mlanthology.org/icmlw/2023/dvijotham2023icmlw-algorithms/}
}