Understanding DDPM Latent Codes Through Optimal Transport
Abstract
Diffusion models have recently outperformed alternative approaches to model the distribution of natural images. Such diffusion models allow for deterministic sampling via the probability flow ODE, giving rise to a latent space and an encoder map. While having important practical applications, such as the estimation of the likelihood, the theoretical properties of this map are not yet fully understood. In the present work, we partially address this question for the popular case of the VP-SDE (DDPM) approach. We show that, perhaps surprisingly, the DDPM encoder map coincides with the optimal transport map for common distributions; we support this claim by extensive numerical experiments using advanced tensor train solver for multidimensional Fokker-Planck equation. We provide additional theoretical evidence for the case of multivariate normal distributions.
Cite
Text
Khrulkov et al. "Understanding DDPM Latent Codes Through Optimal Transport." International Conference on Learning Representations, 2023.Markdown
[Khrulkov et al. "Understanding DDPM Latent Codes Through Optimal Transport." International Conference on Learning Representations, 2023.](https://mlanthology.org/iclr/2023/khrulkov2023iclr-understanding/)BibTeX
@inproceedings{khrulkov2023iclr-understanding,
title = {{Understanding DDPM Latent Codes Through Optimal Transport}},
author = {Khrulkov, Valentin and Ryzhakov, Gleb and Chertkov, Andrei and Oseledets, Ivan},
booktitle = {International Conference on Learning Representations},
year = {2023},
url = {https://mlanthology.org/iclr/2023/khrulkov2023iclr-understanding/}
}