DDPM Score Matching Is Asymptotically Efficient

Abstract

The success of score-based generative models (SGMs), and particularly denoising diffusion probabilistic models (DDPMs), rests on the statistical technique of *score matching*, for which rigorous guarantees are nascent. In fact, recent work has shown that for estimation in parametric models, a variant of score matching known as implicit score matching is provably statistically inefficient for multimodal densities that are common in practice. In contrast, under mild conditions, we show that denoising score matching in DDPMs is asymptotically efficient, i.e., the DDPM estimator is asymptotically normal with covariance matrix given by the inverse Fisher information. Our proof is based on a pointwise relationship between the empirical risks of DDPM and maximum likelihood estimation.