How Were You Created? Explaining Synthetic Face Images Generated by Diffusion Models

Abstract

Diffusion probabilistic models have been shown to surpass the capabilities of Generative Adversarial Networks (GANs), setting a new standard in the field of generative models. While the emphasis has been predominantly on producing high-quality images, the explainability of diffusion models has often been overlooked. In this work, we introduce a state-of-the-art approach designed to explain face images generated by diffusion models. To this end, we introduce a novel methodology, named the Explainable DIffusion PRobabilistic (EDIPR) model, based on a classification framework. EDIPR consists of three stages: an initial clustering stage, which serves as the pre-processing step to discover groups of similar face images in the training set; a synthesizing stage, carried out by a diffusion model; and an explaining stage, which allows determining which training images contributed the most to the generation of a new face image. To provide explainability, we also introduced two influence scores as quantitative metrics: the Normalized Influence Score (NIS) and the class-Normalized Influence Score (cNIS). These scores provide the probability that a specific training image, or class, contributes to the generation of a new synthetic image. Our results on synthetic images generated based on the real images of the FFHQ dataset show that EDIPR provides robust and plausible explanations linking the training images to the synthetic images at three levels of granularity: the region, the image, and the class level.