D-LUT: Photorealistic Style Transfer via Diffusion Process

Abstract

Post-editing color in photographs is a crucial process for enhancing a photograph's aesthetic value. Traditionally this process has required a significant investment of time and manual effort. Previous color transfer algorithms achieved through encoder-decoder deep learning architectures have simplified this process. However these techniques may introduce artifacts and decrease image quality. Moreover previous approaches are not explainable making the method less user-friendly. In addition the computational requirements of these models limit their deployment across various devices. To address these challenges we introduce the Diffusion-based Look-Up Table (D-LUT). This approach is artifact-free explainable computationally efficient and does not require pretraining stage. It derives a 3D Look-Up Table (3D LUT) for transitioning between the color styles of different images. Specifically this 3D LUT is obtained using a score-matching algorithm followed by color distribution alignment through Langevin dynamics. Our proposed D-LUT approach has achieved state-of-the-art performance while requiring significantly less GPU memory than previous baselines. Importantly the 3D LUTs explicitly derived from the D-LUT algorithm enable color style transfer across broader visual modalities such as real-time color transfer for videos.