ML Anthology

Diffusion-Enhanced PatchMatch: A Framework for Arbitrary Style Transfer with Diffusion Models

Mark Hamazaspyan, Shant Navasardyan
CVPRW 2023 pp. 797-805
doi:10.1109/CVPRW59228.2023.00087 /cvprw/2023/hamazaspyan2023cvprw-diffusionenhanced/

Abstract

Diffusion models have gained immense popularity in recent years due to their impressive ability to generate high-quality images. The opportunities that diffusion models provide are numerous, from text-to-image synthesis to image restoration and enhancement, as well as image compression and inpainting. However, expressing image style in words can be a challenging task, making it difficult for diffusion models to generate images with specific style without additional optimization techniques. In this paper, we present a novel method, Diffusion-Enhanced PatchMatch (DEPM), that leverages Stable Diffusion for style transfer without any finetuning or pretraining. DEPM captures high-level style features while preserving the fine-grained texture details of the original image. By enabling the transfer of arbitrary styles during inference, our approach makes the process more flexible and efficient. Moreover, its optimization-free nature makes it accessible to a wide range of users.

CVPRW Semantic Scholar

Cite

Text

Hamazaspyan and Navasardyan. "Diffusion-Enhanced PatchMatch: A Framework for Arbitrary Style Transfer with Diffusion Models." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023. doi:10.1109/CVPRW59228.2023.00087

Markdown

[Hamazaspyan and Navasardyan. "Diffusion-Enhanced PatchMatch: A Framework for Arbitrary Style Transfer with Diffusion Models." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023.](https://mlanthology.org/cvprw/2023/hamazaspyan2023cvprw-diffusionenhanced/) doi:10.1109/CVPRW59228.2023.00087

BibTeX

@inproceedings{hamazaspyan2023cvprw-diffusionenhanced,
  title     = {{Diffusion-Enhanced PatchMatch: A Framework for Arbitrary Style Transfer with Diffusion Models}},
  author    = {Hamazaspyan, Mark and Navasardyan, Shant},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2023},
  pages     = {797-805},
  doi       = {10.1109/CVPRW59228.2023.00087},
  url       = {https://mlanthology.org/cvprw/2023/hamazaspyan2023cvprw-diffusionenhanced/}
}