Perceptual Artifacts Localization for Image Synthesis Tasks

Lingzhi Zhang, Zhengjie Xu, Connelly Barnes, Yuqian Zhou, Qing Liu, He Zhang, Sohrab Amirghodsi, Zhe Lin, Eli Shechtman, Jianbo Shi

ICCV 2023 pp. 7579-7590

doi:10.1109/ICCV51070.2023.00697 /iccv/2023/zhang2023iccv-perceptual/

Abstract

Recent advancements in deep generative models have facilitated the creation of photo-realistic images across various tasks. However, these generated images often exhibit perceptual artifacts in specific regions, necessitating manual correction. In this study, we present a comprehensive empirical examination of Perceptual Artifacts Localization (PAL) spanning diverse image synthesis endeavors. We introduce a novel dataset comprising 10,168 generated images, each annotated with per-pixel perceptual artifact labels across ten synthesis tasks. A segmentation model, trained on our proposed dataset, effectively localizes artifacts across a range of tasks. Additionally, we illustrate its proficiency in adapting to previously unseen models using minimal training samples. We further propose an innovative zoom-in inpainting pipeline that seamlessly rectifies perceptual artifacts in the generated images. Through our experimental analyses, we elucidate several invaluable downstream applications, such as automated artifact rectification, non-referential image quality evaluation, and abnormal region detection in images. The dataset and code are released here: https://owenzlz.github.io/PAL4VST

PDF ICCV Semantic Scholar

Cite

Text

Zhang et al. "Perceptual Artifacts Localization for Image Synthesis Tasks." International Conference on Computer Vision, 2023. doi:10.1109/ICCV51070.2023.00697

Markdown

[Zhang et al. "Perceptual Artifacts Localization for Image Synthesis Tasks." International Conference on Computer Vision, 2023.](https://mlanthology.org/iccv/2023/zhang2023iccv-perceptual/) doi:10.1109/ICCV51070.2023.00697

BibTeX

@inproceedings{zhang2023iccv-perceptual,
  title     = {{Perceptual Artifacts Localization for Image Synthesis Tasks}},
  author    = {Zhang, Lingzhi and Xu, Zhengjie and Barnes, Connelly and Zhou, Yuqian and Liu, Qing and Zhang, He and Amirghodsi, Sohrab and Lin, Zhe and Shechtman, Eli and Shi, Jianbo},
  booktitle = {International Conference on Computer Vision},
  year      = {2023},
  pages     = {7579-7590},
  doi       = {10.1109/ICCV51070.2023.00697},
  url       = {https://mlanthology.org/iccv/2023/zhang2023iccv-perceptual/}
}