E3: Ensemble of Expert Embedders for Adapting Synthetic Image Detectors to New Generators Using Limited Data

Aref Azizpour, Tai D. Nguyen, Manil Shrestha, Kaidi Xu, Edward Kim, Matthew C. Stamm

CVPRW 2024 pp. 4334-4344

doi:10.1109/CVPRW63382.2024.00437 /cvprw/2024/azizpour2024cvprw-e3/

Abstract

As generative AI progresses rapidly, new synthetic image generators continue to emerge at a swift pace. Traditional detection methods face two main challenges in adapting to these generators: the forensic traces of synthetic images from new techniques can vastly differ from those learned during training, and access to data for these new generators is often limited. To address these issues, we introduce the Ensemble of Expert Embedders (E3), a novel continual learning framework for updating synthetic image detectors. E3 enables the accurate detection of images from newly emerged generators using minimal training data. Our approach does this by first employing transfer learning to develop a suite of expert embedders, each specializing in the forensic traces of a specific generator. Then, all embeddings are jointly analyzed by an Expert Knowledge Fusion Network to produce accurate and reliable detection decisions. Our experiments demonstrate that E3 outperforms existing continual learning methods, including those developed specifically for synthetic image detection.

PDF CVPRW Semantic Scholar

Cite

Text

Azizpour et al. "E3: Ensemble of Expert Embedders for Adapting Synthetic Image Detectors to New Generators Using Limited Data." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024. doi:10.1109/CVPRW63382.2024.00437

Markdown

[Azizpour et al. "E3: Ensemble of Expert Embedders for Adapting Synthetic Image Detectors to New Generators Using Limited Data." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024.](https://mlanthology.org/cvprw/2024/azizpour2024cvprw-e3/) doi:10.1109/CVPRW63382.2024.00437

BibTeX

@inproceedings{azizpour2024cvprw-e3,
  title     = {{E3: Ensemble of Expert Embedders for Adapting Synthetic Image Detectors to New Generators Using Limited Data}},
  author    = {Azizpour, Aref and Nguyen, Tai D. and Shrestha, Manil and Xu, Kaidi and Kim, Edward and Stamm, Matthew C.},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2024},
  pages     = {4334-4344},
  doi       = {10.1109/CVPRW63382.2024.00437},
  url       = {https://mlanthology.org/cvprw/2024/azizpour2024cvprw-e3/}
}