ML Anthology

A Semiotic Methodology for Assessing the Compositional Effectiveness of Generative Text-to-Image Models (Midjourney and DALL·E)

Enzo D'armenio, Adrien Deliège, Maria Giulia Dondero
ECCVW 2024 pp. 112-127
doi:10.1007/978-3-031-92089-9_8 /eccvw/2024/daposarmenio2024eccvw-semiotic/

Abstract

In this paper, we intend to propose a unified methodology in order to assess the effectiveness of text-to-image generation models. Existing evaluation methods are based on criteria such as fidelity and alignment between verbal prompts and produced images, but the human and CLIPScore evaluations mix together criteria of position, action, and photorealism. Instead, we intend to adapt the model analysis elaborated in visual semiotics in order to identify a set of discrete visual composition criteria upon which to establish the accuracy of the assessments. We will therefore distinguish three fundamental dimensions -plastic categories, multimodal translation, and enunciation- each of them in multiple specific sub-criteria. We will then test these criteria on Midjourney and DALL $\cdot $ · E while providing the abstract structure of the prompts in order to allow them to be used in future quantitative analyses.

PDF ECCVW Semantic Scholar

Cite

Text

D'armenio et al. "A Semiotic Methodology for Assessing the Compositional Effectiveness of Generative Text-to-Image Models (Midjourney and DALL·E)." European Conference on Computer Vision Workshops, 2024. doi:10.1007/978-3-031-92089-9_8

Markdown

[D'armenio et al. "A Semiotic Methodology for Assessing the Compositional Effectiveness of Generative Text-to-Image Models (Midjourney and DALL·E)." European Conference on Computer Vision Workshops, 2024.](https://mlanthology.org/eccvw/2024/daposarmenio2024eccvw-semiotic/) doi:10.1007/978-3-031-92089-9_8

BibTeX

@inproceedings{daposarmenio2024eccvw-semiotic,
  title     = {{A Semiotic Methodology for Assessing the Compositional Effectiveness of Generative Text-to-Image Models (Midjourney and DALL·E)}},
  author    = {D'armenio, Enzo and Deliège, Adrien and Dondero, Maria Giulia},
  booktitle = {European Conference on Computer Vision Workshops},
  year      = {2024},
  pages     = {112-127},
  doi       = {10.1007/978-3-031-92089-9_8},
  url       = {https://mlanthology.org/eccvw/2024/daposarmenio2024eccvw-semiotic/}
}