Exploring the Boundaries of Content Moderation in Text-to-Image Generation
Abstract
This paper analyzes the community safety guidelines of five text-to-image (T2I) generation platforms and audits five T2I models, focusing on prompts related to the representation of humans in areas that might lead to societal stigma. While current research primarily focuses on ensuring safety by restricting the generation of harmful content, our study offers a complementary perspective. We argue that the concept of safety is difficult to define and operationalize, reflected in a discrepancy between the officially published safety guidelines and the actual behavior of the T2I models, and leading at times to over-censorship. Our findings call for more transparency and an inclusive dialogue about the platforms’ content moderation practices, bearing in mind their global cultural and social impact.
Cite
Text
Riccio et al. "Exploring the Boundaries of Content Moderation in Text-to-Image Generation." European Conference on Computer Vision Workshops, 2024. doi:10.1007/978-3-031-92089-9_11Markdown
[Riccio et al. "Exploring the Boundaries of Content Moderation in Text-to-Image Generation." European Conference on Computer Vision Workshops, 2024.](https://mlanthology.org/eccvw/2024/riccio2024eccvw-exploring/) doi:10.1007/978-3-031-92089-9_11BibTeX
@inproceedings{riccio2024eccvw-exploring,
title = {{Exploring the Boundaries of Content Moderation in Text-to-Image Generation}},
author = {Riccio, Piera and Curto, Georgina and Oliver, Nuria},
booktitle = {European Conference on Computer Vision Workshops},
year = {2024},
pages = {161-178},
doi = {10.1007/978-3-031-92089-9_11},
url = {https://mlanthology.org/eccvw/2024/riccio2024eccvw-exploring/}
}