De-Stereotyping Text-to-Image Models Through Prompt Tuning
Abstract
Recent text-to-image (TTI) generation models have been reported to generate images demographically stereotyped in various sensitive attributes such as gender or race. This may seriously harm the fairness of the generative model to be deployed. We propose a novel and efficient framework to de-stereotype the existing TTI model through soft prompt tuning. Utilizing a newly designed de-stereotyping loss, we train a small number of parameters consisting of the soft prompt. We demonstrate that our framework effectively balances the generated images with respect to sensitive attributes, which can also generalize to unseen text prompts.
Cite
Text
Kim et al. "De-Stereotyping Text-to-Image Models Through Prompt Tuning." ICML 2023 Workshops: DeployableGenerativeAI, 2023.Markdown
[Kim et al. "De-Stereotyping Text-to-Image Models Through Prompt Tuning." ICML 2023 Workshops: DeployableGenerativeAI, 2023.](https://mlanthology.org/icmlw/2023/kim2023icmlw-destereotyping/)BibTeX
@inproceedings{kim2023icmlw-destereotyping,
title = {{De-Stereotyping Text-to-Image Models Through Prompt Tuning}},
author = {Kim, Eunji and Kim, Siwon and Shin, Chaehun and Yoon, Sungroh},
booktitle = {ICML 2023 Workshops: DeployableGenerativeAI},
year = {2023},
url = {https://mlanthology.org/icmlw/2023/kim2023icmlw-destereotyping/}
}