Adversarial Text to Continuous Image Generation

Kilichbek Haydarov, Aashiq Muhamed, Xiaoqian Shen, Jovana Lazarevic, Ivan Skorokhodov, Chamuditha Jayanga Galappaththige, Mohamed Elhoseiny

CVPR 2024 pp. 6316-6326

doi:10.1109/CVPR52733.2024.00604 /cvpr/2024/haydarov2024cvpr-adversarial/

Abstract

Existing GAN-based text-to-image models treat images as 2D pixel arrays. In this paper we approach the text-to-image task from a different perspective where a 2D image is represented as an implicit neural representation (INR). We show that straightforward conditioning of the unconditional INR-based GAN method on text inputs is not enough to achieve good performance. We propose a word-level attention-based weight modulation operator that controls the generation process of INR-GAN based on hypernetworks. Our experiments on benchmark datasets show that HyperCGAN achieves competitive performance to existing pixel-based methods and retains the properties of continuous generative models.

PDF CVPR Semantic Scholar

Cite

Text

Haydarov et al. "Adversarial Text to Continuous Image Generation." Conference on Computer Vision and Pattern Recognition, 2024. doi:10.1109/CVPR52733.2024.00604

Markdown

[Haydarov et al. "Adversarial Text to Continuous Image Generation." Conference on Computer Vision and Pattern Recognition, 2024.](https://mlanthology.org/cvpr/2024/haydarov2024cvpr-adversarial/) doi:10.1109/CVPR52733.2024.00604

BibTeX

@inproceedings{haydarov2024cvpr-adversarial,
  title     = {{Adversarial Text to Continuous Image Generation}},
  author    = {Haydarov, Kilichbek and Muhamed, Aashiq and Shen, Xiaoqian and Lazarevic, Jovana and Skorokhodov, Ivan and Galappaththige, Chamuditha Jayanga and Elhoseiny, Mohamed},
  booktitle = {Conference on Computer Vision and Pattern Recognition},
  year      = {2024},
  pages     = {6316-6326},
  doi       = {10.1109/CVPR52733.2024.00604},
  url       = {https://mlanthology.org/cvpr/2024/haydarov2024cvpr-adversarial/}
}