Emergent Communication in Interactive Sketch Question Answering
Abstract
Vision-based emergent communication (EC) aims to learn to communicate through sketches and demystify the evolution of human communication. Ironically, previous works neglect multi-round interaction, which is indispensable in human communication. To fill this gap, we first introduce a novel Interactive Sketch Question Answering (ISQA) task, where two collaborative players are interacting through sketches to answer a question about an image. To accomplish this task, we design a new and efficient interactive EC system, which can achieve an effective balance among three evaluation factors, including the question answering accuracy, drawing complexity and human interpretability. Our experimental results demonstrate that multi-round interactive mechanism facilitates tar- geted and efficient communication between intelligent agents. The code will be released.
Cite
Text
Lei et al. "Emergent Communication in Interactive Sketch Question Answering." Neural Information Processing Systems, 2023.Markdown
[Lei et al. "Emergent Communication in Interactive Sketch Question Answering." Neural Information Processing Systems, 2023.](https://mlanthology.org/neurips/2023/lei2023neurips-emergent/)BibTeX
@inproceedings{lei2023neurips-emergent,
title = {{Emergent Communication in Interactive Sketch Question Answering}},
author = {Lei, Zixing and Zhang, Yiming and Xiong, Yuxin and Chen, Siheng},
booktitle = {Neural Information Processing Systems},
year = {2023},
url = {https://mlanthology.org/neurips/2023/lei2023neurips-emergent/}
}