ML Anthology

Two Body Problem: Collaborative Visual Task Completion

Unnat Jain, Luca Weihs, Eric Kolve, Mohammad Rastegari, Svetlana Lazebnik, Ali Farhadi, Alexander G. Schwing, Aniruddha Kembhavi
CVPR 2019
doi:10.1109/CVPR.2019.00685 /cvpr/2019/jain2019cvpr-two/

Abstract

Collaboration is a necessary skill to perform tasks that are beyond one agent's capabilities. Addressed extensively in both conventional and modern AI, multi-agent collaboration has often been studied in the context of simple grid worlds. We argue that there are inherently visual aspects to collaboration which should be studied in visually rich environments. A key element in collaboration is communication that can be either explicit, through messages, or implicit, through perception of the other agents and the visual world. Learning to collaborate in a visual environment entails learning (1) to perform the task, (2) when and what to communicate, and (3) how to act based on these communications and the perception of the visual world. In this paper we study the problem of learning to collaborate directly from pixels in AI2-THOR and demonstrate the benefits of explicit and implicit modes of communication to perform visual tasks. Refer to our project page for more details: https://prior.allenai.org/projects/two-body-problem

PDF CVPR Semantic Scholar

Cite

Text

Jain et al. "Two Body Problem: Collaborative Visual Task Completion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. doi:10.1109/CVPR.2019.00685

Markdown

[Jain et al. "Two Body Problem: Collaborative Visual Task Completion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.](https://mlanthology.org/cvpr/2019/jain2019cvpr-two/) doi:10.1109/CVPR.2019.00685

BibTeX

@inproceedings{jain2019cvpr-two,
  title     = {{Two Body Problem: Collaborative Visual Task Completion}},
  author    = {Jain, Unnat and Weihs, Luca and Kolve, Eric and Rastegari, Mohammad and Lazebnik, Svetlana and Farhadi, Ali and Schwing, Alexander G. and Kembhavi, Aniruddha},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year      = {2019},
  doi       = {10.1109/CVPR.2019.00685},
  url       = {https://mlanthology.org/cvpr/2019/jain2019cvpr-two/}
}