AGIL: Learning Attention from Human for Visuomotor Tasks
Abstract
When intelligent agents learn visuomotor behaviors from human demonstrations, they may benefit from knowing where the human is allocating visual attention, which can be inferred from their gaze. A wealth of information regarding intelligent decision making is conveyed by human gaze allocation; hence, exploiting such information has the potential to improve the agents' performance. With this motivation, we propose the AGIL (Attention Guided Imitation Learning) framework. We collect high-quality human action and gaze data while playing Atari games in a carefully controlled experimental setting. Using these data, we first train a deep neural network that can predict human gaze positions and visual attention with high accuracy (the gaze network) and then train another network to predict human actions (the policy network). Incorporating the learned attention model from the gaze network into the policy network significantly improves the action prediction accuracy and task performance.
Cite
Text
Zhang et al. "AGIL: Learning Attention from Human for Visuomotor Tasks." Proceedings of the European Conference on Computer Vision (ECCV), 2018. doi:10.1007/978-3-030-01252-6_41Markdown
[Zhang et al. "AGIL: Learning Attention from Human for Visuomotor Tasks." Proceedings of the European Conference on Computer Vision (ECCV), 2018.](https://mlanthology.org/eccv/2018/zhang2018eccv-agil/) doi:10.1007/978-3-030-01252-6_41BibTeX
@inproceedings{zhang2018eccv-agil,
title = {{AGIL: Learning Attention from Human for Visuomotor Tasks}},
author = {Zhang, Ruohan and Liu, Zhuode and Zhang, Luxin and Whritner, Jake A. and Muller, Karl S. and Hayhoe, Mary M. and Ballard, Dana H.},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2018},
doi = {10.1007/978-3-030-01252-6_41},
url = {https://mlanthology.org/eccv/2018/zhang2018eccv-agil/}
}