Learning to Count Objects in Natural Images for Visual Question Answering
Abstract
Visual Question Answering (VQA) models have struggled with counting objects in natural images so far. We identify a fundamental problem due to soft attention in these models as a cause. To circumvent this problem, we propose a neural network component that allows robust counting from object proposals. Experiments on a toy task show the effectiveness of this component and we obtain state-of-the-art accuracy on the number category of the VQA v2 dataset without negatively affecting other categories, even outperforming ensemble models with our single model. On a difficult balanced pair metric, the component gives a substantial improvement in counting over a strong baseline by 6.6%.
Cite
Text
Zhang et al. "Learning to Count Objects in Natural Images for Visual Question Answering." International Conference on Learning Representations, 2018.Markdown
[Zhang et al. "Learning to Count Objects in Natural Images for Visual Question Answering." International Conference on Learning Representations, 2018.](https://mlanthology.org/iclr/2018/zhang2018iclr-learning-a/)BibTeX
@inproceedings{zhang2018iclr-learning-a,
title = {{Learning to Count Objects in Natural Images for Visual Question Answering}},
author = {Zhang, Yan and Hare, Jonathon and Prügel-Bennett, Adam},
booktitle = {International Conference on Learning Representations},
year = {2018},
url = {https://mlanthology.org/iclr/2018/zhang2018iclr-learning-a/}
}