ML Anthology

Knowing When to Look for What and Where: Evaluating Generation of Spatial Descriptions with Adaptive Attention

Mehdi Ghanimifard, Simon Dobnik
ECCVW 2018 pp. 153-161
doi:10.1007/978-3-030-11018-5_14 /eccvw/2018/ghanimifard2018eccvw-knowing/

Abstract

We examine and evaluate adaptive attention [ 17 ] (which balances the focus on visual features and focus on textual features) in generating image captions in end-to-end neural networks, in particular how adaptive attention is informative for generating spatial relations. We show that the model generates spatial relations more on the basis of textual rather than visual features and therefore confirm the previous observations that the learned visual features are missing information about geometric relations between objects.

PDF ECCVW Semantic Scholar

Cite

Text

Ghanimifard and Dobnik. "Knowing When to Look for What and Where: Evaluating Generation of Spatial Descriptions with Adaptive Attention." European Conference on Computer Vision Workshops, 2018. doi:10.1007/978-3-030-11018-5_14

Markdown

[Ghanimifard and Dobnik. "Knowing When to Look for What and Where: Evaluating Generation of Spatial Descriptions with Adaptive Attention." European Conference on Computer Vision Workshops, 2018.](https://mlanthology.org/eccvw/2018/ghanimifard2018eccvw-knowing/) doi:10.1007/978-3-030-11018-5_14

BibTeX

@inproceedings{ghanimifard2018eccvw-knowing,
  title     = {{Knowing When to Look for What and Where: Evaluating Generation of Spatial Descriptions with Adaptive Attention}},
  author    = {Ghanimifard, Mehdi and Dobnik, Simon},
  booktitle = {European Conference on Computer Vision Workshops},
  year      = {2018},
  pages     = {153-161},
  doi       = {10.1007/978-3-030-11018-5_14},
  url       = {https://mlanthology.org/eccvw/2018/ghanimifard2018eccvw-knowing/}
}