Evaluating Text-to-Image Matching Using Binary Image Selection (BISON)

Hexiang Hu, Ishan Misra, Laurens van der Maaten

ICCVW 2019 pp. 1887-1890

doi:10.1109/ICCVW.2019.00237 /iccvw/2019/hu2019iccvw-evaluating/

Abstract

Providing systems the ability to relate linguistic and visual content is one of the hallmarks of computer vision. Tasks such as text-based image retrieval and image captioning were designed to test this ability, but come with evaluation measures that have high variance or are difficult to interpret. We study an alternative task for systems that match text and images: given a text query, the system is asked to select the image that best matches the query from a pair of semantically similar images. The system's accuracy on this Binary Image SelectiON (BISON) task provides a robust and interpretable measure of its ability to match linguistic content with fine-grained visual structure. We gather a BISON dataset that complements the COCO dataset and use it to evaluate modern text-based image retrieval systems.

PDF ICCVW Semantic Scholar

Cite

Text

Hu et al. "Evaluating Text-to-Image Matching Using Binary Image Selection (BISON)." IEEE/CVF International Conference on Computer Vision Workshops, 2019. doi:10.1109/ICCVW.2019.00237

Markdown

[Hu et al. "Evaluating Text-to-Image Matching Using Binary Image Selection (BISON)." IEEE/CVF International Conference on Computer Vision Workshops, 2019.](https://mlanthology.org/iccvw/2019/hu2019iccvw-evaluating/) doi:10.1109/ICCVW.2019.00237

BibTeX

@inproceedings{hu2019iccvw-evaluating,
  title     = {{Evaluating Text-to-Image Matching Using Binary Image Selection (BISON)}},
  author    = {Hu, Hexiang and Misra, Ishan and van der Maaten, Laurens},
  booktitle = {IEEE/CVF International Conference on Computer Vision Workshops},
  year      = {2019},
  pages     = {1887-1890},
  doi       = {10.1109/ICCVW.2019.00237},
  url       = {https://mlanthology.org/iccvw/2019/hu2019iccvw-evaluating/}
}