ML Anthology

Are All Combinations Equal? Combining Textual and Visual Features with Multiple Space Learning for Text-Based Video Retrieval

Damianos Galanopoulos, Vasileios Mezaris
ECCVW 2022 pp. 627-643
doi:10.1007/978-3-031-25069-9_40 /eccvw/2022/galanopoulos2022eccvw-all/

Abstract

In this paper we tackle the cross-modal video retrieval problem and, more specifically, we focus on text-to-video retrieval. We investigate how to optimally combine multiple diverse textual and visual features into feature pairs that lead to generating multiple joint feature spaces, which encode text-video pairs into comparable representations. To learn these representations our proposed network architecture is trained by following a multiple space learning procedure. Moreover, at the retrieval stage, we introduce additional softmax operations for revising the inferred query-video similarities. Extensive experiments in several setups based on three large-scale datasets (IACC.3, V3C1, and MSR-VTT) lead to conclusions on how to best combine text-visual features and document the performance of the proposed network. (Source code is made publicly available at: https://github.com/bmezaris/TextToVideoRetrieval-TtimesV )

PDF ECCVW Semantic Scholar

Cite

Text

Galanopoulos and Mezaris. "Are All Combinations Equal? Combining Textual and Visual Features with Multiple Space Learning for Text-Based Video Retrieval." European Conference on Computer Vision Workshops, 2022. doi:10.1007/978-3-031-25069-9_40

Markdown

[Galanopoulos and Mezaris. "Are All Combinations Equal? Combining Textual and Visual Features with Multiple Space Learning for Text-Based Video Retrieval." European Conference on Computer Vision Workshops, 2022.](https://mlanthology.org/eccvw/2022/galanopoulos2022eccvw-all/) doi:10.1007/978-3-031-25069-9_40

BibTeX

@inproceedings{galanopoulos2022eccvw-all,
  title     = {{Are All Combinations Equal? Combining Textual and Visual Features with Multiple Space Learning for Text-Based Video Retrieval}},
  author    = {Galanopoulos, Damianos and Mezaris, Vasileios},
  booktitle = {European Conference on Computer Vision Workshops},
  year      = {2022},
  pages     = {627-643},
  doi       = {10.1007/978-3-031-25069-9_40},
  url       = {https://mlanthology.org/eccvw/2022/galanopoulos2022eccvw-all/}
}