A Simple Baseline for Audio-Visual Scene-Aware Dialog
Abstract
The recently proposed audio-visual scene-aware dialog task paves the way to a more data-driven way of learning virtual assistants, smart speakers and car navigation systems. However, very little is known to date about how to effectively extract meaningful information from a plethora of sensors that pound the computational engine of those devices. Therefore, in this paper, we provide and carefully analyze a simple baseline for audio-visual scene-aware dialog which is trained end-to-end. Our method differentiates in a data-driven manner useful signals from distracting ones using an attention mechanism. We evaluate the proposed approach on the recently introduced and challenging audio-visual scene-aware dataset, and demonstrate the key features that permit to outperform the current state-of-the-art by more than 20% on CIDEr.
Cite
Text
Schwartz et al. "A Simple Baseline for Audio-Visual Scene-Aware Dialog." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. doi:10.1109/CVPR.2019.01283Markdown
[Schwartz et al. "A Simple Baseline for Audio-Visual Scene-Aware Dialog." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.](https://mlanthology.org/cvpr/2019/schwartz2019cvpr-simple/) doi:10.1109/CVPR.2019.01283BibTeX
@inproceedings{schwartz2019cvpr-simple,
title = {{A Simple Baseline for Audio-Visual Scene-Aware Dialog}},
author = {Schwartz, Idan and Schwing, Alexander G. and Hazan, Tamir},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2019},
doi = {10.1109/CVPR.2019.01283},
url = {https://mlanthology.org/cvpr/2019/schwartz2019cvpr-simple/}
}