Extending Long Short-Term Memory for Multi-View Structured Learning
Abstract
Long Short-Term Memory (LSTM) networks have been successfully applied to a number of sequence learning problems but they lack the design flexibility to model multiple view interactions, limiting their ability to exploit multi-view relationships. In this paper, we propose a Multi-View LSTM (MV-LSTM), which explicitly models the view-specific and cross-view interactions over time or structured outputs. We evaluate the MV-LSTM model on four publicly available datasets spanning two very different structured learning problems: multimodal behaviour recognition and image captioning. The experimental results show competitive performance on all four datasets when compared with state-of-the-art models.
Cite
Text
Rajagopalan et al. "Extending Long Short-Term Memory for Multi-View Structured Learning." European Conference on Computer Vision, 2016. doi:10.1007/978-3-319-46478-7_21Markdown
[Rajagopalan et al. "Extending Long Short-Term Memory for Multi-View Structured Learning." European Conference on Computer Vision, 2016.](https://mlanthology.org/eccv/2016/rajagopalan2016eccv-extending/) doi:10.1007/978-3-319-46478-7_21BibTeX
@inproceedings{rajagopalan2016eccv-extending,
title = {{Extending Long Short-Term Memory for Multi-View Structured Learning}},
author = {Rajagopalan, Shyam Sundar and Morency, Louis-Philippe and Baltrusaitis, Tadas and Goecke, Roland},
booktitle = {European Conference on Computer Vision},
year = {2016},
pages = {338-353},
doi = {10.1007/978-3-319-46478-7_21},
url = {https://mlanthology.org/eccv/2016/rajagopalan2016eccv-extending/}
}