Multiple-Weight Recurrent Neural Networks
Abstract
Recurrent neural networks (RNNs) have enjoyed great success in speech recognition, natural language processing, etc. Many variants of RNNs have been proposed, including vanilla RNNs, LSTMs, and GRUs. However, current architectures are not particularly adept at dealing with tasks involving multi-faceted contents. In this work, we solve this problem by proposing Multiple-Weight RNNs and LSTMs, which rely on multiple weight matrices in an attempt to mimic the human ability of switching between contexts. We present a framework for adapting RNN-based models and analyze the properties of this approach. Our detailed experimental results show that our model outperforms previous work across a range of different tasks and datasets.
Cite
Text
Cao et al. "Multiple-Weight Recurrent Neural Networks." International Joint Conference on Artificial Intelligence, 2017. doi:10.24963/IJCAI.2017/205Markdown
[Cao et al. "Multiple-Weight Recurrent Neural Networks." International Joint Conference on Artificial Intelligence, 2017.](https://mlanthology.org/ijcai/2017/cao2017ijcai-multiple/) doi:10.24963/IJCAI.2017/205BibTeX
@inproceedings{cao2017ijcai-multiple,
title = {{Multiple-Weight Recurrent Neural Networks}},
author = {Cao, Zhu and Wang, Linlin and de Melo, Gerard},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2017},
pages = {1483-1489},
doi = {10.24963/IJCAI.2017/205},
url = {https://mlanthology.org/ijcai/2017/cao2017ijcai-multiple/}
}