Multi-Modal Multi-Label Emotion Recognition with Heterogeneous Hierarchical Message Passing
Abstract
As an important research issue in affective computing community, multi-modal emotion recognition has become a hot topic in the last few years. However, almost all existing studies perform multiple binary classification for each emotion with focus on complete time series data. In this paper, we focus on multi-modal emotion recognition in a multi-label scenario. In this scenario, we consider not only the label-to-label dependency, but also the feature-to-label and modality-to-label dependencies. Particularly, we propose a heterogeneous hierarchical message passing network to effectively model above dependencies. Furthermore, we propose a new multi-modal multi-label emotion dataset based on partial time-series content to show predominant generalization of our model. Detailed evaluation demonstrates the effectiveness of our approach.
Cite
Text
Zhang et al. "Multi-Modal Multi-Label Emotion Recognition with Heterogeneous Hierarchical Message Passing." AAAI Conference on Artificial Intelligence, 2021. doi:10.1609/AAAI.V35I16.17686Markdown
[Zhang et al. "Multi-Modal Multi-Label Emotion Recognition with Heterogeneous Hierarchical Message Passing." AAAI Conference on Artificial Intelligence, 2021.](https://mlanthology.org/aaai/2021/zhang2021aaai-multi/) doi:10.1609/AAAI.V35I16.17686BibTeX
@inproceedings{zhang2021aaai-multi,
title = {{Multi-Modal Multi-Label Emotion Recognition with Heterogeneous Hierarchical Message Passing}},
author = {Zhang, Dong and Ju, Xincheng and Zhang, Wei and Li, Junhui and Li, Shoushan and Zhu, Qiaoming and Zhou, Guodong},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2021},
pages = {14338-14346},
doi = {10.1609/AAAI.V35I16.17686},
url = {https://mlanthology.org/aaai/2021/zhang2021aaai-multi/}
}