Multi-Modal Information Fusion for Action Unit Detection in the Wild
Abstract
Action Unit (AU) detection is an important research branch in affective computing, which better understands human emotional intentions and responds more naturally to their needs and desires. In this paper, we present our latest progress techniques in the 5th Affective Behavior Analysis in-the-wild (ABAW) competition, including data balancing by marking, extracting features visual through models trained in face database and audio through deep networks and traditional methods, proposing model structures for mapping multimodal information to a unify multimodal vector space and fusing results from multiple models. These methods are effective on the official validation dataset of the Aff-Wild2. The final F1 in the 5th ABAW competition test dataset achieves 54.22%, 4.33% higher than the best results in the 3rd ABAW competition.
Cite
Text
Deng et al. "Multi-Modal Information Fusion for Action Unit Detection in the Wild." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023. doi:10.1109/CVPRW59228.2023.00622Markdown
[Deng et al. "Multi-Modal Information Fusion for Action Unit Detection in the Wild." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023.](https://mlanthology.org/cvprw/2023/deng2023cvprw-multimodal/) doi:10.1109/CVPRW59228.2023.00622BibTeX
@inproceedings{deng2023cvprw-multimodal,
title = {{Multi-Modal Information Fusion for Action Unit Detection in the Wild}},
author = {Deng, Yuanyuan and Liu, Xiaolong and Meng, Liyu and Jiang, Wenqiang and Dong, Youqiang and Liu, Chuanhe},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
year = {2023},
pages = {5855-5862},
doi = {10.1109/CVPRW59228.2023.00622},
url = {https://mlanthology.org/cvprw/2023/deng2023cvprw-multimodal/}
}