EmotiEffNet and Temporal Convolutional Networks in Video-Based Facial Expression Recognition and Action Unit Detection
Abstract
This paper examines the video-based facial expression recognition and action unit detection tasks. We propose to use pre-trained EmotiEffNet models for frame-level facial feature extraction and feed them into the Temporal Convolutional Networks to take into account the dynamics of facial expressions. In addition, we study the possibility of combining facial processing with audio feature extraction to improve the accuracy of audio-visual expression recognition. Experimental results for two tasks from the sixth Affective Behavior Analysis in-the-Wild challenge demonstrate that our approach lets us significantly improve quality metrics on validation sets compared to existing non-ensemble techniques. As a result, our approach took third place in the action unit detection and fourth place in the expression recognition.
Cite
Text
Savchenko and Sidorova. "EmotiEffNet and Temporal Convolutional Networks in Video-Based Facial Expression Recognition and Action Unit Detection." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024. doi:10.1109/CVPRW63382.2024.00488Markdown
[Savchenko and Sidorova. "EmotiEffNet and Temporal Convolutional Networks in Video-Based Facial Expression Recognition and Action Unit Detection." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024.](https://mlanthology.org/cvprw/2024/savchenko2024cvprw-emotieffnet/) doi:10.1109/CVPRW63382.2024.00488BibTeX
@inproceedings{savchenko2024cvprw-emotieffnet,
title = {{EmotiEffNet and Temporal Convolutional Networks in Video-Based Facial Expression Recognition and Action Unit Detection}},
author = {Savchenko, Andrey V. and Sidorova, Anna P.},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
year = {2024},
pages = {4849-4859},
doi = {10.1109/CVPRW63382.2024.00488},
url = {https://mlanthology.org/cvprw/2024/savchenko2024cvprw-emotieffnet/}
}