Facial Affect Recognition Using Semi-Supervised Learning with Adaptive Threshold

Abstract

Automatic facial affect recognition has wide applications in areas like education, gaming, software development, automotives, medical care, etc. but it is non trivial task to achieve appreciable performance on in-the-wild data sets. Though these datasets represent real-world scenarios better than in-lab data sets, they suffer from the problem of incomplete labels due to difficulty in annotation. Inspired by semi-supervised learning, this paper presents our submission to the Multi-Task-Learning (MTL) Challenge and Learning from Synthetic Data (LSD) Challenge at the 4th Affective Behavior Analysis in-the-wild (ABAW) 2022 Competition. The three tasks that are considered in MTL challenge are valence-arousal estimation, classification of expressions into basic emotions and detection of action units. Our method Semi-supervised Learning based Multi-task Facial Affect Recognition titled SS-MFAR uses a deep residual network as backbone along with task specific classifiers for each of the tasks. It uses adaptive thresholds for each expression class to select confident samples using semi-supervised learning from samples with incomplete labels. The performance is validated on challenging s-Aff-Wild2 dataset. Source code is available at https://github.com/1980x/ABAW2022DMACS .