Local Group Invariance for Heart Rate Estimation from Face Videos in the Wild

Abstract

We study the impact of prior knowledge about invariance for the task of heart rate estimation from face videos in the wild (e.g. in presence of disturbing factors like rigid head motion, talking, facial expressions and natural illumination conditions under different scenarios). We introduce features invariant with respect to the action of a differentiable local group of local transformations. As result, the energy of the blood volume signal is re-arranged in vector space with a more concentrated distribution. The uncertainty in the feature distribution is incorporated with a model that leverages the local invariance of the heart rate. During experiments the method achieved strong estimation performance of heart rate from face videos in the wild. To demonstrate the potential of the approach it is compared against recent algorithms on data collected to study the impact of the mentioned nuisance attributes. To facilitate future comparisons, we made the code and data for reproducing the results publicly available.