An End-to-End Efficient Framework for Remote Physiological Signal Sensing

Abstract

Remote photoplethysmography (rPPG) is utilized to estimate the heart activities from videos, which has drawn great interest from both researchers and companies recently. Many existing rPPG deep-learning based approaches focus on measuring the average heart rate (HR) from facial videos, which do not provide enough detailed information for many applications. To recover more detailed rPPG signals for the challenge on Remote Physiological Signal Sensing (RePSS), we propose an end-to-end efficient framework, which measures the average heart rate and estimates corresponding Blood Volume Pulse (BVP) curves simultaneously. For efficiently extracting features containing rPPG information, we adopt the temporal and spatial convolution as Feature Extractor, which alleviates the cost of calculation. Then, BVP Estimation Network estimates the frame-level BVP signal based on the feature maps via a simple 1DCNN. To improve the learning of BVP Estimation Net-work, we further introduce Heartbeat Measuring Network to predict the video-level HR based on global rPPG information. These two networks facilitate each other via super-vising Feature Extractor from different level to promote the accuracy of BVP signal and HR. The proposed method obtains the score 168.08 (MIBI), winning the third place in this challenge.