Gaze Estimation from Multimodal Kinect Data

Abstract

This paper addresses the problem of free gaze estima-tion under unrestricted head motion. More precisely, unlike previous approaches that mainly focus on estimating gaze towards a small planar screen, we propose a method to es-timate the gaze direction in the 3D space. In this context the paper makes the following contributions: (i) leverag-ing on Kinect device, we propose a multimodal method that rely on depth sensing to obtain robust and accurate head pose tracking even under large head pose, and on the visual data to obtain the remaining eye-in-head gaze directional information from the eye image; (ii) a rectification scheme of the image that exploits the 3D mesh tracking, allowing to conduct a head pose free eye-in-head gaze directional esti-mation; (iii) a simple way of collecting ground truth data thanks to the Kinect device. Results on three users demon-strate the great potential of our approach. 1.