Are We Friends? End-to-End Prediction of Child Rapport in Guided Play

Abstract

Close, fulfilling interactions with other classmates are an important part of a child’s learning experience, and have been shown to improve educational outcomes. This is especially apparent in guided play activities in which children need to coordinate to succeed. However, until the recent publication of the UpStory dataset, no child-child interaction dataset with explicit control for rapport was available, leading to a lack of methods for automatic rapport prediction in pair play. In this study, we perform the first-of-its-kind evaluation of end-to-end Computer Vision techniques for child-child rapport prediction, and compare our Deep Learning-based results to the feature-based Machine Learning approaches reported in the UpStory paper. The results show that, under a thorough training and evaluation procedure, end-to-end learning under-performs when compared to feature-based methods.