Online Social Behavior Modeling for Multi-Target Tracking

Abstract

People are often seen together. We use this simple observation to provide crucial additional information and increase the robustness of a video tracker. The goal of this paper is to show how, in situations where offline training data is not available, a social behavior model (SBM) can be inferred online and then integrated within the tracking algorithm. We start with tracklets (short term confident tracks) obtained using an existing tracker. The SBM, a graphical model, captures the spatio-temporal relationships between the tracklets and is learned online from the video. The final probability of association between the tracklets is obtained by a combination of individual target characteristics (e.g., their appearance), as well as the learned relationship model between them. The entire system is causal whereby the results at any given time depend only upon the part of the video already observed. Experimental results on three state-of-the-art datasets show that, without having access to any offline training data or the entire test video a priori (conditions that may be restrictive for many application domains), our proposed method obtains results similar to those that do impose the above conditions.