Joint Object Class Sequencing and Trajectory Triangulation (JOST)

Abstract

We introduce the problem of joint object class sequencing and trajectory triangulation (JOST), which is defined as the reconstruction of the motion path of a class of dynamic objects through a scene from an unordered set of images. We leverage standard object detection techinques to identify object instances within a set of registered images. Each of these object detections defines a single 2D point with a corresponding viewing ray. The set of viewing rays attained from the aggregation of all detections belonging to a common object class is then used to estimate a motion path denoted as the object class trajectory. Our method jointly determines the topology of the objects motion path and reconstructs the 3D object points corresponding to our object detections. We pose the problem as an optimization over both the unknown 3D points and the topology of the path, which is approximated by a Generalized Minimum Spanning Tree (GMST) on a multipartite graph and then refined through a continuous optimization over the 3D object points. Experiments on synthetic and real datasets demonstrate the effectiveness of our method and the feasibility to solve a previously intractable problem.