Atomic Scenes for Scalable Traffic Scene Recognition in Monocular Videos

Abstract

The efficacy of advance warning systems (AWS) in automobiles can be significantly enhanced by semantic recognition of traffic scenes that pose a potential danger. However, the complexity of road scenes and the need for real-time solutions pose key challenges. This paper proposes a novel framework for monocular traffic scene recognition, relying on a decomposition into high-order and atomic scenes to meet those challenges. High-order scenes carry semantic meaning useful for AWS applications, while atomic scenes are easy to learn and represent elemental behaviors based on 3D localization of individual traffic participants. Atomic scenes allow our framework to be scalable, since a few of them combine to influence prediction for a wide array of high-order scenes. We propose a novel hierarchical model that captures co-occurence and mutual exclusion relationships while incorporating both low-level trajectory features and high-level scene features, with parameters learned using a structured support vector machine. We propose efficient inference that exploits the structure of our model to obtain real-time rates. Further, we demonstrate experiments in a large-scale dataset for scene recognition that consists of challenging traffic videos of inner-city scenes with ground truth annotations of scene types and object bounding boxes, as well as state-of-the-art 3D object localization outputs. Our experiments show the advantages of our approach relative to several baselines on a novel Inner-City dataset.