Multi-View Reprojection Architecture for Orientation Estimation

Abstract

In autonomous driving scenarios, pose estimation of surrounding vehicles and other objects is required in decision making and planning. This paper proposes Multi-View Reprojection Architecture, a flexible, highly accurate architecture to adopt any 2D detection network and extend it to regress the orientation of object's 3D bounding box and its dimensions. In contrast to previous techniques, our network incorporates geometric constraints on 3D box imposed by 2D detection box. In particular, we regress dimensions, orientation and reprojected boxes in multi-view obtained from novel 3D reconstruction layer using perspective geometry. In 3D reconstruction layer, we use an iterative refinement strategy to accurately recover 3D boxes even when 2D boxes are truncated. The proposed architecture is shown to outperform state-of-the-art methods on the challenging KITTI car orientation benchmark and obtain top results on 3D detection benchmark while running in real time, making it suitable for autonomous vehicles.