A 2D-3D Object Detection System for Updating Building Information Models with Mobile Robots

Abstract

Automation in facility management and construction could significantly improve efficiency and productivity of the building industry. However, for robots and autonomous systems to operate effectively in dynamic and unstructured environments such as construction sites, they must be able to infer or obtain a semantic model of the environment. We propose a system for identifying common objects in a worksite and automatically adding them to an existing geometric model of the environment. The system is composed of two components: A novel 2D-3D object detection network designed to detect and localize a worksite objects, and a multi-object Kalman filter tracking system used to filter false-positive detections. The proposed system is validated using data collected with a purpose-built mobile robot. The mobile robot captures images of the worksite with an RGB-D camera and uses the proposed system to spawn newly placed objects in an existing geometric model of the worksite environment. The annotated RGB-D images are made publicly available to accelerate future research in this field.