ToF-360 - A Panoramic Time-of-Flight RGB-D Dataset for Single Capture Indoor Semantic 3D Reconstruction

Abstract

3D scene understanding is a key research topic for various automation areas. Many RGB-D datasets today focus on reconstruction of entire scenes. However, their scanning processes are time-consuming, requiring multiple or continuous recordings using a scanner with a limited angle of view. Such datasets often contain data affected by stitching artifacts or poor quality annotation masks projected directly from 3D to image. In this paper, we present ToF-360. This is the first RGB-D dataset obtained by a unique Time-of-Flight (ToF) sensor capable of 360deg omnidirectional RGB-D scanning within seconds. In addition to the raw data in a fisheye format and equi-rectangular projection (ERP) images from the device, we provide manually labeled high-quality, pixel-level, 2D semantics and room layout annotations and introduce a benchmark for three practical tasks: 2D semantic segmentation, 3D semantic segmentation, and layout estimation. We demonstrate that our dataset helps to better represent real-world scenarios and push the limits of existing state-of-the-art methods. The dataset is publicly available at https://doi.org/10.57967/hf/5074