RP3D: A Roadside Perception Framework for 3D Object Detection via Multi-View Sensor Fusion

Abstract

Roadside perception is playing an increasingly important role in intelligent transportation systems (ITS), which can provide global object detection information for traffic dispatching and expand the sensing range of individual vehicles. However, existing approaches mainly focus on single-view or single-sensor perception, resulting in low perception accuracy and limited field of view in complex traffic scenarios. To solve such issues, this paper proposes a novel roadside perception framework for 3D object detection (RP3D) via multi-view cameras and LiDARs fusion. Firstly, a feature attention-guided lightweight 3D object detector is constructed for real-time object detection with multiple roadside LiDARs. Then, a 2D detector based on the efficient model NanoDet is adopted for multi-view image recognition. Moreover, a modified Hungarian algorithm is introduced to flexibly fuse multi-view and multi-sensor heterogeneous perception information from 2D and 3D detectors and improve detection accuracy. Furthermore, our proposal is deployed on a real-world V2X test field with four cameras and two LiDARs mounted on two roadside platforms. Experiments on DAIR-V2X-I and SCUT-V2R datasets demonstrate that the proposed method performs well in object detection accuracy and real-time performance in roadside perception scenes.