Multiple Object Detection and Tracking in the Thermal Spectrum

Abstract

Multiple Object Tracking (MOT) is an integral part of machine vision research. Most tracking-by-detection based MOT solutions utilize video streams from RGB cameras for their operation. However, for real-world applications, it is necessary to utilize sensors that operate in different spectrums to accommodate for varying lighting conditions. Since object detection is the first step of the tracking pipeline in tracking-by-detection approaches, we compare the performance of state-of-the-art object detectors when trained on color images to their performance when trained on thermal images. We introduce a new dataset for multiple object tracking with thermal images and corresponding RGB images and show that state-of-the-art trackers perform better on thermal images, especially in poor lighting conditions. Finally, we propose the use of a dynamic cut-off thresh-old for tracking-by-detection approaches that factors the size of a predicted box to enhance the tracker association. Our dataset and source code are publicly available at https://github.com/wassimea/thermalMOT