Towards Autonomous Mining via Intelligent Excavators

Abstract

In this paper we present our first step solution towards global challenge of safety, productivity, profitability and energy-efficiency in mining. Our solution (intelligent excavator) provides complete monitoring solution for excavators that relies on deep neural networks to produce accurate, actionable data for mine. Our solution helps mines to increase shovel efficiency, reduce unexpected downtime cost, enable planned maintenance. We use a multi-frame convolutional LSTM-based object detection approach to accumulate valuable information across video frames without significant computational overhead. Our experiments on dataset captured in several mines across the world show that we can detect objects of interest with accuracy of more than 90% on 10 FPS. Furthermore, we show that our approach generalizes well to mining sites and equipment types not encountered in our training set. Finally, our work on detecting the types of objects encountered in a mining equipment could be used as a first step in developing a perception module that could provide autonomous excavators with the required knowledge of their environment in order to make optimal decisions.