Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Reward Signals

Abstract

Connector insertion and many other tasks com- monly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related physical ef- fects with first-order modeling, traditional control methodologies often result in brittle and inaccu- rate controllers, which have to be manually tuned. Reinforcement learning (RL) methods have been demonstrated to be capable of learning controllers in such environments from autonomous interac- tion with the environment, but running RL algo- rithms in the real world poses sample efficiency and safety challenges. Moreover, in practical real- world settings we cannot assume access to perfect state information or dense reward signals. In this paper we consider a variety of difficult industrial insertion tasks with visual inputs and different natural reward specifications, namely sparse re- wards and goal images. We show that methods that combine RL with prior information, such as classical controllers or demonstrations, can solve these tasks directly by real-world interaction.