Balanced Reward-Inspired Reinforcement Learning for Autonomous Vehicle Racing

Abstract

Autonomous vehicle racing has attracted extensive interest due to its great potential in autonomous driving at the extreme limits. Model-based and learning-based methods are being widely used in autonomous racing. However, model-based methods cannot cope with the dynamic environments when only local perception is available. As a comparison, learning-based methods can handle complex environments under local perception. Recently, deep reinforcement learning (DRL) has gained popularity in autonomous racing. DRL outperforms conventional learning- based methods by handling complex situations and leveraging local information. DRL algorithms, such as the proximal policy algorithm, can achieve a good balance between the execution time and safety in autonomous vehicle competition. However, the training outcomes of conventional DRL methods exhibit inconsistent correctness in decision-making. The instability in decision-making introduces safety concerns in autonomous vehicle racing, such as collisions into track boundaries. The proposed algorithm is capable to avoid collisions and improve the training quality. Simulation results on a physical engine demonstrate that the proposed algorithm outperforms other DRL algorithms in achieving safer control during sharp bends, fewer collisions into track boundaries, and higher training quality among multiple tracks.