Regularized Parameter Uncertainty for Improving Generalization in Reinforcement Learning
Abstract
In order for reinforcement learning (RL) agents to be deployed in real-world environments they must be able to generalize to unseen environments. However RL struggles with out-of-distribution generalization often due to over-fitting the particulars of the training environment. Although regularization techniques from supervised learning can be applied to avoid over-fitting the differences between supervised learning and RL limit their application. To address this we propose the Signal-to-Noise Ratio regulated Parameter Uncertainty Network (SNR PUN) for RL. We introduce SNR as a new measure of regularizing the parameter uncertainty of a network and provide a formal analysis explaining why SNR regularization works well for RL. We demonstrate the effectiveness of our proposed method to generalize in several simulated environments; and in a physical system showing the possibility of using SNR PUN for applying RL to real-world applications.
Cite
Text
Moure et al. "Regularized Parameter Uncertainty for Improving Generalization in Reinforcement Learning." Conference on Computer Vision and Pattern Recognition, 2024. doi:10.1109/CVPR52733.2024.02247Markdown
[Moure et al. "Regularized Parameter Uncertainty for Improving Generalization in Reinforcement Learning." Conference on Computer Vision and Pattern Recognition, 2024.](https://mlanthology.org/cvpr/2024/moure2024cvpr-regularized/) doi:10.1109/CVPR52733.2024.02247BibTeX
@inproceedings{moure2024cvpr-regularized,
title = {{Regularized Parameter Uncertainty for Improving Generalization in Reinforcement Learning}},
author = {Moure, Pehuen and Cheng, Longbiao and Ott, Joachim and Wang, Zuowen and Liu, Shih-Chii},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2024},
pages = {23805-23814},
doi = {10.1109/CVPR52733.2024.02247},
url = {https://mlanthology.org/cvpr/2024/moure2024cvpr-regularized/}
}