Variational Optimization Based Reinforcement Learning for Infinite Dimensional Stochastic Systems

Abstract

Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.