Sequential Bayesian Optimisation for Spatial-Temporal Monitoring

Abstract

Bayesian Optimisation has received considerable attention in recent years as a general methodol-ogy to find the maximum of costly-to-evaluate objective functions. Most existing BO work fo-cuses on where to gather a set of samples with-out giving special consideration to the sampling sequence, or the costs or constraints associated with that sequence. However, in real-world sequential decision problems such as robotics, the order in which samples are gathered is paramount, especially when the robot needs to optimise a temporally non-stationary objective function. Additionally, the state of the environ-ment and sensing platform determine the type and cost of samples that can be gathered. To address these issues, we formulate Sequential Bayesian Optimisation (SBO) with side-state in-formation within a Partially Observed Markov Decision Process (POMDP) framework that can accommodate discrete and continuous observa-tion spaces. We build on previous work using Monte-Carlo Tree Search (MCTS) and Upper Confidence bound for Trees (UCT) for POMDPs and extend it to work with continuous state and observation spaces. Through a series of experi-ments on monitoring a spatial-temporal process with a mobile robot, we show that our UCT-based SBO POMDP optimisation outperforms myopic and non-myopic alternatives. 1