Implementing Human Information-Seeking Behaviour with Action-Agnostic Bayesian Surprise

Abstract

In this paper, we aim to establish a link between model learning and the mechanism of curiosity. The main hypothesis developed is that exploration bonuses, as proposed in the reinforcement learning literature, are linked to Bayesian estimation principles through the construction of a parametric model of the causal relationships between actions and observations. At odd with the classic action-conditional Bayesian surprise widely used in the "curiosity" literature, action is here treated as an external variable, unknowingly of the agent's own control policy. It is thus called the "agnostic" Bayesian surprise (ABS), interpreted as an estimate of the information transfer between the observed data (including observations and motor commands) and the model parameters. We present here the general guidelines of this approach, and show results suggesting that action selection guided by information transfer can account for certain experimental, behavioral, and neurological data in humans.