Modelling Latent Dynamical Systems with Recognition-Parametrised Models

Abstract

We introduce a new approach to learning latent Markovian dynamical processes underlying observed time series data: the recognition-parametrised latent dynamical system (RP-LDS). The RP-LDS resolves issues in two broad classes of state-of-the-art latent time series models, while maintaining expressivity through a complex neural network-based link between observations and latents. As opposed to *generative* or auto-encoding approaches, the RP-LDS does not learn an explicit model reconstructing observations from latents, thus allowing it to avoid parameter bias and focus model capacity on recognition. As opposed to *contrastive* approaches, the RP-LDS utilises efficient message-passing to propagate posterior uncertainty and achieve maximum-likelihood learning. The RP-LDS matches the performance of state-of-the-art methods on both linear and nonlinear toy problems. We apply the RP-LDS to video of a swinging pendulum with background distractors and show that it is able to recover the underlying latent system despite not being in model class.