Functional Tensors for Probabilistic Programming

Abstract

It is a significant challenge to design probabilistic programming systems that can accommodate a wide variety of inference strategies within a unified framework. Noting that the versatility of modern automatic differentiation frameworks is based in large part on the unifying concept of tensors, we describe a software abstraction, functional tensors, that captures many of the benefits of tensors, while also being able to describe continuous probability distributions. We demonstrate the versatility of functional tensors by integrating them into the modeling frontend and inference backend of the Pyro probabilistic programming language. As an example application, we perform approximate inference on a switching linear dynamical system.