Likelihood-Free Inference with Deep Gaussian Processes

Abstract

In recent years, surrogate models have been successfully used in likelihood-free inference to decrease the number of simulator evaluations. The most data-efficient solution for this task has been achieved by Bayesian Optimization with Gaussian Processes (GPs). While this combination works well for unimodal target distributions, it appears restrictive in more irregular cases. On the other hand, neural network approaches are extremely adaptable given sufficient data, which are rarely available when working with computationally expensive simulators. In this extended abstract, we address a trade-off between data-efficiency and flexibility by proposing a Deep Gaussian Process (DGP) surrogate model that can handle more irregularly behaved target distributions with few simulator evaluations. Our experiments show how DGPs can outperform GPs on objective functions with multimodal distributions, maintaining a comparable performance in unimodal cases. At the same time, DGPs in general require much fewer data to achieve the same performance as Mixture Density Networks and Masked Autoregressive Flows. This confirms that DGPs as surrogate models for Bayesian Optimization provide a good tradeoff between data-efficiency and flexibility for likelihood-free inference with computationally intensive simulators.