Beyond What's Normal: Bimodal and Heaviside Alternatives to Gaussian Process Regression

Abstract

Gaussian process regression is a fundamental tool of applied machine learning, valued for its analytical tractability and ability to quantify uncertainty. Its power stems from the advantageous properties of the Gaussian distribution, particularly the fact that its marginal and conditional distributions are also Gaussian. We present two further stochastic process regressors that meet these same stability criteria. Furthermore, these regressors provide useful properties that supplement those offered by Gaussian process regression, namely a bimodal predictive distribution and a finite-support predictive distribution respectively, and we illustrate the practical advantages these bring through example applications. Importantly, the proposed stochastic process regressors maintain computational complexity and analytical interpretability equivalent to that of Gaussian process regression. We also provide open-source implementations to facilitate adoption and further development by the research community.