Variational Bayes Gaussian Splatting

Abstract

3D Gaussian Splatting has shown that mixture models can be used to represent high-dimensional data, such as 3D scene representations. Currently, the most prevalent method for optimizing these models is by backpropagating gradients of an image reconstruction loss through a differentiable rendering pipeline. These methods are susceptible to catastrophic forgetting in many real-world situations, where data is continually gathered through sensory observations. This paper proposes Variational Bayes Gaussian Splatting (VBGS), where we cast learning as variational inference over model parameters. Through conjugacy of the multivariate Gaussian, we find a closed-form update rule for the variational posterior, which allows us to continually apply updates from partial data, using only a single update step for each observation.