latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction
Abstract
We present latentSplat, a method to predict semantic Gaussians in a 3D latent space that can be splatted and decoded by a light-weight generative 2D architecture. Existing methods for generalizable 3D reconstruction either do not scale to large scenes and resolutions, or are limited to interpolation of close input views. latentSplat combines the strengths of regression-based and generative approaches while being trained purely on readily available real video data. The core of our method are variational 3D Gaussians, a representation that efficiently encodes varying uncertainty within a latent space consisting of 3D feature Gaussians. From these Gaussians, specific instances can be sampled and rendered via efficient splatting and a fast, generative decoder. We show that latentSplat outperforms previous works in reconstruction quality and generalization, while being fast and scalable to high-resolution data.
Cite
Text
Wewer et al. "latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction." Proceedings of the European Conference on Computer Vision (ECCV), 2024. doi:10.1007/978-3-031-73021-4_27Markdown
[Wewer et al. "latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction." Proceedings of the European Conference on Computer Vision (ECCV), 2024.](https://mlanthology.org/eccv/2024/wewer2024eccv-latentsplat/) doi:10.1007/978-3-031-73021-4_27BibTeX
@inproceedings{wewer2024eccv-latentsplat,
title = {{latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction}},
author = {Wewer, Christopher and Raj, Kevin and Ilg, Eddy and Schiele, Bernt and Lenssen, Jan E.},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2024},
doi = {10.1007/978-3-031-73021-4_27},
url = {https://mlanthology.org/eccv/2024/wewer2024eccv-latentsplat/}
}