Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis

Abstract

Generalizable 3D Gaussian splitting (3DGS) can reconstruct new scenes from sparse-view observations in a feed-forward inference manner, eliminating the need for scene-specific retraining required in conventional 3DGS. However, existing methods rely heavily on epipolar priors, which can be unreliable in complex real-world scenes, particularly in non-overlapping and occluded regions. In this paper, we propose eFreeSplat, an efficient feed-forward 3DGS-based model for generalizable novel view synthesis that operates independently of epipolar line constraints. To enhance multiview feature extraction with 3D perception, we employ a self-supervised Vision Transformer (ViT) with cross-view completion pre-training on large-scale datasets. Additionally, we introduce an Iterative Cross-view Gaussians Alignment method to ensure consistent depth scales across different views. Our eFreeSplat represents a new paradigm for generalizable novel view synthesis. We evaluate eFreeSplat on wide-baseline novel view synthesis tasks using the RealEstate10K and ACID datasets. Extensive experiments demonstrate that eFreeSplat surpasses state-of-the-art baselines that rely on epipolar priors, achieving superior geometry reconstruction and novel view synthesis quality.

Cite

Text

Min et al. "Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis." Neural Information Processing Systems, 2024. doi:10.52202/079017-1251

Markdown

[Min et al. "Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis." Neural Information Processing Systems, 2024.](https://mlanthology.org/neurips/2024/min2024neurips-epipolarfree/) doi:10.52202/079017-1251

BibTeX

@inproceedings{min2024neurips-epipolarfree,
  title     = {{Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis}},
  author    = {Min, Zhiyuan and Luo, Yawei and Sun, Jianwen and Yang, Yi},
  booktitle = {Neural Information Processing Systems},
  year      = {2024},
  doi       = {10.52202/079017-1251},
  url       = {https://mlanthology.org/neurips/2024/min2024neurips-epipolarfree/}
}