4D-LRM: Large Space-Time Reconstruction Model from and to Any View at Any Time
Abstract
Can we scale 4D pretraining to learn general space-time representations that reconstruct an object from a few views at some times to any view at any time? We provide an affirmative answer with 4D-LRM, the first large-scale 4D reconstruction model that takes input from unconstrained views and timestamps and renders arbitrary novel view-time combinations. Unlike prior 4D approaches, e.g., optimization-based, geometry-based, or generative, that struggle with efficiency, generalization, or faithfulness, 4D-LRM learns a unified space-time representation and directly predicts per-pixel 4D Gaussian primitives from posed image tokens across time, enabling fast, high-quality rendering at, in principle, infinite frame rate. Our results demonstrate that scaling spatiotemporal pretraining enables accurate and efficient 4D reconstruction. We show that 4D-LRM generalizes to novel objects, interpolates across time, and handles diverse camera setups. It reconstructs 24-frame sequences in one forward pass with less than 1.5 seconds on a single A100 GPU.
Cite
Text
Ma et al. "4D-LRM: Large Space-Time Reconstruction Model from and to Any View at Any Time." Advances in Neural Information Processing Systems, 2025.Markdown
[Ma et al. "4D-LRM: Large Space-Time Reconstruction Model from and to Any View at Any Time." Advances in Neural Information Processing Systems, 2025.](https://mlanthology.org/neurips/2025/ma2025neurips-4dlrm/)BibTeX
@inproceedings{ma2025neurips-4dlrm,
title = {{4D-LRM: Large Space-Time Reconstruction Model from and to Any View at Any Time}},
author = {Ma, Ziqiao and Chen, Xuweiyi and Yu, Shoubin and Bi, Sai and Zhang, Kai and Ziwen, Chen and Xu, Sihan and Yang, Jianing and Xu, Zexiang and Sunkavalli, Kalyan and Bansal, Mohit and Chai, Joyce and Tan, Hao},
booktitle = {Advances in Neural Information Processing Systems},
year = {2025},
url = {https://mlanthology.org/neurips/2025/ma2025neurips-4dlrm/}
}