Learning to Drive from a World Model
Abstract
Most self-driving systems rely on hand-coded perception outputs and engineered driving rules. Learning directly from human driving data with an end-to-end method can allow for a training architecture that is simpler and scales well with compute and data. In this work, we propose an end-to-end training architecture that uses real driving data to train a driving policy in an on-policy simulator. We show two different methods of simulation, one with reprojective simulation and one with a learned world model. We show that both methods can be used to train a policy that learns driving behavior without any hand-coded driving rules. We evaluate the performance of these policies in a closed-loop simulation and when deployed in a real-world advanced driver-assistance system.
Cite
Text
Goff et al. "Learning to Drive from a World Model." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2025.Markdown
[Goff et al. "Learning to Drive from a World Model." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2025.](https://mlanthology.org/cvprw/2025/goff2025cvprw-learning/)BibTeX
@inproceedings{goff2025cvprw-learning,
title = {{Learning to Drive from a World Model}},
author = {Goff, Mitchell and Hogan, Greg and Hotz, George and du Parc Locmaria, Armand and Raczy, Kacper and Schäfer, Harald and Shihadeh, Adeeb and Zhang, Weixing and Yousfi, Yassine},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
year = {2025},
pages = {1964-1973},
url = {https://mlanthology.org/cvprw/2025/goff2025cvprw-learning/}
}