DreamDrone: Text-to-Image Diffusion Models Are Zero-Shot Perpetual View Generators
Abstract
We introduce DreamDrone, a novel zero-shot and training-free pipeline for generating unbounded flythrough scenes from textual prompts. Different from other methods that focus on warping images frame by frame, we advocate explicitly warping the intermediate latent code of the pre-trained text-to-image diffusion model for high-quality image generation and generalization ability. To further enhance the fidelity of the generated images, we also propose a feature-correspondence-guidance diffusion process and a high-pass filtering strategy to promote geometric consistency and high-frequency detail consistency, respectively. Extensive experiments reveal that DreamDrone significantly surpasses existing methods, delivering highly authentic scene generation with exceptional visual quality, without training or fine-tuning on datasets or reconstructing 3D point clouds in advance.
Cite
Text
Kong et al. "DreamDrone: Text-to-Image Diffusion Models Are Zero-Shot Perpetual View Generators." Proceedings of the European Conference on Computer Vision (ECCV), 2024. doi:10.1007/978-3-031-72624-8_19Markdown
[Kong et al. "DreamDrone: Text-to-Image Diffusion Models Are Zero-Shot Perpetual View Generators." Proceedings of the European Conference on Computer Vision (ECCV), 2024.](https://mlanthology.org/eccv/2024/kong2024eccv-dreamdrone/) doi:10.1007/978-3-031-72624-8_19BibTeX
@inproceedings{kong2024eccv-dreamdrone,
title = {{DreamDrone: Text-to-Image Diffusion Models Are Zero-Shot Perpetual View Generators}},
author = {Kong, Hanyang and Lian, Dongze and Mi, Michael Bi and Wang, Xinchao},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2024},
doi = {10.1007/978-3-031-72624-8_19},
url = {https://mlanthology.org/eccv/2024/kong2024eccv-dreamdrone/}
}