Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors
Abstract
We present ``Magic123'', a two-stage coarse-to-fine approach for high-quality, textured 3D mesh generation from a single image in the wild using *both 2D and 3D priors*. In the first stage, we optimize a neural radiance field to produce a coarse geometry. In the second stage, we adopt a memory-efficient differentiable mesh representation to yield a high-resolution mesh with a visually appealing texture. In both stages, the 3D content is learned through reference-view supervision and novel-view guidance by a joint 2D and 3D diffusion prior. We introduce a trade-off parameter between the 2D and 3D priors to control the details and 3D consistencies of the generation. Magic123 demonstrates a significant improvement over previous image-to-3D techniques, as validated through extensive experiments on diverse synthetic and real-world images.
Cite
Text
Qian et al. "Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors." International Conference on Learning Representations, 2024.Markdown
[Qian et al. "Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors." International Conference on Learning Representations, 2024.](https://mlanthology.org/iclr/2024/qian2024iclr-magic123/)BibTeX
@inproceedings{qian2024iclr-magic123,
title = {{Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors}},
author = {Qian, Guocheng and Mai, Jinjie and Hamdi, Abdullah and Ren, Jian and Siarohin, Aliaksandr and Li, Bing and Lee, Hsin-Ying and Skorokhodov, Ivan and Wonka, Peter and Tulyakov, Sergey and Ghanem, Bernard},
booktitle = {International Conference on Learning Representations},
year = {2024},
url = {https://mlanthology.org/iclr/2024/qian2024iclr-magic123/}
}