DreamTexture: High-Fidelity Synthetic 3D Data Generation Through Decoupled Geometry and Texture Synthesis

Abstract

Recent single-image to 3D generation methods commonly adopt multi-view diffusion and large reconstruction models to achieve fast 3D content generation. Despite the impressive generation speed and geometrical consistency, there are multiple deficiencies including texture distortion, color deviation, and insufficient resolution. To address these deficiencies, we present DreamTexture, a high-fidelity method for decoupling geometry and texture synthesis in two stages. The main idea is to combine the powerful geometrical generation capability of the large multi-view Gaussian model with the texture alignment ability of the synchronized multi-view diffusion strategy. To synthesize subject-driven personalized textures for decorating the 3D object, we train a personalized diffusion model to generate subject-driven multi-view images. These images are then mapped to the texture domain to synthesize high-resolution textures. To improve the quality of multi-view images for further optimizing texture details, we instantiate two scaling factors to re-balance the contributions of backbone features and skip features in personalized diffusion. Experimental results on public datasets demonstrate that DreamTexture significantly outperforms the latest state-of-the-art methods, both qualitatively and quantitatively. Notably, our framework can generate high-fidelity 3D assets with detailed textures from scratch, featuring remarkable training scalability.