Photorealistic Text-to-3D Avatar Generation with Constrained Geometry and Appearance

Abstract

Powered by large-scale text-to-image generation models, text-to-3D avatar generation has made promising progress. However, most methods fail to produce photorealistic results. Towards more practical avatar generation, we present SEEAvatar, a method for generating photorealistic 3D avatars from text with SElf-Evolving constraints for decoupled geometry and appearance. For geometry generation, we propose to constrain the optimized avatar with the evolving template avatar for global shape constraint and the static human prior for local part maintenance. For appearance generation, we use physically based rendering pipeline to generate realistic textures, with the lightness constraint applied on albedo textures to suppress incorrect lighting effects. Experiments show that our method outperforms previous methods on both global and local geometry and appearance quality by a large margin. Since our method can produce high-quality meshes and textures, such assets can be directly applied in classic graphics pipelines for practical applications. Project page at: https://yoxu515.github.io/SEEAvatar/ .