用图像提示生成高保真3D人像,身份一致且细节丰富。
GaussianIP: Identity-Preserving Realistic 3D Human Generation via Human-Centric Diffusion Prior
- 分两阶段生成:先快速对齐身份,再细化面部和衣物细节。
- 仅需少量训练步数即达到高视觉质量,优于传统方法。
- 适合需要真实感3D人像生成的研究者与创作者。
文本引导的3D人体生成得益于高效的3D表示和2D提升方法(如Score Distillation Sampling, SDS)。然而,现有方法训练时间长,且常缺乏精细的面部与服装细节。本文提出GaussianIP,一种基于图像提示生成身份一致、高保真3D人体的两阶段框架。核心思想是利用以人为中心的知识促进生成过程。第一阶段提出自适应人体蒸馏采样(AHDS),快速生成与图像提示高度一致、外观逼真的3D人体,相比传统SDS方法更契合人本生成流程,显著减少训练步数并提升视觉质量。第二阶段设计视图一致性精炼(VCR)策略,通过多视角图像迭代优化,利用互注意力与距离引导的注意力融合,确保各视角纹理一致性,最终直接重建出高质量3D人体。大量实验表明,GaussianIP在视觉质量与训练效率上均优于现有方法,尤其在身份保持方面表现突出。代码已开源:https://github.com/silence-tang/GaussianIP。
原文摘要 · Abstract (English)
Text-guided 3D human generation has advanced with the development of efficient 3D representations and 2D-lifting methods like Score Distillation Sampling (SDS). However, current methods suffer from prolonged training times and often produce results that lack fine facial and garment details. In this paper, we propose GaussianIP, an effective two-stage framework for generating identity-preserving realistic 3D humans from text and image prompts. Our core insight is to leverage human-centric knowledge to facilitate the generation process. In stage 1, we propose a novel Adaptive Human Distillation Sampling (AHDS) method to rapidly generate a 3D human that maintains high identity consistency with the image prompt and achieves a realistic appearance. Compared to traditional SDS methods, AHDS better aligns with the human-centric generation process, enhancing visual quality with notably fewer training steps. To further improve the visual quality of the face and clothes regions, we design a View-Consistent Refinement (VCR) strategy in stage 2. Specifically, it produces detail-enhanced results of the multi-view images from stage 1 iteratively, ensuring the 3D texture consistency across views via mutual attention and distance-guided attention fusion. Then a polished version of the 3D human can be achieved by directly perform reconstruction with the refined images. Extensive experiments demonstrate that GaussianIP outperforms existing methods in both visual quality and training efficiency, particularly in generating identity-preserving results. Our code is available at: https://github.com/silence-tang/GaussianIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。