文本生成可替换服装的分离式3D虚拟人,细节更真实。
DAGSM: Disentangled Avatar Generation with GS-enhanced Mesh
- 将人体与衣物分开展成带2D高斯的网格,提升纹理表现力。
- 支持换装和动画,生成图像质量优于现有方法。
- 适合需要精细控制服装的数字人应用开发。
文本驱动的虚拟人生成因其便捷性受到广泛关注。然而,现有方法通常将穿着衣物的人体建模为单一3D模型,限制了服装替换能力并降低用户控制性。为此,我们提出DAGSM,一种从文本提示生成分离式人体与衣物的新框架。具体地,我们将衣着人体各部分(如身体、上衣/下装)分别建模为一个GS增强网格(GSM),即在传统网格基础上附加2D高斯以更好处理复杂纹理(如毛织物、半透明衣物),并实现逼真布料动画。生成过程先构建裸体,再依次生成衣物,引入基于语义的算法实现人体-衣物及衣物间良好分离。为提升纹理质量,提出视图一致的纹理优化模块,包含跨视图注意力机制保证风格一致性,以及入射角加权去噪(IAW-DE)策略更新外观。大量实验表明,DAGSM生成高质量分离式虚拟人,支持服装替换与真实动画,视觉质量超越基线方法。
原文摘要 · Abstract (English)
Text-driven avatar generation has gained significant attention owing to its convenience. However, existing methods typically model the human body with all garments as a single 3D model, limiting its usability, such as clothing replacement, and reducing user control over the generation process. To overcome the limitations above, we propose DAGSM, a novel pipeline that generates disentangled human bodies and garments from the given text prompts. Specifically, we model each part (e.g., body, upper/lower clothes) of the clothed human as one GS-enhanced mesh (GSM), which is a traditional mesh attached with 2D Gaussians to better handle complicated textures (e.g., woolen, translucent clothes) and produce realistic cloth animations. During the generation, we first create the unclothed body, followed by a sequence of individual cloth generation based on the body, where we introduce a semantic-based algorithm to achieve better human-cloth and garment-garment separation. To improve texture quality, we propose a view-consistent texture refinement module, including a cross-view attention mechanism for texture style consistency and an incident-angle-weighted denoising (IAW-DE) strategy to update the appearance. Extensive experiments have demonstrated that DAGSM generates high-quality disentangled avatars, supports clothing replacement and realistic animation, and outperforms the baselines in visual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。