单图重建3D人物,解决几何模糊与数据少难题
SAT: Supervisor Regularization and Animation Augmentation for Two-process Monocular Texture 3D Human Reconstruction
- 分两阶段统一学习多种几何先验,避免视角不一致
- 引入监督特征正则化提升几何融合效果,减少面部失真
- 在线动画增强生成大量训练样本,缓解数据稀缺
单目纹理3D人体重建旨在仅通过一张正面人体RGB图像构建完整的3D数字化身。然而,单张2D图像固有的几何模糊性以及3D人体训练数据的匮乏是该领域发展的主要障碍。现有方法通常依赖先验几何估计网络(如SMPL模型和法线图)生成人体几何形态,但难以有效融合多模态信息,导致视点不一致问题,例如面部畸变。为此,本文提出一种两阶段3D人体重建框架SAT,能够以统一方式无缝学习多种先验几何,并输出高质量带纹理的3D化身。为促进几何学习,我们设计了监督特征正则化模块:通过结构相同的多视角网络提供中间特征作为训练监督,实现更优的几何先验融合。针对数据稀缺问题,进一步提出在线动画增强模块——通过一次前向传播的动画网络,从原始3D人体数据中在线生成大量样本用于模型训练。在两个基准测试上的大量实验表明,该方法优于当前最优技术。
原文摘要 · Abstract (English)
Monocular texture 3D human reconstruction aims to create a complete 3D digital avatar from just a single front-view human RGB image. However, the geometric ambiguity inherent in a single 2D image and the scarcity of 3D human training data are the main obstacles limiting progress in this field. To address these issues, current methods employ prior geometric estimation networks to derive various human geometric forms, such as the SMPL model and normal maps. However, they struggle to integrate these modalities effectively, leading to view inconsistencies, such as facial distortions. To this end, we propose a two-process 3D human reconstruction framework, SAT, which seamlessly learns various prior geometries in a unified manner and reconstructs high-quality textured 3D avatars as the final output. To further facilitate geometry learning, we introduce a Supervisor Feature Regularization module. By employing a multi-view network with the same structure to provide intermediate features as training supervision, these varied geometric priors can be better fused. To tackle data scarcity and further improve reconstruction quality, we also propose an Online Animation Augmentation module. By building a one-feed-forward animation network, we augment a massive number of samples from the original 3D human data online for model training. Extensive experiments on two benchmarks show the superiority of our approach compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。